LLVM has support for short vector, Single Instruction - Multiple Data (SIMD) instructions which are common place in multiple well-established ISAs. But how well it fits exotic SIMD instruction sets and micro-architectures? In this talk, we provide a beginner's experience with the implementation of compiler support for a custom SIMD unit with unusual characteristics, for the acceleration of AI operations in extremely resource-constrained CPUs. We show that LLVM SIMD support is general enough and well documented, allowing even new LLVM developers to add compiler support for innovative hardware designs in very short time. Our SIMD unit has some unique characteristics: it reuses the integer register file, it follows a multi-stage/Very long Instruction Word (VLIW)-like design, it includes GPU-like features (eg. saturation, swizzling etc) and it is portable among various instruction sets. In our presentation, we will go over the details of our micro-architecture and how the LLVM infrastructure allowed us to overcome its challenges. We will also present the performance speedups (up to 10$\times$) we obtained on two CPU implementations of our SIMD unit on an FPGA and their compiler support implemented in two different LLVM backends, the RISC-V and the SPARC one.