We present Baltoro, an MLIR-based AI compiler built on IREE for RISC-V CPUs and accelerated compute, focused on fast deployment of generative AI workloads at the edge. Baltoro provides first-class support for quantization, scalable vector code generation for RISC-V, and rapid enablement of custom and standard RISC-V extensions. We have validated compilation on real RISC-V vector silicon, with performance results on open-source LLMs and vision-language models.

This talk covers how Baltoro enables day-zero support for new model architectures via drop-in custom MLIR templates, and how its pipeline supports modern quantization formats (Q4_0/Q8_0, Q4_K, MXFP4/MXFP8) by extending IREE's data-tiling framework to handle quantized tensors and their scale/zero-point metadata. We'll show how Baltoro's lowering pipeline targets scalable vector architectures like RVV for vector-length-agnostic code generation, and how its modular design enables rapid support for extensions like SpacemiT's IME and Andes Vector Dot Product extension. We conclude with Baltoro's future roadmap.