Understanding Mistral Small 3.2
Mistral AI has released an updated version of its open-source model, Mistral Small 3.2, building on the earlier 3.1 version. This new release aims to enhance specific functionalities without altering the overall architecture. The focus is on improving instruction-following, output reliability, and function calling capabilities. The update is particularly beneficial for businesses with limited computing resources, as it can operate on a single Nvidia A100/H100 GPU.
Key Enhancements
- Instruction-following accuracy increased from 82.75% to 84.78%.
- Performance on external datasets improved significantly, with Wildbench rising nearly 10 percentage points and Arena Hard more than doubling.
- The rate of infinite generations reduced from 2.11% to 1.29%, enhancing reliability for developers.
- Minor regressions in some benchmarks, such as MMLU, indicate a trade-off for targeted improvements.
The Bigger Picture
Mistral Small 3.2 reflects a commitment to refining AI models rather than overhauling them. This approach offers developers a more reliable tool for creating applications, especially in regulated markets like the EU. While it may not drastically change competitive dynamics, the improvements in instruction compliance and tool usage make it a valuable option for enterprises. However, those seeking significant performance boosts may still prefer the earlier version, Small 3.1, as a reference point.
