Ollama and MLX-VLM Accuracy Review (Qwen3-VL and Mistral Small 3.2)

I was running detail tests to compare accuracy for the same models (Qwen3-VL and Mistral Small 3.2) running on Ollama and MLX-VLM (recent 0.3.7 version). MLX-VLM runs faster, but with lower accuracy. The same is valid across different models.

 

Comparing Qwen3-VL AI Models for OCR Task

I'm comparing the Qwen3-VL 8B BF16 and Qwen3-VL 30B Q8 models for OCR and structured data extraction tasks. Based on my findings, the quantized 30B model runs faster and with better accuracy than the 8B BF16 model, despite using more memory. 

 

Qwen3-VL Accuracy Differences on Ollama vs MLX

I run couple of tests with structured data extraction using newest Qwen3-VL model on Mac Mini M4 Pro with 64GB. I discovered the same Qwen3-VL model with the same level of quantantization performs differently on Ollama vs. MLX. It seems model conversion step is crucial and we must evaluate model performance on different platforms before going to production. 

 

Qwen3-VL New Models Comparison and Performance on Mac Mini M4

I run and compare newest Qwen3-VL models in Sparrow. Qwen3-VL models run fast and provide good accuracy. 

 

Ollama Support in Sparrow and Update to Latest MLX

I explain whats new in Sparrow and what was updated in the recent version.

 

Ollama vs MLX Inference Speed on Mac Mini M4 Pro 64GB

MLX runs faster on first inference, but thanks to model caching or other optimizations by Ollama, second and next inference runs faster on Ollama. 

 

Advanced Structured Data Processing in Sparrow

I added instruction and validation functionality into Sparrow. This allows to process business logic with document data directly through Sparrow query. For example, it allows to check if given fields are present in the document.