Mistral OCR 4 + Sparrow: Document to JSON

Follow-up to the Mistral OCR + Sparrow integration video. Mistral released OCR 4 — the latest model with improved accuracy, native bounding box extraction, and structural block labels. One model string change in Sparrow to pick it up.

In this video, extracting a long financial statement table into structured JSON using Mistral OCR 4 + Mistral Small 4.

Sparrow is open source and local-first by design — documents never leave your infrastructure unless you choose the cloud backend.

Mistral OCR + Sparrow: Document to JSON

Integrated Mistral OCR as a new cloud inference backend into Sparrow, an open-source document extraction platform. This gives Sparrow a full cloud option alongside its existing local backends (MLX, vLLM), so users without GPU infrastructure can still run enterprise-grade document extraction.

Pipeline: Mistral OCR converts the document to structured HTML, then Mistral Small extracts and transforms the data into JSON based on a defined schema with field-level hints.

In this video, extracting a bonds portfolio table with hint-driven rules:

• Instrument name normalization (extracting issuer brand from full fund names)
• European number formatting (period as thousands separator, comma as decimal)
• Percentage formatting with sign preservation
• Derived risk classification computed from profit/loss percentage

Same Sparrow API, same schema and hint format as local backends — just switch the backend flag to run on Mistral Cloud instead of MLX or vLLM.

Sparrow is open source and local-first by design — documents never leave your infrastructure unless you choose the cloud backend.

⭐ GitHub: github.com/katanaml/sparrow
🌐 Live demo: sparrow.katanaml.io 

 

Fast Large Table Extraction: Sparrow + dots.ocr to JSON

Sparrow provides table processing mode. It is optimized to handle large tables, it comes with separate template script (new templates can be easily added) to process dots.ocr markdown output into structure JSON with field mapping.

 

Local OCR Comparison: dots.ocr More Accurate, DeepSeek-OCR 2 Faster (Sparrow + MLX)

I run local tests with Sparrow to compare DeepSeek OCR2 and dots.ocr (by RedNote), both run on MLX-VLM in FP16 precision. Dots.ocr consistently beats DeepSeek OCR2 in accuracy, but DeepSeek OCR2 deliveres much better inference performance.

 

GLM-OCR vs DeepSeek OCR 2: Which One Wins at Markdown Extraction?

I compare two OCR models using real test cases: GLM OCR and DeepSeek OCR2. Both are evaluated on their ability to extract document content and convert it into well-structured Markdown. I demonstrate which model performs better and which one is faster. 

 

Get Vision LLMs to Follow Your Rules: Prompt-Guided JSON Formatting

JSON query helps to fetch structured output with Vision LLM and extract document data. I describe how to improve such output with additional rules provided through LLM prompt. In this video I share example of number formatting, based on applied rule LLM will output values in requested format. 

 

Vision LLM Output Control for Better OCR with Prompt Hints

I explain my approach to enforce better OCR output from vision LLMs with prompt hints. This allows to set rules for output data validation and formatting.

 

DeepSeek OCR Markdown Processing in Sparrow for Large Tables

I describe new functionality in Sparrow, where DeepSeek OCR is used to extract text data in markdown format and in the next step instruction LLM inference is utilized to convert data into structured JSON format. This approach helps to improve large table processing and avoid vision LLM hallucinations. 

 

DeepSeek OCR Review

I'm testing structured data extraction with DeepSeek OCR. It works well and gives good data accuracy and performance to disrupt traditional cloud based document processing solutions.

 

Structured Data Retrieval with Sparrow using OCR and Vision LLM [Improved Accuracy]

I explain improvements I'm adding into Sparrow to achieve better accuracy for structured data. I'm using a method, where I run OCR step first, then construct advanced prompt with injected OCR data. This prompt is sent along with image to Vision LLM for structured data retrieval. All this happens as part of a single pipeline.

 

Comparing Qwen3-VL AI Models for OCR Task

I'm comparing the Qwen3-VL 8B BF16 and Qwen3-VL 30B Q8 models for OCR and structured data extraction tasks. Based on my findings, the quantized 30B model runs faster and with better accuracy than the 8B BF16 model, despite using more memory. 

 

Qwen3-VL New Models Comparison and Performance on Mac Mini M4

I run and compare newest Qwen3-VL models in Sparrow. Qwen3-VL models run fast and provide good accuracy. 

 

Ollama Support in Sparrow and Update to Latest MLX

I explain whats new in Sparrow and what was updated in the recent version.

 

Structured Data Query with Sparrow AI Agent

Sparrow comes with option to extract stuctured data with query. In this video I explain how you can define such query to fetch array and field data.

 

Vision LLM with MLX: Extracting Electric Meter Data in Production

In this video, I share my experience using the MLX backend to run Vision LLM (with MLX-VLM) for structured data extraction in a production environment. See how I used Sparrow to accurately read electric meter data and learn practical tips for deploying similar solutions. 

 

How to Extract Financial Statement Data with Sparrow & Vision LLM

Extract financial statement data with Sparrow and Vision LLM in this quick tutorial! Sparrow auto-detects tables, builds clear grids, and uses OCR for accurate Vision LLM results, preventing errors. Runs locally with no cloud dependency, making it great for private financial documents. Perfect for anyone handling sensitive financial data. 

 

Boost Vision LLM Accuracy with OCR Text Integration

I show an interesting approach where I send both an image and OCR text to a Vision LLM. The prompt is constructed to instruct the Vision LLM to prioritize the OCR text. This allows the use of a Vision LLM for structured output construction while relying on external OCR text, giving you more control over the results.

 

Solving Vision LLM Number Formatting Issues Using PaddleOCR and Sparrow

Discover how to fix number formatting errors in vision LLMs like Mistral! In this video, I show how Mistral misreads "56,000" as "56000" and how combining PaddleOCR’s text extraction with Sparrow’s spatial data processing solves this hallucination issue.

 

PaddleOCR 3.0: Supercharge Your AI

I upgraded to PaddleOCR 3.0 and explain the new PaddleOCR API integration. My goal is to integrate OCR result output with Vision LLM processing to enhance large-scale, structured table data output. 

 

Box Annotations in Sparrow for Structured Data Extraction

Check out my video on Box Annotations in Sparrow for Structured Data Extraction! I’ll show you how the Qwen2.5 vision model pulls bounding box annotations from images based on what you need. Plus, create simple descriptions and confidence score boxes.