Monday, December 23, 2024

Stateless MLX Inference with FastAPI in Sparrow

›
I show how to run inference with MLX in stateless mode, when loaded model is released after inference completes. This is useful when inferen...
Tuesday, December 17, 2024

Streamlined Table Data Extraction with Sparrow | Table Transformer, Qwen2 VL, MLX, & Mac Mini M4 Pro

›
Learn how to streamline table data extraction with Sparrow, Table Transformer, Qwen2 VL, and MLX on the Mac Mini M4 Pro. Simplify your workf...
Monday, December 9, 2024

Structured Output from Multipage PDF with Sparrow (Qwen2 Vision LLM and MLX)

›
I explain how multipage PDFs are handled in Sparrow to extract structured data in a single call.   
Tuesday, December 3, 2024

Sparrow Apple MLX Backend on Mac Mini M4 (Qwen2 72B 4bit)

›
I show how I’m running the Qwen2 72B 4bit model locally on a Mac Mini M4 for Sparrow’s backend. MLX (and MLX-VLM) is the main platform I’m u...
Monday, November 25, 2024

Batch Inference with Qwen2 Vision LLM (Sparrow)

›
I'm explaining several hints how to optimize Qwen2 Visual LLM performance for batch processing.   
Sunday, November 17, 2024

Visual LLM Structured Output Validation with Sparrow

›
I explain how Sparrow validates the structured output of visual LLMs to ensure it complies with the JSON schema provided in the query. This ...
Sunday, November 10, 2024

Extracting Financial Market Stock Data from Images with Vision LLM

›
In this video, I demonstrate how to extract financial market stock data from images using the powerful Vision LLM Qwen2, all within a Gradio...