LLM Microservice with Instruction Calling

I describe the idea of implementing interaction with LLM through a concept of microservice with instruction calling. This works great for enterprise application use cases, such as data validation, workflor decisions.

 

Local LLM Instruction Processing with Sparrow

I explain how to execute instructions with a payload using a local LLM. This is useful when you want to process your data with an LLM and provide contextual instructions, specifying the desired outcome of what needs to be achieved. 

 

Vision LLM on Mac Mini M4 Pro: Real-World MLX Performance

I discuss the real-world MLX performance of Sparrow for structured data extraction with public access. The current Sparrow online instance runs on a Mac Mini M4 Pro with 64GB of memory. On average, it processes one page in 100 seconds. I explain why tokens-per-second measurements can be misleading when evaluating structured data extraction. 

 

Running Vision Models on Apple Silicon with MLX-VLM

I show and explain how to run Qwen and Mistral vision models on Apple Silicon with MLX-VLM. I share technical tips about how to run both models and show how to pass query prompt.

 

Dashboard with Gradio Python

This video showcases the Sparrow dashboard, where you can view statistics on document data extraction events processed by Sparrow. This elegant dashboard is built with Python using Gradio, a server-side web UI framework.

 

Extract Structured Data from Documents with Sparrow (Free Tier Available)

I built Sparrow for document data extraction 🚀 It's fully open-source and runs locally on your machine You can extract structured data from any document using powerful Mistral 24B 8bit and Qwen 2.5 72B 4bit models It's free to try with no registration (3 calls per 6 hours, max 3-page documents) and doesn't send your documents to third parties 

 

Oracle DB 23ai Free Connection Pool in Python

I describe how to connect to Oracle DB from Python. I explain why DB connection pool is important for better performance. Connection is done through thin oracledb mode, without installing Oracle Client.