Document AI: How To Convert Colab ML Notebook Into FastAPI App

I explain how I converted Donut ML model fine-tuning code implemented as Colab notebook into API running as FastAPI app. I share several hints how to simplify code refactoring efforts. 

 

Speeding Up FastAPI App with Background Tasks

FastAPI runs background tasks in a parallel thread, which prevents blocking app endpoints when a long task executes. I explain it in this video and show the benefit of running time-consuming operations in background tasks.

 

Optimizing FastAPI for Concurrent Users when Running Hugging Face ML Models

To serve multiple concurrent users accessing FastAPI endpoint running Hugging Face API, you must start the FastAPI app with several workers. It will ensure current user requests will not be blocked if another request is already running. I show and describe it in this video. 

 

Optimizing ML Model Loading Time Using LRU Cache in FastAPI

Are you facing challenges with the time it takes to load large ML models in your backend API? This video presents a practical solution: utilizing LRU cache with properly annotated functions. Implementing this approach will make your model cached in memory, eliminating the need for disk reads on subsequent calls. Enhance the efficiency and performance of your ML workflow by incorporating LRU cache techniques. Join us to learn more about this valuable strategy!