Donut ML Model Fine-Tuning with Hugging Face API

I explain how Donut ML model can be fine-tuned on your own dataset by following different approaches. Either with PyTorch Lighting or Hugging Face Trainer API. I explain the pros and cons of both and what works best for me.

 

Document Data Extraction - Data Mapping for Donut Model Fine-Tuning Dataset (Document AI)

I explain the current status of my work related to dataset preparation for ML Donut model fine-tuning. I plan to use this model to run data extraction tasks from invoice documents. I share hints about data mapping and how to structure data to achieve better fine-tuning results.

 

Preparing Dataset for Donut Fine-Tuning (part 3, Document AI)

In this episode, I explain redesigned Sparrow UI for data annotation. Sparrow UI is improved with Streamlit Grid component (aggrid). I show how to group related fields generated by OCR into a single entity and map it with the label. I will briefly review the code and discuss how you can set up a grid component in Streamlit - a convenient and helpful UI element.

 

Preparing Dataset for Donut Fine-Tuning (part 2, Document AI)

I explain how to group OCR results into a single entity using Sparrow annotation tool. This is useful for such fields as an address, item description - when field text is based on multiple words.

 

How To Fine-tune Donut Model

Donut is an awesome Document AI model to extract data from docs. I share my experiences in fine-tuning the model, with CORD dataset, based on example from Transformers Tutorials.

 

Donut 🍩 - ChatGPT for Document AI

Donut - OCR-free Document Understanding Transformer. This ML model can process documents (images, scans) and return JSON structured info about the content. It works for different use cases: form understanding, visual question answering about the document, document image classification.