Skip to content

Document and Multimodal AI

Read receipts, scans, photos, tables and policy documents with code and AI, and show exactly where every value came from. You build Larkfield's document pipeline, compare the PDF text layer, OCR and a vision model on the same pages, and send only the uncertain values to a person.

Read every lesson as a guest, or sign in to enroll and track your progress.

What you'll learn

By the end of this course, you will be able to:

  • Choose extraction methods based on document characteristics.
  • Preserve layout and source provenance.
  • Validate structured fields and route uncertain cases.
  • Evaluate accuracy, cost and failure modes across formats.

About this course

Larkfield’s help desk gets documents with its tickets: receipts for warranty and refund claims, some as PDFs from the shop’s till, some as scans and some as phone photos, a few in French. The purchasing team gets supplier price lists and spreadsheets. Today an agent opens every file and types the order number, the date, the total and the items by hand. It is slow, and sometimes the same receipt is typed twice with two different totals.

Grace, the support lead, asks: “Can the system read the receipt, show us exactly where each value came from, and send us only the ones it is not sure about?” In this course, you build that system, doc-extract, one module at a time.

First you look inside the files. A PDF from the till already holds its text; a scan is only a picture of text; a photo is a picture taken at an angle, in poor light. You route each file to the way of reading that fits it, and you compare the PDF text layer, OCR and a vision model on the same pages: what each one gets right, what it gets wrong, and what it costs per page.

Then you keep what makes a value trustworthy: its source. Every value carries its file, page and box on the page, or its sheet and cell, so a person can check it in one click. You validate totals and dates, send uncertain documents to a review queue, record corrections without erasing the original, and measure the pipeline field by field. The documents are made by a program for this course, and every model answer in the lessons comes from a real recorded call.

Before you start

You completed Building Reliable Applications with LLM APIs and RAG, or you have the same skills. You do not need an account or money. The text layer and OCR run on your computer's CPU, the documents are made by the course's own generator, and every vision-model answer in the lessons was recorded, so the project replays it. A live call with your own key is optional.

Courses to take first: Building Reliable Applications with LLM APIs, RAG: Building AI That Uses Your Data.

Who this is for

People who have built an application on a model API with structured output and a RAG assistant over documents, as in Building Reliable Applications with LLM APIs and RAG, and who now must get reliable values out of PDFs, scans, photos and spreadsheets. Developers, data engineers and AI engineers who automate work with documents.

Lead or use AI, but do not build it?

You can study the ideas of this course without writing code. Read modules 1, 4 and 5 (about 5–7 hours), answer their knowledge checks, and get a free concepts certificate. You can come back for the practice later.

The paths for people who lead or use AI take this course this way. See the paths

How we made this course

Experienced AI practitioners designed this course. AI helped us draft the content; people chose the teaching approach and the examples, and reviewed every lesson. The outputs that you see are the actual results of running the code. How we make our courses

Inside the course

Module 2Prepare and extract content

3 lessons

Get the text out, the right way for each file. Stop bad files at the door, compare the PDF text layer, OCR and a vision model on the same pages, and prepare images without losing the original.

  1. Input handling50 min
  2. Text and OCR paths60 min
  3. Image preparation50 min

Module 4ConceptsUse multimodal models deliberately

3 lessons

Use a vision model where it helps, and only there. Design its input and schema, combine it with the rules that already work, and keep a missing value missing instead of a plausible guess.

  1. Input and prompt design55 min
  2. Combine methods55 min
  3. Handle visual uncertainty50 min

Final module · choose at least oneCase studies

2 case studies

Two end-to-end case studies on real public data. Read a public set of labelled receipts, and recover a multi-page table from a public report and check it against the published data. Complete at least one for the certificate; each one is a portfolio piece.

Complete at least one case study for the certificate. The others are optional. What each one covers

  1. Read real receipts150 min
  2. Extract a table from a public report120 min

After the case studiesFinal assignment

Use every module in one practice project. You collect evidence, and check it against the course outcomes.

  1. Final assignment180 min

Case studies

Realistic projects from start to finish, on real data, with every output explained. Each one makes a portfolio piece. Complete at least one for the certificate; the others are optional.

  1. Read real receipts2 hr 30 min

    Run the course's pipeline on a public set of labelled receipt images, and measure its field accuracy against the published labels.

  2. Extract a table from a public report2 hr

    Recover a multi-page table from a public-domain statistics report with its provenance, and check every cell against the agency's published data file.

Reviews

Loading the reviews…

    Took this course? Sign in to review it. Learners can write a review after they complete 25% of the lessons.

    Share this course