Set up the extraction project

What you will be able to do

Get the course’s project, doc-extract, onto your computer. Make the course’s 65 documents, run the tests, and open one receipt as a PDF, a scan and a phone photo. You also learn why the course makes its own documents: every right answer is known before the first line of code reads them.

Before you startWhat to know and to have readyAbout 45 minutes

What you need to know

• A schema for a model’s answer, and why a missing value is null, from Define a schema in Building Reliable Applications with LLM APIs.
• How a PDF stores its text with positions on the page, from Parse and normalize in RAG: Building AI That Uses Your Data.
• A terminal, a virtual environment and pip, from Python for Practical AI Engineering.

What you need on your computer

• Python 3.12 or newer, VS Code (or any text editor) and a terminal.
• About 450 MB of free disk space: about 430 MB for the virtual environment and 17 MB for the documents.
• An internet connection for the install. After that, everything runs offline.
• No account, no key and no money. The text layer and OCR run on your computer’s CPU. The vision model’s answers were recorded by the course team, and the project replays them.

Tested with Python 3.12 on macOS (Apple silicon). The Windows (PowerShell) and Linux commands are written, not tested.

A shoebox of receipts

Think of the end of the year, when you fill in an expense report. You empty a shoebox on the table. One receipt is a PDF that a shop emailed to you. One is a scan that your bank made. One is a photo that you took with your phone, on the kitchen table, a little crooked. Two are in French, because you were in Montréal for a week.

You type the date and the total of each one into a spreadsheet. It is slow, but it works, for one reason: you hold the paper. If a number looks strange, you look at the receipt again. You know where every number came from.

Now imagine that a program types the numbers for you. You get a spreadsheet back, very fast. Is it right? To answer, you need two things. You need the right answer for each receipt, to compare. And you need to know where on the receipt the program found each number, so that you can look at it.

The problem: values without a source, and no right answers

Larkfield’s help desk gets documents with its tickets. Customers attach receipts to warranty and refund claims: PDFs from Larkfield’s till, scans, phone photos, some in French. The purchasing team gets price lists and spreadsheets from suppliers. Today an agent opens every file and types the order number, the date, the total and the items by hand.

This is slow. It also fails in a quiet way. Last month the same receipt arrived on two tickets, and two agents typed it. One typed a total of 211.68, the other 211.86. Both values look right. Nobody can say which one is right without opening the file again, because the spreadsheet does not say where the number came from.

Grace leads the support team. She asks: “Can the system read the receipt, show us exactly where each value came from, and send us only the ones it is not sure about?”

Mei is the data engineer who owns the files that come in. Before she writes any code, she sees a second problem. How will the team know if the system is right? Real receipts hold names, addresses and card numbers, so the team cannot share them freely. And nobody has written down the right answer for each one. A test without right answers can only say “it ran”, not “it was right”.

The idea: documents with known true values

Mei’s answer is to make the documents. A program writes the true values first: the shop, the date, the items, the tax and the total. Then it draws a receipt that prints those values. Then it makes harder copies, as a scanner or a phone would. A copy can be at low resolution, crooked, faded, in shadow, or with a part cut off.

Because the program wrote each value before it drew the receipt, the right answer is known exactly. It is called the ground truth: the values that are true for each document, against which you compare what a reader gives. The name comes from map making. A map maker checks a picture taken from the air against measurements made on the ground.

The generator writes the truth first and draws the documents from it. So the right answer for every document is known before any reader runs.

Text description of the diagram

The flow starts with the true values of a receipt: the shop, the date, the items, the tax and the total. The generator picks them with a fixed random seed, 13, so every run picks the same values. The values go two ways. They are written to truth.jsonl, one line per document. And they are drawn as a receipt: a PDF that holds its text. From some PDFs, the generator makes a scan, which is a picture of the page, at low resolution, crooked or faded. From others, it makes a phone photo, with an angle, a shadow, blur or a part cut off. All 65 files go to data/documents/. Then --check compares the bytes of every file with the list of checksums that the course ships, SHA256SUMS. If they all match, you have the course’s exact files. If some differ, you copy the files from the labs repository.

A checksum is a short fingerprint of a file’s bytes. If one byte changes, the fingerprint changes. The project ships the fingerprints of the course’s files. So --check can tell you if your computer made exactly the same documents as the course team’s computer. If it did, your numbers match the lessons.

A tiny example: one receipt and its truth

Here is receipt D011, a PDF from Larkfield’s till in Vancouver. It is the first picture further down this page. The receipt prints a pressure washer at 189.00, two taxes and a total. The generator wrote this truth for it:

Field	True value	What the receipt prints
vendor	Larkfield	LARKFIELD
order_number	LK-103119	Order: LK-103119
date	2026-08-22	Date: 2026-08-22
subtotal	189.00	Subtotal 189.00
tax	22.68	GST 5% 9.45 and PST 7% 13.23
total	211.68	TOTAL CAD 211.68
shipping	(not on the document)	nothing: a till receipt has no shipping line

Check the sum yourself: 189.00 + 9.45 + 13.23 = 211.68. The tax is 22.68, because the receipt prints two tax lines and the truth adds them. The shipping is “not on the document”, and the truth says so. It does not say 0.00, because the receipt does not print 0.00.

Now the agent who typed 211.86 has a clear answer: the truth says 211.68. In the real help desk, there is no truth file. Instead, the system will show the agent where on the receipt it read 211.68. That is the other half of Grace’s question, and this course builds it.

Larkfield’s document set

The course’s generator makes 65 documents. Every document in this course is made by the course’s generator; no real customer or shop is in it.

Kind	How many	What they are
Receipts	53	Larkfield’s till receipts and online orders, as PDFs, scans and phone photos; 10 in French; 6 from another shop, Maple & Pine Hardware, a Larkfield partner store
Policies	2	the returns policy in two columns; the Quebec legal warranty, in French
Price list	1	a supplier’s price list whose table continues on page 2
Spreadsheet	1	the same price list as an Excel file
Warranty claim	1	page 1 a form, page 2 a scanned receipt
Problem files	7	a copy, a second scan, a broken download, a locked PDF, a file too large, a photo with a .pdf name, a text file

Each document has an ID in arrival order, from D001 to D065. The lessons use these IDs. The team uses them too.

Mei owns this set and the files that come in. Grace and her agents need the fields. Camille, an agent in Montréal, brings the French receipts. Priya, the data scientist, checks the true values and later measures the pipeline. Amira, a developer, connects the pipeline to the ticket system. Omar, the product manager, asks what each page costs. Tomás, an AI engineer, wants to send everything to a vision model. Sam is new to the team and asks the questions a beginner asks.

Set up the project

You copy the snapshot start of the project into a new folder, make a virtual environment, make the documents and run the tests. A snapshot is a copy of the project at one point of the course. If your project breaks later, you copy the snapshot of the last module that you finished, and you continue.

1. Open a terminal and go to your projects folder, for example ~/projects. If you have Git, clone the labs repository once. If you have the clone already, run git -C nextia-learning-labs pull instead.

   cd ~/projects
   git clone https://github.com/Nextia-Intelligence/nextia-learning-labs.git

   Without Git, download the ZIP file of the repository, and unzip it in ~/projects. Its folder is then nextia-learning-labs-main. Use that name in the next steps.

2. Copy the snapshot start into a new folder doc-extract, and go into it:

   macOS / Linux

   cp -R nextia-learning-labs/C13/snapshots/start doc-extract
   cd doc-extract

   Windows (PowerShell)

   Copy-Item -Recurse nextia-learning-labs\C13\snapshots\start doc-extract
   cd doc-extract

3. Open the folder doc-extract in VS Code. All the commands in this course run in this folder.

4. Make the virtual environment and activate it:

   macOS / Linux

   python3 -m venv .venv
   source .venv/bin/activate

   Windows (PowerShell)

   py -m venv .venv
   .venv\Scripts\Activate.ps1

   The prompt now starts with (.venv).

5. Install the packages. This step downloads several hundred megabytes, so it can take a few minutes on a slow connection.

   python -m pip install -r requirements-dev.txt

   This is the only install of the course. The OCR engine is in the list already, although Module 2 uses it first. Its models come inside the package, so OCR needs no download later.

6. Make the documents and compare them with the course’s checksums:

   python -m doc_extract.generate --check

   Wrote 65 documents to data/documents (17.3 MB) and data/truth.jsonl
     receipt 53, policy 2, price_list 1, spreadsheet 1, claim 1, bad 7
   All 65 files match the course's checksums.

   It takes about 10 seconds. “bad” is the generator’s name for the seven problem files.

7. Run the tests:

   python -m pytest -q

   .........                                                                [100%]
   9 passed in 5.61s

   Your time can differ. The tests check the set: 65 documents of every kind, and the same bytes from the same seed. They also check that the true totals add up, except on two receipts that print a wrong total on purpose.

You now have a working project, the 65 documents and their truth. The project does not read any document yet. It only knows what it made.

Look at one receipt three ways

Start with the list of documents:

python -m doc_extract.documents

It prints one line per document. Here are four of the 65:

ID    file                               kind        source      lang pages
D011  receipt-lk-103119.pdf              receipt     digital_pdf en   1
D020  scan0002.jpg                       receipt     scan_image  en   1
D036  IMG_4817.jpg                       receipt     photo       fr   1
D058  warranty-claim-wc-2026-0418.pdf    claim       mixed_pdf   en   2

The source column says how the generator made each file. A real help desk does not get this column: a customer sends a file called scan0002.jpg or IMG_4817.jpg, and nothing more. In the next lesson, you find the same answer by looking inside the file.

Now look at one document and its truth. D036 is Camille’s kind of receipt: a phone photo of a till receipt from Montréal.

python -m doc_extract.documents D036

D036  IMG_4817.jpg
  kind receipt, source photo, language fr, pages 1, 186,813 bytes
  how it was made: {"tilt": 0.07, "angle": -5.0, "layout": "till"}
  vendor          Larkfield
  order_number    LK-706142
  date            2026-08-09
  currency        CAD
  subtotal        880.00
  shipping        (not on the document)
  tax             131.78
  total           1011.78
  payment_method  Cash
    item: 1 x Bac à compost @ 64.00 = 64.00
    item: 1 x Nettoyeur haute pression PW-2200 @ 189.00 = 189.00
    item: 1 x Barbecue au gaz Ember GR-310 @ 449.00 = 449.00
    item: 2 x Jardinière en cèdre @ 89.00 = 178.00

The line “how it was made” says that the photo was taken at an angle of −5 degrees, with some tilt. The truth is written in one form for every receipt: dates as 2026-08-09, money with a point and two decimals. The photo prints them in French: 09/08/2026 and 1 011,78 $. Turning printed text into the truth’s form is the work of Define the extraction contract.

Draw a picture of three receipts: the PDF D011, the scan D020 and the photo D036.

python -m doc_extract.show D011
python -m doc_extract.show D020
python -m doc_extract.show D036

D011  receipt-lk-103119.pdf  (2,249 bytes, digital_pdf)
  page 1: 315 x 471 pixels at 100 dpi -> out/show/D011-p1.png
D020  scan0002.jpg  (224,481 bytes, scan_image)
  page 1: 1275 x 1650 pixels at 100 dpi -> out/show/D020-p1.png
D036  IMG_4817.jpg  (186,813 bytes, photo)
  page 1: 1200 x 1600 pixels at 100 dpi -> out/show/D036-p1.png

Open the three pictures in out/show/ in VS Code. They look like this:

Three receipts from the set: a PDF from the till (D011), a scan of an online order (D020) and a phone photo of a French till receipt (D036). To a person, all three are easy to read.

Text description of the diagram

On the left, D011: a narrow receipt from Larkfield’s till in Vancouver. The text is sharp and black on white. It prints a pressure washer at 189.00, GST of 9.45 and PST of 13.23. The total is CAD 211.68, paid in cash. In the middle, D020: an online order confirmation on a letter-size page, scanned. The page is a little grey and turned by 1.5 degrees. It lists three items and a total of 416.80. On the right, D036: a phone photo of a French till receipt from Montréal. The receipt lies on a wooden table and is turned by about 5 degrees. It lists four items in French, a sous-total of 880,00, two taxes, TPS and TVQ, and a total of 1 011,78 dollars.

Sam looks at the three pictures. “They are all easy to read. Why does a program need three different ways?” Look at the file sizes in the output. The PDF D011 has 2,249 bytes. The scan D020 has 224,481 bytes: a hundred times more, for about the same amount of text. Those numbers are a hint about what is inside each file. The next lesson opens the files and shows it.

What is in the project

File or folder	What it is
doc_extract/generate.py	the generator: the 65 documents and their truth
doc_extract/documents.py	lists the documents and shows the truth of one
doc_extract/show.py	draws each page of a document as a picture in out/show/
doc_extract/files.py	opens a page as a picture; decides “is it a PDF?” by the first bytes
data/SHA256SUMS	the checksums of the course’s 65 files
data/dataset.md	the dataset card: how the set was made, its licence (CC0) and its limits
tests/	9 tests of the set
README.md	every command, and the reset path

The folder out/ holds everything that the commands write. You can delete it at any time; the commands make it again.

When to use a generated document set

Use one when you need exact right answers and you cannot share real documents. It is also the best way to test one problem at a time. In this set, D025 is a scan turned by 90 degrees and nothing else, so a failure on D025 has one cause.

Do not trust it to tell you how often each problem happens. Real receipts vary much more: other fonts, folded paper, stamps, handwriting, glare. A reader that works on these 65 documents can still fail on real ones. That is why the course ends with a case study on a public set of real receipt photos with their own labels.

The cost is the generator itself: about 1,200 lines of code that somebody must write and check. And somebody must still read the documents. The course author opened every kind of document and compared the printed values with the truth.

Common mistakes

Common mistake

Some files differ from the course's checksums

Symptom: generate --check ends with lines like these (the file names are an example), and stops with an error:

2 files differ from the course's files: IMG_4810.jpg, IMG_4817.jpg
Your numbers can differ a little. To use the course's files, copy them from the labs repository.

Cause: another operating system, or another version of a picture library, can draw a picture a few bytes differently. The documents are still good receipts. But a later lesson’s numbers can then differ a little from yours.

Fix: copy the course’s files. Copy the folder documents and the file truth.jsonl from nextia-learning-labs/C13/data/ into your project’s data/ folder, and replace the files that are there. Then run python -m pytest -q again.

Common mistake

No module named pytest, or truth.jsonl does not exist

Symptom: a command prints one of these lines and stops. The paths are shortened here.

…/bin/python3.12: No module named pytest

…/doc-extract/data/truth.jsonl does not exist. Make the documents first: python -m doc_extract.generate

Cause: “No module named pytest” means that the virtual environment is not active. The second message means that you have not made the documents yet, or that the terminal is in another folder.

Fix: go to the folder with cd ~/projects/doc-extract. Activate the environment: source .venv/bin/activate on macOS and Linux, .venv\Scripts\Activate.ps1 on Windows. Then run python -m doc_extract.generate --check.

Reset. If your project breaks, rename the folder, and do the steps above again with a new copy of start. To reset only the outputs, delete the folder out/. The snapshots README shows the same steps for every later snapshot.

Your turn: a photo with a part missing

Change one thing: look at a different photo. D045 is a phone photo of a till receipt, and the photo cuts off the bottom of the receipt. Before you run anything, predict: which fields does the truth mark “not on the document”? Then run python -m doc_extract.documents D045 and python -m doc_extract.show D045, and open the picture.

Show the answer

  subtotal        358.95
  shipping        (not on the document)
  tax             46.66
  total           (not on the document)
  payment_method  (not on the document)

The photo stops just after the tax line, so the total and the payment are not on it. The receipt printed them, but the photo lost them. The truth writes them as “not on the document”, because no reader can see them in this file. Shipping is not on the document either, for another reason: a till receipt never prints a shipping line.

A reader that “finds” a total for D045 invents it. You could compute 358.95 + 46.66 = 405.61, and that is probably the total. But it is a guess, not a value read from the page. The extraction contract, in the last lesson of this module, says how to write such a field.

Knowledge check

Check your understanding

Why does the course make its own documents instead of using a folder of real receipts? Give two reasons, and one limit of this choice.

Show the answer

• The right answers are known exactly. The generator writes the true values first and draws the document from them, so every reader can be scored field by field. A folder of real receipts has no truth until somebody types it, and the typing can be wrong.
• No personal data. Real receipts hold names, addresses and card numbers. Generated ones can be shared, published (CC0) and used in tests.
• Also good: each problem appears alone (a turned scan, a cut-off photo), so a failure has one cause.

Limit: generated documents are less varied than real ones. A result on them says what a method does on these problems, not how often the problems happen. The real-receipt case study checks this.

Check your understanding

Predict: Tomás says 'the PDF D011 and the scan D020 are both receipts, so they are the same kind of file'. The PDF has 2,249 bytes; the scan has 224,481. What do the sizes suggest about what is inside each file?

Show the answer

The PDF is small because it stores the text itself: a few hundred characters with their fonts and positions. The scan is a hundred times larger because it stores a picture: about two million pixels, whatever is printed on them. So the PDF already holds its text, and a program can read it directly. The scan holds only dots of colour, and a program must recognise the letters in them first. The next lesson checks this by looking inside the files.

Check your understanding

Practical check: show that your project works. What do `generate --check` and the tests print, and what does `documents D036` give as the date and the total?

Show the answer

generate --check prints Wrote 65 documents to data/documents (17.3 MB) and data/truth.jsonl and All 65 files match the course's checksums. (or a list of files that differ, which you fix by copying the course’s files). The tests print 9 passed. documents D036 gives the date 2026-08-09 and the total 1011.78. The photo prints them as 09/08/2026 and 1 011,78 $: the truth always uses one form.

Case studies in this course

At the end of the course, you use what you learn on a realistic project, from the data to the recommendation. Complete at least one for the certificate; the others are optional.

• Read real receiptsRun the course's pipeline on a public set of labelled receipt images, and measure its field accuracy against the published labels.
• Extract a table from a public reportRecover a multi-page table from a public-domain statistics report with its provenance, and check every cell against the agency's published data file.

Recap

• Grace wants values with their source, and only the uncertain ones for review. To test a reader, you also need the right answer for every document.
• The course’s generator writes the ground truth first and draws each document from it: 65 documents, 17.3 MB, with truth.jsonl. --check compares their checksums with the course’s.
• In D011, the truth adds the two tax lines: 9.45 + 13.23 = 22.68. Then 189.00 + 22.68 = 211.68. A value that the document does not show is written “not on the document”, not 0.00.
• Your project works: python -m pytest -q prints 9 passed, and show draws any page in out/show/.
• A PDF, a scan and a photo can look the same to a person, but their sizes differ by a hundred times.

Next, you look inside each file and choose the right way to read it.

Next: Document types