Document types

What you will be able to do

Look inside each file before you read it. Find out what it really is: a PDF that holds its text, a picture of a page, a phone photo or a spreadsheet. Then choose the way to read it, page by page.

Before you startWhat to know and to have readyAbout 50 minutes

What you need to know

• The course’s document set and its truth, from Larkfield’s document set in Set up the extraction project.
• How a PDF stores text as characters with positions, from Parse and normalize in RAG: Building AI That Uses Your Data.

What you need on your computer

• Your folder doc-extract with its active virtual environment, and python -m pytest -q printing 9 passed.
• The labs folder nextia-learning-labs next to it. You copy one new file of the project from it.

Tested with Python 3.12 on macOS (Apple silicon). The Windows (PowerShell) and Linux commands are written, not tested.

A book and a photo of a book

Think of a book that you read on a tablet. You can search for a word, and the tablet jumps to it. You can copy a sentence and paste it into a message. The tablet knows every letter, because the book file holds the letters.

Now take a photo of a printed page with your phone. On the screen, it looks almost the same. But you cannot search it, and you cannot copy a sentence. The photo holds only coloured dots. To get the words, somebody, or a program, must look at the dots and recognise the letters, one by one.

The two look the same to you. Inside, they are two different things, and each one needs a different way to read it.

The problem: the name and the look do not say what a file is

Larkfield’s documents arrive as attachments. Their names say little: scan0001.pdf, IMG_4817.jpg, receipt-scan.pdf. The previous lesson showed that two receipts can look alike and still differ by a hundred times in size.

Tomás has a simple plan. “Every PDF is just an image. Send every page to a vision model, and it reads everything.” A vision model is a language model that also accepts pictures. It is a kind of multimodal model: a model that works with more than one kind of input, such as text and images. The plan sounds easy. Look at what it costs and what it gives on a PDF from the till.

• Time. Reading the text of a till PDF directly takes about 5.4 milliseconds per page on the course team’s computer. The vision model took 7.35 seconds per page (median). That is more than a thousand times slower.
• Money. The vision model costs about US$0.0113 per page. Reading the PDF’s text costs nothing.
• Errors. On receipt D009, a French PDF receipt, the model read the shop’s name as “LARKEFIELD”. The PDF holds the letters “LARKFIELD”, exactly. The model added a letter that was never there.

Tomás’s plan treats a file that already holds its text as a picture of text. It pays more, waits longer and gets a value wrong that the file had right.

The opposite mistake fails too. Mei tries a PDF text reader on every file whose name ends in .pdf. On the scanned receipt D019, it returns no text at all: zero lines. And on receipt-scan.pdf (D064), it stops with an error:

pdfplumber.utils.exceptions.PdfminerException: No /Root object! - Is this really a PDF?

D064 is not a PDF. It is a JPEG photo that somebody saved with a .pdf name. The name lied, and the reader trusted it.

The idea: look inside, then route

Mei’s rule is: never decide by the name or the look; decide by what is inside the file. She asks three questions.

1. What format is it really? The first bytes of a file say this. Most formats start with a fixed signature, called a magic number. A PDF starts with %PDF-. A JPEG starts with the bytes FF D8 FF. An Excel file is a ZIP archive, so it starts with PK, and it has a folder xl/ inside.
2. For a PDF: does each page hold its text? A PDF made by a program stores its characters, with their font and position. This is the text layer. A scanned PDF stores only a picture of the page, with no characters. A page with 30 or more characters in its text layer is read as text. A page with fewer is a picture.
3. For a picture: is it a scan or a phone photo? A scan is grey text on a white page. A phone photo shows a background around the paper: a table, a hand, a shadow. They need different preparation, in Module 2.

The answer is a route: the way to read the file, or each page of it. There are four routes:

• text: read the text layer directly.
• ocr: the page is a picture, so recognise the letters in it. OCR, optical character recognition, is a program that finds text in a picture and returns each piece of text with its position.
• sheet: a spreadsheet; read its cells.
• stop: the file cannot be read as it is; say why.

Choosing a route for each input is called routing. A post office routes each letter by its address, not by the colour of the envelope.

A tiny example: the first bytes of five files

Open a Python prompt in your project folder (type python), and read the first 8 bytes of five files:

>>> from pathlib import Path
>>> for name in ["receipt-lk-103119.pdf", "scan0002.jpg", "aquaflow-price-list.xlsx", "receipt-scan.pdf", "notes.txt"]:
...     print(f"{name:<26}", Path("data/documents", name).read_bytes()[:8])
...
receipt-lk-103119.pdf      b'%PDF-1.3'
scan0002.jpg               b'\xff\xd8\xff\xe0\x00\x10JF'
aquaflow-price-list.xlsx   b'PK\x03\x04\x14\x00\x00\x00'
receipt-scan.pdf           b'\xff\xd8\xff\xe0\x00\x10JF'
notes.txt                  b'Customer'

Type exit() to leave the prompt. Read the five lines one at a time:

• receipt-lk-103119.pdf starts with %PDF-1.3: a real PDF, version 1.3.
• scan0002.jpg starts with \xff\xd8\xff: a JPEG picture. The letters JF are the start of “JFIF”, a common JPEG form.
• aquaflow-price-list.xlsx starts with PK: a ZIP archive, as every .xlsx file is.
• receipt-scan.pdf starts with \xff\xd8\xff, exactly like the JPEG. Its name says PDF; its bytes say JPEG. The bytes win.
• notes.txt starts with the word Customer: plain text, no known signature. No route reads it.

The first bytes answer question 1. For question 2, look inside two PDFs. The picture below draws what a PDF reader finds on two receipt pages. Each line of the text layer has a blue box; each picture has an orange box.

What a PDF reader finds inside two receipts. The till receipt D011 holds 467 characters in 29 lines, each with its box (blue). The scan D019 holds one picture that covers the page (orange), and no characters at all.

Text description of the diagram

On the left, the till receipt D011. A PDF reader finds 29 lines of text in it, and the picture draws each line’s box in blue. The lines hold the shop’s name, the address, the date, the numbers, the item, the taxes, the total and the payment. There is no picture on the page. On the right, the scanned receipt D019. The page looks like a small receipt on a grey page. But the PDF reader finds one picture, in orange, that covers the whole page, and no characters. To the reader, the shop’s name and the total on this page are dots, not letters.

To you, both pages show a receipt. To a program, D011 is text with positions, and D019 is one picture. A text reader gets everything from D011 and nothing from D019. OCR can read D019, but it would be a waste on D011: the PDF gives the exact text for free. The picture also shows bounding boxes. A bounding box is the rectangle around a piece of text, given by its left, top, right and bottom edges. The next lesson uses these boxes.

Route every file

Mei writes the three questions as code: route.py. The diagram shows its decisions.

Document-type routing. The first bytes give the format; a PDF is then routed page by page, and a picture is sorted into scan or photo. The numbers are the course's 65 files.

Text description of the diagram

A file arrives with any name. The router reads its first bytes and follows one of four branches. If the file starts with %PDF-, it is a PDF, and each page gets its own question: does the page hold 30 or more characters in its text layer? If yes, the page’s route is text. In the set, 25 PDFs have a text layer on every page. If no, the page is a picture of a page, and its route is ocr. If the file starts with the signature of a JPEG, a PNG or a TIFF, it is a picture. The router then asks if there is a background around the paper: yes means a phone photo, no means a scan. Both go to ocr: 20 scans and 15 photos in the set. If the file is a ZIP archive with an xl/ folder inside, it is a spreadsheet, and the route is sheet. Anything else, or a file that cannot be opened, gets the route stop, with the reason: 3 files of the set. A note: a mixed PDF is routed page by page. The warranty claim D058 has page 1 on the text route and page 2 on the ocr route.

Guided practice: route the course’s documents

1. In ~/projects/doc-extract, with the virtual environment active, copy the router and its tests from the snapshot end-of-m01:

   macOS / Linux

   S=../nextia-learning-labs/C13/snapshots/end-of-m01
   cp $S/doc_extract/route.py doc_extract/
   cp $S/tests/test_route.py tests/

   Windows (PowerShell)

   $S = "..\nextia-learning-labs\C13\snapshots\end-of-m01"
   Copy-Item "$S\doc_extract\route.py" doc_extract\
   Copy-Item "$S\tests\test_route.py" tests\

2. Run the tests:

   python -m pytest -q

   .............                                                            [100%]
   13 passed in 6.12s

   The four new tests check the first-bytes rule, the route of each kind of file, scans against photos, and the JPEG with a PDF name.

3. Route every document:

   python -m doc_extract.route

   The table has one line per document. Here are its first lines, a few from the middle, and its end:

   ID    file                               format  kind        routes
   D001  receipt-lk-986875.pdf              pdf     digital     text
   D019  scan0001.pdf                       pdf     scan        ocr
   D020  scan0002.jpg                       jpeg    scan        ocr
   D036  IMG_4817.jpg                       jpeg    photo       ocr
   D054  returns-policy-v4.pdf              pdf     digital     text text
   D057  aquaflow-price-list.xlsx           xlsx    spreadsheet sheet
   D058  warranty-claim-wc-2026-0418.pdf    pdf     mixed       text ocr
   D061  receipt-download.pdf               pdf     unreadable  stop
   D062  receipt-protected.pdf              pdf     unreadable  stop
   D063  scan-raw.tif                       tiff    scan        ocr
   D064  receipt-scan.pdf                   jpeg    scan        ocr  (wrong extension)
   D065  notes.txt                          unknown unreadable  stop
     digital 25, mixed 1, photo 15, scan 20, spreadsheet 1, unreadable 3

   routes has one word per page: D054 has two pages, both text. The kind is what a person would call the file. The last line counts the kinds. The whole table takes about half a second.

4. Look at one document page by page. The warranty claim D058 is a form with a scanned receipt as page 2:

   python -m doc_extract.route D058

   D058  warranty-claim-wc-2026-0418.pdf: format pdf, 2 page(s), kind mixed
     page 1: route text  (230 characters in the text layer; pictures cover 0%)
     page 2: route ocr   (a picture covers the page, no text layer; pictures cover 100%)

The chart shows the last line of the table:

• text route
• ocr route
• text + ocr, or sheet
• stop

The 65 documents by kind, as route.py finds them by looking inside each file. 35 of them are pictures that need OCR; 25 hold their text.

Text description of the chart

A column chart with six columns. Digital PDFs, whose pages all go to the text route: 25. Scans: 20, and phone photos: 15, both on the OCR route. One mixed PDF, with one text page and one OCR page. One spreadsheet, on the sheet route. Three unreadable files on the stop route. So more than half of the set, 35 documents, are pictures, and 25 hold their text.

The 25 documents that hold their text need no OCR: on them, the text layer is exact, fast and free. A single method for all 65 would be wrong on one of the two groups.

The mixed claim shows why the route belongs to a page, not to a file. Page 1 holds 230 characters: the claim number, the customer, the product and the problem. Page 2 is a scanned receipt pasted into the PDF. A router that looked only at page 1 would send page 2 to a text reader. The receipt’s total would then disappear, without any error.

One file, two routes. Page 1 of the claim D058 is text (blue boxes, from its text layer); page 2 is one picture of a scanned receipt (orange), so it goes to OCR.

Text description of the diagram

Page 1 of the claim is a form typed on a computer. The PDF reader finds its lines, drawn in blue. They hold the title “Larkfield · Warranty claim form”, the claim number WC-2026-0418, the customer C-0022 and the product, a garden hose. Then come the order number LK-921048, the problem, “The motor stops after two minutes”, and “Receipt: attached on page 2”. Page 2 is a grey scanned page with a small till receipt in its lower half. The PDF reader finds one picture, in orange, that covers the whole page, and no text. The receipt’s total, CAD 307.45, is on page 2, so only OCR can find it.

Photo or scan?

Both go to ocr, so why does the router tell them apart? Because they fail in different ways. A scan is usually flat and evenly lit, but it can be turned, faded or at low resolution. A photo has a background, an angle, a shadow and blur. Module 2 prepares each one differently before OCR.

The router uses a simple sign, in _looks_like_photo: it shrinks the picture to 64 × 64 pixels and looks at the border. A scan’s border is white or light grey. A photo’s border is the table or the floor: coloured or dark. This rule is right for every scan and photo in the set; a test checks it. On real photos, a white table can fool it. Then the cost is small: the photo gets the scan’s preparation, and Module 2 measures what preparation changes.

When to route, and what it costs

Route whenever your inputs come from more than one source. A help desk gets PDFs from tills, scans from offices and photos from phones, all mixed together. Route page by page, because one PDF can hold both kinds of page.

The router is cheap. It reads the first bytes, and, for a PDF, counts the characters on each page. For all 65 documents it took about half a second on the course team’s computer, with no model and no money.

Routing is not a full check of the file. D061, a download that stopped half way, and D062, a locked PDF, both get stop with the same reason: “cannot open”. The router knows that it cannot read them, but not why. The first lesson of Module 2 adds intake checks that say why, and stop the file with a clear message.

You may not need a router when every file comes from one source, for example a system that only receives PDFs from its own till. Even then, keep the first-bytes check: one JPEG with a .pdf name is enough to crash a PDF reader.

Common mistakes

Common mistake

Deciding the format by the file name

Symptom: a PDF reader stops on a file whose name ends in .pdf:

pdfplumber.utils.exceptions.PdfminerException: No /Root object! - Is this really a PDF?

Another PDF library, PDFium, gives PdfiumError: Failed to load document (PDFium: Data format error). for the same file.

Cause: the file is not a PDF. D064 is a JPEG that somebody saved as receipt-scan.pdf. The code chose the reader by the name.

Diagnose: read the first bytes. route D064 says it at once:

D064  receipt-scan.pdf: format jpeg  <- the name says .pdf, 1 page(s), kind scan
  page 1: route ocr   (a scanned image (white page); pictures cover 100%)

Fix: choose the reader by the first bytes, everywhere. In this project, route.sniff and files.is_pdf both do. Keep the original name in the record, so a person can see that the name was wrong.

Common mistake

A text reader that returns nothing, without an error

Symptom: the text reader runs on the scanned PDF D019 and returns zero lines. No error appears. Later, every field of D019 is empty.

Cause: D019 is a PDF, but its page holds only a picture. A text reader reads characters, and there are none.

Fix: route before you read. An empty text layer on a page that has a large picture means ocr, not “no text”. A page whose result is empty should always be a signal, never a silent pass.

Your turn: predict four routes

Change one thing: route documents that you have not seen yet. For each one, predict the format, the kind and the route. Use only the file name and what you know about the set. Then run python -m doc_extract.route with each ID.

1. D021 scan0003.pdf
2. D033 scan-partner-1.jpg
3. D055 garantie-legale-quebec.pdf (2 pages; page 2 is a table that lies on its side)
4. D063 scan-raw.tif

Show the answer

D021  scan0003.pdf: format pdf, 1 page(s), kind scan
  page 1: route ocr   (a picture covers the page, no text layer; pictures cover 100%)
D033  scan-partner-1.jpg: format jpeg, 1 page(s), kind scan
  page 1: route ocr   (a scanned image (white page); pictures cover 100%)
D055  garantie-legale-quebec.pdf: format pdf, 2 page(s), kind digital
  page 1: route text  (1079 characters in the text layer; pictures cover 0%)
  page 2: route text  (186 characters in the text layer; pictures cover 0%)
D063  scan-raw.tif: format tiff, 1 page(s), kind scan
  page 1: route ocr   (a scanned image (white page); pictures cover 100%)

• D021 is a PDF, but a scanned one: ocr. The .pdf name told you nothing about the route.
• D033 is a scan from the partner shop. Its route is the same as Larkfield’s scans. Its layout is different, and that matters later, not here.
• D055 is the surprise for many learners. Page 2 lies on its side, so it looks like a picture. But the PDF stores its text, and it only tells the viewer to turn the page. So both pages are text. The next lesson looks at turned pages.
• D063 is a TIFF scan. The route is ocr, but the file is 9.1 MB. Module 2’s intake stops files over 8 MB before any route runs.

Knowledge check

Check your understanding

Tomás says: 'Every PDF is an image. Send every page to the vision model.' Give two measured reasons why this is wrong for Larkfield's till PDFs, and say what to do instead.

Show the answer

• Time and money. The text layer took about 5.4 ms per page. The vision model took about 7.35 s per page (median) and cost about US$0.0113 per page. That is more than a thousand times slower, for a value that the file already holds.
• Errors. On D009, the model read “LARKEFIELD”; the text layer holds “LARKFIELD” exactly. A model can misread a clean page; a text layer cannot.

Instead: route by what is inside. A page with a text layer goes to text. Only picture pages go to OCR, and the vision model is kept for pages where it adds something, which Module 2 measures.

Check your understanding

Predict: a customer attaches 'invoice.pdf'. Its first bytes are %PDF-1.7. Page 1 has 1,200 characters in its text layer; page 2 has 4 characters and one picture that covers the page. What are the kind and the routes, and why does the page matter?

Show the answer

The format is PDF (the first bytes). Page 1 has 1,200 characters, more than 30: route text. Page 2 has only 4 characters and a picture that covers the page: route ocr. So the kind is mixed, with routes text ocr, like the claim D058. The page matters. A decision for the whole file would read page 2 with the text reader and get 4 characters. It would lose what the picture shows, with no error.

Check your understanding

Practical check: after this lesson, how many tests pass, what is the last line of `python -m doc_extract.route`, and what does it print for D064?

Show the answer

python -m pytest -q prints 13 passed. The last line of the table is digital 25, mixed 1, photo 15, scan 20, spreadsheet 1, unreadable 3. For D064, the table shows jpeg scan ocr (wrong extension), and route D064 says format jpeg <- the name says .pdf: the file is a JPEG with a PDF name, routed to OCR.

Recap

• A file’s name and look do not say what it is. Decide by what is inside. The first bytes give the format; for a PDF, the text layer of each page shows if it holds characters.
• Four routes: text (read the text layer), ocr (recognise the letters in a picture), sheet (read the cells), stop (say why). Routing is per page: the claim D058 is text ocr.
• The set has 25 digital files, 20 scans, 15 photos, 1 mixed, 1 spreadsheet and 3 unreadable. The router takes about half a second for all 65.
• A vision model on every page is slower (7.35 s against 5.4 ms per page) and costs US$0.0113 a page. It can also misread a clean page (“LARKEFIELD” on D009).
• python -m pytest -q prints 13 passed.

Next, you read the text layer with its boxes, and see how the order of the lines changes what a page means.

Next: Layout and meaning