receipity/receipt-parsing-spec.md

Receipt Parsing Logic Specification

Implementation guide (how to add a shop): docs/receipt-parsing.md Keyword config: receipt-parsing-keywords.json

Purpose: define rule-based (non-AI, regex/heuristic) logic for turning raw OCR text lines from a supermarket receipt into structured data: items, quantities, unit prices, line totals, discounts, and receipt totals.

This spec is based on analysis of real receipts from multiple Dutch stores (Jumbo, Lidl, Kruidvat, World Toko), which use at least four different line layouts for the same underlying data. The parser must handle all of them without assuming a single fixed format.

---

1. Input assumptions

• Input is structured OCR from ML Kit: blocks → lines → elements, each element carrying its own pixel bounding box (left, top, right, bottom) on the receipt image. Do not rely on the flattened .text property for parsing — it discards the two-column layout.
• Prices use European format: comma as decimal separator (e.g. 2,89), period as thousands separator if present (rare on receipts this size).
• OCR may introduce noise: misread characters, extra/missing spaces, occasional dropped or merged lines. The parser should be tolerant of minor spacing irregularities but does not need to do spell-correction.
• Receipt photos vary in resolution and crop; all column thresholds must be computed relative to the detected content width, not as fixed pixels.

2. Preprocessing (run before classification)

1. Extract every text element with its bounding box from ML Kit.
2. Rebuild rows — do not trust ML Kit's line segmentation:
   • Sort elements by Y-center (top to bottom).
   • Cluster into rows where Y-centers fall within one line height of each other. Threshold = median_element_height × row_cluster_height_factor (default factor 1.2, configurable in spatial_parsing).
   • Within each row, sort elements left-to-right by X position.
3. Join element text within a row (single spaces) for keyword/regex checks.
4. Trim leading/trailing whitespace; collapse multiple internal spaces to one.
5. Normalize decimal separators: treat , as the decimal point in any number matching \d+[,.]\d{2}. Convert to . internally for storage.
6. Do NOT drop short rows outright (e.g. a lone BROCCOLI or a lone 2 X 0,95) — both are meaningful in different receipt layouts.

3. Core row classification

Classify every row (not flat OCR line) into one of these categories. Within a row, use X-position column logic alongside regex:

• Compute content width = rightmost − leftmost element across the receipt.
• Price column = rightmost price_column_width_fraction (default 25%) of that width. A row has a price when a token in the price column matches \d+[,.]\d{2}. Everything left of the price column is the candidate name.
• A row with no price-column match is name-only.

Check categories in this priority order (first match wins):

3.0 QUANTITY_MODIFIER (checked first — unconditional)

• If the entire row matches (not a substring): ^\d+\s*[xX]\s*\d+[,.]\d{2}\s*(PER\s+STUK|PER\s+KG|ST\.?|STUKS?)?$
• Classify as QUANTITY_MODIFIER unconditionally. A pure N X price row can never be a standalone product, even if a price-column token is present.

3.1 STORE_HEADER

• The first 1–5 lines of the receipt, before the first recognizable item/price line appears.
• Often contains a store name in large/stylized text (e.g. "JUMBO", "Lidl", "Kruidvat"), an address, phone number, or website.
• Heuristic: lines before the first line matching an ITEM or QUANTITY_MODIFIER pattern (see below), or before a line matching OMSCHRIJVING / Artikel / similar column-header keywords.
• Extract the store name from this block using a known store-name list (maintain and grow this list — seen so far: JUMBO, LIDL, KRUIDVAT, ALBERT HEIJN, ALDI, and independent/ethnic grocers whose name appears as free text, e.g. "World Toko" — for these, take the first non-address line as a best-effort store name).

3.2 COLUMN_HEADER (ignore, not an item)

• Lines that are just table headers, not data.
• Match keywords: OMSCHRIJVING, BEDRAG IN, Artikel, Prijs, Aantal Artikel, or lines that are purely =, -, or # separator characters.

3.3 ITEM (product line)

• A line containing a product name AND a price at or near the end of the line.
• Price pattern: \d+[,.]\d{2} anchored near line end (allow trailing single-letter suffixes like B or C — see section 6, BTW category codes — these are not part of the price).
• Everything before the matched price is the product name (trim trailing spaces).
• This is the "complete" case: name + price on the same line, quantity of 1 implied unless a QUANTITY_MODIFIER line follows (section 3.4).

3.4 QUANTITY_MODIFIER (sub-row, belongs to previous or next ITEM)

See §3.0 — pure N X price rows are detected before any other type. The rules below describe how they attach to items:

• This row does NOT introduce a new product — it modifies the item on an adjacent row.
• Two observed variants — the parser must detect which applies:
  • Variant A (Jumbo-style): product name appears on its own line with NO price. The next line is N X unit_price. The line total (N × unit_price) appears as its own value either on the same N X price line's right-hand column, or as a separate line immediately after. Associate this quantity/total back to the product name line above it.
  • Variant B (Kruidvat-style): product name and N X unit_price appear on the SAME line, description-style (e.g. ZEEPTABLET NEUTRAL 2 X 3,99 PER STUK), with the computed total price in a separate right-aligned amount column/line.
• In both variants: quantity = N, unit_price = price, line_total = N * unit_price (compute this yourself, then cross-check against any total value found nearby — if they don't match within 0.01, flag the item for manual review rather than guessing).

3.5 DISCOUNT

• A line representing a price reduction, not a product.
• Detection rules (match ANY of):
  • Line starts with or contains ACTIE (Jumbo pattern), e.g. ACTIE AARDBEIEN with an associated negative amount.
  • Line contains KORTING (Kruidvat/general Dutch for "discount"), e.g. KORTING NEUTRAL ZE.HP.
  • Line contains In prijs verlaagd (Lidl pattern — "price reduced").
  • The associated amount is negative (prefixed with -), e.g. -2,00, -1,55, -0,21, -0,50.
• A discount line should be linked to the item immediately preceding it when possible (the discount usually applies to the last product listed above it). Store it as a discount field on that item rather than as a separate negative "product."
• If a discount line cannot be confidently linked to a specific item (e.g. it's a receipt-wide discount near the total, like Totaal korting: -2,21), store it as a receipt-level total_discount field instead.

3.6 SUBTOTAL / TOTAL

• Lines matching keywords: Totaal, SUBTOTAAL, Total, TOTAAL.
• There may be MULTIPLE lines containing "Totaal" on one receipt (e.g. a running subtotal, then a final Totaal, then "Totaal korting" near the bottom, then "Totaal incl. BTW"). Rules:
  • The FIRST large bold "Totaal" line following the last item is the receipt grand total — use this as the authoritative total to validate your parsed items against.
  • Any "Totaal korting" line is the sum of all discounts — use this as a cross-check: sum of all discount fields you extracted should equal this value (within 0.01).
  • "Totaal incl. BTW" / "Totaal excl. BTW" relate to tax breakdown, not needed for item-level data, but can be stored as metadata.

3.7 PAYMENT (ignore for item extraction, optionally store as metadata)

• Lines matching: Betaald, BETALING, VISA, Mastercard, PIN, Contactloze betaling, AKKOORD, Kaartnr, Transactie, Terminal, Merchant, Auth. code, Kaart.
• These lines and everything below/around them (transaction IDs, auth codes, terminal numbers) should be excluded from item parsing entirely. Treat "first payment-related line after the last Totaal line" as the end of the item-relevant section of the receipt.

3.8 TAX / BTW BREAKDOWN (ignore for item extraction)

• Lines matching: BTW, Bedr.Excl, Bedr.Incl, a table with columns like B 9% / C 21%, BTW-Code.
• These describe tax categories, not individual products. Some product lines end in a single letter (B or C) marking their tax category — strip this letter before/after price extraction, don't treat it as part of the price or name.

3.9 NOISE / FOOTER (ignore entirely)

• Loyalty program mentions, website URLs, "Bewaar uw kassabon", return policy text, barcodes represented as text/numbers, store hours, social media handles, "Bedankt en graag tot ziens", promotional taglines. Match a broad keyword/pattern list and treat anything after the payment block as low-priority/ignorable by default.

4. Item construction algorithm (pseudocode)

Uses a one-row lookahead buffer — do not emit an item until the next row is inspected (or the receipt ends).

items = []
pending_item = None   # name found, price/qty not yet resolved

for row in classified_rows:
    if row.type == QUANTITY_MODIFIER:
        if pending_item:
            pending_item.quantity = row.qty
            pending_item.unit_price = row.price
            pending_item.line_total = row.qty * row.price
            merge bounding boxes from row into pending_item
            items.append(pending_item)
            pending_item = None
        continue   # orphan modifier without pending → ignore

    if row.type == NAME_ONLY:
        if pending_item:
            items.append(pending_item with needs_review if no price)
        pending_item = { name: row.name, unit_price: None, quantity: 1,
                          line_total: None, discount: 0, bounds: row.bounds }
        peek next row:
            if next.type == QUANTITY_MODIFIER → defer emit (handled above)
            else → append pending_item flagged needs_review, clear pending
        continue

    if row.type == ITEM:
        if pending_item:
            items.append(pending_item with needs_review if no price)
        pending_item = { name: row.name, unit_price: row.price,
                          quantity: 1, line_total: row.price,
                          discount: 0, bounds: row.bounds }
        peek next row:
            if next.type == QUANTITY_MODIFIER → defer emit (modifier overrides qty/unit)
            else → append pending_item, clear pending
        continue

    if row.type == DISCOUNT:
        ... (unchanged: attach to pending_item or receipt.total_discount)

    if row.type in (SUBTOTAL, TOTAL):
        flush pending_item
        if is_grand_total(row):
            receipt.total = row.amount
            break

flush any trailing pending_item

Each emitted item keeps source bounding box(es) (normalized 0–1 on the receipt image) for highlight overlays in the review UI.

5. Validation pass (do this after parsing, before showing results to user)

1. Sum line_total across all parsed items.
2. Add receipt.total_discount if it wasn't already folded into individual item discounts.
3. Compare the result to receipt.total (the grand total line).
4. If the receipt prints an item count (e.g. Aantal Artikel: N), compare N to the number of parsed items.
5. If totals match within 0.01 and item count matches (when present) and no item is flagged needs_manual_review → high confidence.
6. If any check fails → mark the whole receipt as "needs review".

6. Known store-specific quirks to encode as configurable rules

Keep these as a lookup table / config, not hardcoded logic, so new stores can be added without rewriting the parser:

Store	Quirk
Jumbo	Multi-unit items: name on own line, then N X price line, then total. Discounts prefixed ACTIE <name> with negative amount. Tax category letter (B/C) sometimes appended to far-right column, not the price.
Lidl	Discounts appear as In prijs verlaagd -0,50 style lines. Tax breakdown at the bottom uses Bedr.Excl / BTW / Bedr.Incl labels with B 9% / C 21% category rows — do not parse these rows as products.
Kruidvat	Multi-unit items combine name + N X price PER STUK on one line. KORTING <description> lines carry the discount, may appear one or two lines after the item, not always immediately after.
World Toko	Simple Artikel / Prijs two-column layout, mostly one item per line with price directly alongside; some multi-unit items still use a following N X price line (Variant A).

Add new rows to this table as new receipt formats are encountered — this is meant to grow with real-world use rather than be exhaustive on day one.

7. Output schema (what the parser should produce per receipt)

{
  "store": "Jumbo",
  "date": "2026-08-30",
  "items": [
    {
      "name": "Jumbo Bio Volle Melk",
      "quantity": 1,
      "unit_price": 2.09,
      "discount": 0.00,
      "line_total": 2.09
    },
    {
      "name": "Biologisch Ei 6 st",
      "quantity": 2,
      "unit_price": 2.79,
      "discount": 0.00,
      "line_total": 5.58
    },
    {
      "name": "Aardbeien",
      "quantity": 2,
      "unit_price": 3.99,
      "discount": -2.00,
      "line_total": 5.98
    }
  ],
  "total_discount": -2.21,
  "receipt_total": 82.99,
  "validation_passed": true
}

8. Explicit non-goals for this parser

• No attempt to categorize items (groceries vs household, etc.) — out of scope.
• No attempt to normalize/deduplicate product names across receipts at parse time (e.g. matching "Jumbo Bio Volle Melk" to a previous purchase) — that's a separate matching step done after storage, not part of this parsing spec.
• No language detection/translation — assume Dutch keyword lists for now (ACTIE, KORTING, Totaal, Betaald, etc.), but structure the keyword lists as swappable config so other languages can be added.