receipity/receipt-parsing-spec.md

# Receipt Parsing Logic Specification

**Implementation guide (how to add a shop):** [`docs/receipt-parsing.md`](docs/receipt-parsing.md)
**Keyword config:** [`receipt-parsing-keywords.json`](receipt-parsing-keywords.json)

Purpose: define rule-based (non-AI, regex/heuristic) logic for turning raw
OCR text lines from a supermarket receipt into structured data: items,
quantities, unit prices, line totals, discounts, and receipt totals.

This spec is based on analysis of real receipts from multiple Dutch stores
(Jumbo, Lidl, Kruidvat, World Toko), which use at least four different line
layouts for the same underlying data. The parser must handle all of them
without assuming a single fixed format.

---

## 1. Input assumptions

- Input is structured OCR from ML Kit: **blocks → lines → elements**, each
  element carrying its own pixel bounding box (`left`, `top`, `right`,
  `bottom`) on the receipt image. Do **not** rely on the flattened `.text`
  property for parsing — it discards the two-column layout.
- Prices use European format: comma as decimal separator (e.g. `2,89`),
  period as thousands separator if present (rare on receipts this size).
- OCR may introduce noise: misread characters, extra/missing spaces,
  occasional dropped or merged lines. The parser should be tolerant of
  minor spacing irregularities but does not need to do spell-correction.
- Receipt photos vary in resolution and crop; all column thresholds must be
  computed **relative to the detected content width**, not as fixed pixels.

## 2. Preprocessing (run before classification)

1. Extract every text **element** with its bounding box from ML Kit.
2. **Rebuild rows** — do not trust ML Kit's line segmentation:
   - Sort elements by Y-center (top to bottom).
   - Cluster into rows where Y-centers fall within one line height of each
     other. Threshold = `median_element_height × row_cluster_height_factor`
     (default factor **1.2**, configurable in `spatial_parsing`).
   - Within each row, sort elements left-to-right by X position.
3. Join element text within a row (single spaces) for keyword/regex checks.
4. Trim leading/trailing whitespace; collapse multiple internal spaces to one.
5. Normalize decimal separators: treat `,` as the decimal point in any number
   matching `\d+[,.]\d{2}`. Convert to `.` internally for storage.
6. Do NOT drop short rows outright (e.g. a lone `BROCCOLI` or a lone
   `2 X 0,95`) — both are meaningful in different receipt layouts.

## 3. Core row classification

Classify every **row** (not flat OCR line) into one of these categories.
Within a row, use **X-position column logic** alongside regex:

- Compute content width = rightmost − leftmost element across the receipt.
- **Price column** = rightmost `price_column_width_fraction` (default **25%**)
  of that width. A row has a price when a token in the price column matches
  `\d+[,.]\d{2}`. Everything left of the price column is the candidate name.
- A row with no price-column match is **name-only**.

Check categories in this priority order (first match wins):

### 3.0 QUANTITY_MODIFIER (checked first — unconditional)

- If the **entire row** matches (not a substring):
  `^\d+\s*[xX]\s*\d+[,.]\d{2}\s*(PER\s+STUK|PER\s+KG|ST\.?|STUKS?)?$`
- Classify as QUANTITY_MODIFIER **unconditionally**. A pure `N X price` row
  can never be a standalone product, even if a price-column token is present.

### 3.1 STORE_HEADER
- The first 1–5 lines of the receipt, before the first recognizable
  item/price line appears.
- Often contains a store name in large/stylized text (e.g. "JUMBO",
  "Lidl", "Kruidvat"), an address, phone number, or website.
- Heuristic: lines before the first line matching an ITEM or
  QUANTITY_MODIFIER pattern (see below), or before a line matching
  `OMSCHRIJVING` / `Artikel` / similar column-header keywords.
- Extract the store name from this block using a known store-name list
  (maintain and grow this list — seen so far: JUMBO, LIDL, KRUIDVAT,
  ALBERT HEIJN, ALDI, and independent/ethnic grocers whose name appears
  as free text, e.g. "World Toko" — for these, take the first non-address
  line as a best-effort store name).

### 3.2 COLUMN_HEADER (ignore, not an item)
- Lines that are just table headers, not data.
- Match keywords: `OMSCHRIJVING`, `BEDRAG IN`, `Artikel`, `Prijs`,
  `Aantal Artikel`, or lines that are purely `=`, `-`, or `#` separator
  characters.

### 3.3 ITEM (product line)
- A line containing a product name AND a price at or near the end of
  the line.
- Price pattern: `\d+[,.]\d{2}` anchored near line end (allow trailing
  single-letter suffixes like ` B` or ` C` — see section 6, BTW category
  codes — these are not part of the price).
- Everything before the matched price is the product name (trim
  trailing spaces).
- This is the "complete" case: name + price on the same line, quantity
  of 1 implied unless a QUANTITY_MODIFIER line follows (section 3.4).

### 3.4 QUANTITY_MODIFIER (sub-row, belongs to previous or next ITEM)

See **§3.0** — pure `N X price` rows are detected before any other type.
The rules below describe how they attach to items:

- This row does NOT introduce a new product — it modifies the item on
  an adjacent row.
- Two observed variants — the parser must detect which applies:
  - **Variant A (Jumbo-style):** product name appears on its own line
    with NO price. The next line is `N X unit_price`. The line total
    (`N × unit_price`) appears as its own value either on the same
    `N X price` line's right-hand column, or as a separate line
    immediately after. Associate this quantity/total back to the
    product name line above it.
  - **Variant B (Kruidvat-style):** product name and `N X unit_price`
    appear on the SAME line, description-style (e.g.
    `ZEEPTABLET NEUTRAL 2 X 3,99 PER STUK`), with the computed total
    price in a separate right-aligned amount column/line.
- In both variants: `quantity = N`, `unit_price = price`,
  `line_total = N * unit_price` (compute this yourself, then cross-check
  against any total value found nearby — if they don't match within
  0.01, flag the item for manual review rather than guessing).

### 3.5 DISCOUNT
- A line representing a price reduction, not a product.
- Detection rules (match ANY of):
  - Line starts with or contains `ACTIE` (Jumbo pattern), e.g.
    `ACTIE AARDBEIEN` with an associated negative amount.
  - Line contains `KORTING` (Kruidvat/general Dutch for "discount"),
    e.g. `KORTING NEUTRAL ZE.HP`.
  - Line contains `In prijs verlaagd` (Lidl pattern — "price reduced").
  - The associated amount is negative (prefixed with `-`), e.g. `-2,00`,
    `-1,55`, `-0,21`, `-0,50`.
- A discount line should be linked to the item immediately preceding it
  when possible (the discount usually applies to the last product
  listed above it). Store it as a `discount` field on that item rather
  than as a separate negative "product."
- If a discount line cannot be confidently linked to a specific item
  (e.g. it's a receipt-wide discount near the total, like
  `Totaal korting: -2,21`), store it as a receipt-level
  `total_discount` field instead.

### 3.6 SUBTOTAL / TOTAL
- Lines matching keywords: `Totaal`, `SUBTOTAAL`, `Total`, `TOTAAL`.
- There may be MULTIPLE lines containing "Totaal" on one receipt
  (e.g. a running subtotal, then a final Totaal, then
  "Totaal korting" near the bottom, then "Totaal incl. BTW"). Rules:
  - The FIRST large bold "Totaal" line following the last item is the
    **receipt grand total** — use this as the authoritative total to
    validate your parsed items against.
  - Any "Totaal korting" line is the **sum of all discounts** — use
    this as a cross-check: sum of all `discount` fields you extracted
    should equal this value (within 0.01).
  - "Totaal incl. BTW" / "Totaal excl. BTW" relate to tax breakdown,
    not needed for item-level data, but can be stored as metadata.

### 3.7 PAYMENT (ignore for item extraction, optionally store as metadata)
- Lines matching: `Betaald`, `BETALING`, `VISA`, `Mastercard`, `PIN`,
  `Contactloze betaling`, `AKKOORD`, `Kaartnr`, `Transactie`,
  `Terminal`, `Merchant`, `Auth. code`, `Kaart`.
- These lines and everything below/around them (transaction IDs, auth
  codes, terminal numbers) should be excluded from item parsing
  entirely. Treat "first payment-related line after the last Totaal
  line" as the end of the item-relevant section of the receipt.

### 3.8 TAX / BTW BREAKDOWN (ignore for item extraction)
- Lines matching: `BTW`, `Bedr.Excl`, `Bedr.Incl`, a table with columns
  like `B 9%` / `C 21%`, `BTW-Code`.
- These describe tax categories, not individual products. Some product
  lines end in a single letter (`B` or `C`) marking their tax category
  — strip this letter before/after price extraction, don't treat it as
  part of the price or name.

### 3.9 NOISE / FOOTER (ignore entirely)
- Loyalty program mentions, website URLs, "Bewaar uw kassabon",
  return policy text, barcodes represented as text/numbers, store
  hours, social media handles, "Bedankt en graag tot ziens",
  promotional taglines. Match a broad keyword/pattern list and treat
  anything after the payment block as low-priority/ignorable by default.

## 4. Item construction algorithm (pseudocode)

Uses a **one-row lookahead buffer** — do not emit an item until the next
row is inspected (or the receipt ends).

```
items = []
pending_item = None   # name found, price/qty not yet resolved

for row in classified_rows:
    if row.type == QUANTITY_MODIFIER:
        if pending_item:
            pending_item.quantity = row.qty
            pending_item.unit_price = row.price
            pending_item.line_total = row.qty * row.price
            merge bounding boxes from row into pending_item
            items.append(pending_item)
            pending_item = None
        continue   # orphan modifier without pending → ignore

    if row.type == NAME_ONLY:
        if pending_item:
            items.append(pending_item with needs_review if no price)
        pending_item = { name: row.name, unit_price: None, quantity: 1,
                          line_total: None, discount: 0, bounds: row.bounds }
        peek next row:
            if next.type == QUANTITY_MODIFIER → defer emit (handled above)
            else → append pending_item flagged needs_review, clear pending
        continue

    if row.type == ITEM:
        if pending_item:
            items.append(pending_item with needs_review if no price)
        pending_item = { name: row.name, unit_price: row.price,
                          quantity: 1, line_total: row.price,
                          discount: 0, bounds: row.bounds }
        peek next row:
            if next.type == QUANTITY_MODIFIER → defer emit (modifier overrides qty/unit)
            else → append pending_item, clear pending
        continue

    if row.type == DISCOUNT:
        ... (unchanged: attach to pending_item or receipt.total_discount)

    if row.type in (SUBTOTAL, TOTAL):
        flush pending_item
        if is_grand_total(row):
            receipt.total = row.amount
            break

flush any trailing pending_item
```

Each emitted item keeps **source bounding box(es)** (normalized 0–1 on the
receipt image) for highlight overlays in the review UI.

## 5. Validation pass (do this after parsing, before showing results to user)

1. Sum `line_total` across all parsed items.
2. Add `receipt.total_discount` if it wasn't already folded into
   individual item discounts.
3. Compare the result to `receipt.total` (the grand total line).
4. If the receipt prints an item count (e.g. `Aantal Artikel: N`), compare
   `N` to the number of parsed items.
5. If totals match within 0.01 **and** item count matches (when present)
   **and** no item is flagged `needs_manual_review` → high confidence.
6. If any check fails → mark the **whole receipt** as "needs review".

## 6. Known store-specific quirks to encode as configurable rules

Keep these as a lookup table / config, not hardcoded logic, so new
stores can be added without rewriting the parser:

| Store | Quirk |
|---|---|
| Jumbo | Multi-unit items: name on own line, then `N X price` line, then total. Discounts prefixed `ACTIE <name>` with negative amount. Tax category letter (`B`/`C`) sometimes appended to far-right column, not the price. |
| Lidl | Discounts appear as `In prijs verlaagd -0,50` style lines. Tax breakdown at the bottom uses `Bedr.Excl` / `BTW` / `Bedr.Incl` labels with `B 9%` / `C 21%` category rows — do not parse these rows as products. |
| Kruidvat | Multi-unit items combine name + `N X price PER STUK` on one line. `KORTING <description>` lines carry the discount, may appear one or two lines after the item, not always immediately after. |
| World Toko | Simple `Artikel` / `Prijs` two-column layout, mostly one item per line with price directly alongside; some multi-unit items still use a following `N X price` line (Variant A). |

Add new rows to this table as new receipt formats are encountered — this
is meant to grow with real-world use rather than be exhaustive on day one.

## 7. Output schema (what the parser should produce per receipt)

```json
{
  "store": "Jumbo",
  "date": "2026-08-30",
  "items": [
    {
      "name": "Jumbo Bio Volle Melk",
      "quantity": 1,
      "unit_price": 2.09,
      "discount": 0.00,
      "line_total": 2.09
    },
    {
      "name": "Biologisch Ei 6 st",
      "quantity": 2,
      "unit_price": 2.79,
      "discount": 0.00,
      "line_total": 5.58
    },
    {
      "name": "Aardbeien",
      "quantity": 2,
      "unit_price": 3.99,
      "discount": -2.00,
      "line_total": 5.98
    }
  ],
  "total_discount": -2.21,
  "receipt_total": 82.99,
  "validation_passed": true
}
```

## 8. Explicit non-goals for this parser

- No attempt to categorize items (groceries vs household, etc.) —
  out of scope.
- No attempt to normalize/deduplicate product names across receipts at
  parse time (e.g. matching "Jumbo Bio Volle Melk" to a previous
  purchase) — that's a separate matching step done after storage, not
  part of this parsing spec.
- No language detection/translation — assume Dutch keyword lists for
  now (`ACTIE`, `KORTING`, `Totaal`, `Betaald`, etc.), but structure the
  keyword lists as swappable config so other languages can be added.