receipity/docs/receipt-parsing.md

Receipt parsing (implementation)

This is the working guide for Receipity's receipt parser. Use it when a new shop's receipt does not parse well.

The original design notes live in receipt-parsing-spec.md. Keywords and regexes live in receipt-parsing-keywords.json and are bundled as a Flutter asset.

---

What the parser produces

For each receipt:

Field	Meaning
items[].name	Product name
items[].quantity	Pack count when N X unit was found (otherwise omitted)
items[].unitPrice	Price of one pack when quantity is set
items[].discount	Negative amount folded into the line (promos)
items[].price	Line total after discount (this is what the trip saves)
store	Best-effort store from the header + known_stores
receiptTotal	First grand-total amount after the items
totalDiscount	Receipt-level Totaal korting (and similar)
validationPassed	Item sum matches receiptTotal within €0.01

If validation fails, Review shows: Parsed totals do not match the receipt.

---

Pipeline

1. Preprocess each OCR line: trim, collapse spaces, drop empty lines.
2. Choose a layout
   • Sequential (default): name, price, quantity and discount appear in reading order. Jumbo (printed), Lidl, Kruidvat, World Toko.
   • Column OCR fallback: ML Kit sometimes dumps every name, then every amount. Detected when there is a run of 4+ price-only lines and enough name-like lines. Used for the Jumbo sample dump in tests.
3. Classify each line (first match wins) using the keyword JSON.
4. Build items (pending name, then quantity/discount/price).
5. Validate against the receipt total when one was found.

Do not treat weight in the name (900g, 500ml) as quantity. Only N X next to a price is a quantity.

---

Line types

Priority is the same as the spec:

1. Separators / column headers (OMSCHRIJVING, Artikel, Prijs, …)
2. Tax / BTW tables (BTW, Bedr.Excl, B 9%)
3. Payment (Betaald, VISA, PIN, …) — also matches OCR Betaal d:
4. Noise / footer (URLs, opening hours, thank-you lines)
5. Receipt-level discount total (Totaal korting)
6. Subtotal / tax-inclusive total (ignored for items)
7. Grand total (Totaal, Total) — stops sequential item parsing when an amount is on the same line
8. Discount (ACTIE, KORTING, In prijs verlaagd, or a negative amount)
9. Quantity modifier (2 X 2,79, optional PER STUK, optional line total)
10. Item (name + price at end; optional tax letter B/C is stripped)
11. Price-only
12. Name-only (starts a pending item — Jumbo multi-pack)

Quantity variants

Variant	Example	Store
A — name, then N X unit	AARDBEIEN / 2 X 3,99	Jumbo, World Toko
B — name and N X unit on one line	ZEEPTABLET NEUTRAL 2 X 3,99 PER STUK 7,98	Kruidvat

The JSON standalone quantity regex is extended in code so a right-hand line total is allowed: 2 X 2,79 5,58.

Discounts

A discount is attached to the current / previous item and subtracted from price. Receipt-wide totals (Totaal korting) are not products.

---

How to add a new shop

Work in this order. You usually only need step 1.

1. Keywords (receipt-parsing-keywords.json)

Add to	When
known_stores.list	Header contains a new banner name
column_header_keywords.list	New table titles (Omschrijving, Amount, …)
discount_keywords.list	Promo wording (Bonus, Sparing, …)
total_keywords.*	Other words for subtotal / grand total
payment_keywords.list	Card brands, Betaald met …
tax_breakdown_keywords.list	VAT table labels
noise_footer_keywords.list	Loyalty, hours, slogans that were parsed as items
quantity_modifier_pattern	Only if N X looks different (2x, 2 *, à)

Matching rules implemented in code (not only the JSON comment):

• Keywords of 3 characters or fewer use a word boundary (PIN must not match SPINAZIE).
• Column headers and item-count lines must be the whole line (so Prijs as a header does not hide Lidl's In prijs verlaagd).

After editing the JSON, rebuild the app (it is loaded as an asset at startup). Tests load the same file from the project root.

2. Store quirks table

Add a row here and in receipt-parsing-spec.md section 6:

Store	Layout	Notes
Jumbo	A	ACTIE <name> + negative amount. Tax letter B/C far right. OCR may split names vs amounts → column fallback.
Lidl	sequential	In prijs verlaagd -0,50. Ignore Bedr.Excl / B 9% rows.
Kruidvat	B	N X price PER STUK on the product line. KORTING … may be one line later.
World Toko	A / simple	Artikel / Prijs headers. Weight in the name is not quantity.
(new shop)	A / B / columns	Short description of the odd lines

3. Canonical store name

If OCR should pre-select the supermarket on Review:

• Add the banner string to known_stores.list
• Add an alias in lib/utils/supermarkets.dart (kSupermarketAliases)
• Map the uppercase banner in ReceiptParser._canonicalStore

4. Fixture test

1. Scan a real receipt, copy Raw OCR text from Review.
2. Add a test in test/receipt_parser_test.dart with that dump.
3. Assert product names, a couple of prices, quantity lines, and that headers / totals / payment lines are not items.

Keep the raw dump in the test (or a test/fixtures/ file) so the next person can see the real layout.

5. Only then change parser code

Change lib/services/receipt_parser.dart if the new layout is a new structure (not just new words): extra columns, quantity written as 2 stuks à 1,50, discounts above the product, etc.

Prefer a small, documented branch over a one-off special case for one shop.

---

Files

File	Role
receipt-parsing-keywords.json	Editable keyword / regex config
receipt-parsing-spec.md	Design: classification order, item algorithm
lib/services/receipt_parse_config.dart	Loads JSON, keyword matching
lib/services/receipt_parser.dart	Classify → items → validate
test/receipt_parser_test.dart	Layout fixtures

---

Out of scope (parser)

• Product categories
• Matching names to the existing catalog
• Translating receipts; add another keyword list if you need a second language