improve parsing
This commit is contained in:
davmar
+92
-64
@@ -16,30 +16,53 @@ without assuming a single fixed format.
|
||||
|
||||
## 1. Input assumptions
|
||||
|
||||
- Input is raw OCR text, one line per detected text line, in top-to-bottom
|
||||
order as it appeared on the receipt.
|
||||
- Input is structured OCR from ML Kit: **blocks → lines → elements**, each
|
||||
element carrying its own pixel bounding box (`left`, `top`, `right`,
|
||||
`bottom`) on the receipt image. Do **not** rely on the flattened `.text`
|
||||
property for parsing — it discards the two-column layout.
|
||||
- Prices use European format: comma as decimal separator (e.g. `2,89`),
|
||||
period as thousands separator if present (rare on receipts this size).
|
||||
- OCR may introduce noise: misread characters, extra/missing spaces,
|
||||
occasional dropped or merged lines. The parser should be tolerant of
|
||||
minor spacing irregularities but does not need to do spell-correction.
|
||||
- Receipt photos vary in resolution and crop; all column thresholds must be
|
||||
computed **relative to the detected content width**, not as fixed pixels.
|
||||
|
||||
## 2. Preprocessing (run before classification)
|
||||
|
||||
1. Trim leading/trailing whitespace on every line; collapse multiple
|
||||
internal spaces to one.
|
||||
2. Normalize decimal separators: treat `,` as the decimal point in any
|
||||
number matching `\d+[,.]\d{2}`. Convert to a standard `.` internally
|
||||
for numeric storage, but remember the original format was comma-based
|
||||
(useful for locale-aware re-parsing if needed).
|
||||
3. Drop fully empty lines.
|
||||
4. Do NOT drop short lines outright (e.g. a lone `BROCCOLI` or a lone
|
||||
1. Extract every text **element** with its bounding box from ML Kit.
|
||||
2. **Rebuild rows** — do not trust ML Kit's line segmentation:
|
||||
- Sort elements by Y-center (top to bottom).
|
||||
- Cluster into rows where Y-centers fall within one line height of each
|
||||
other. Threshold = `median_element_height × row_cluster_height_factor`
|
||||
(default factor **1.2**, configurable in `spatial_parsing`).
|
||||
- Within each row, sort elements left-to-right by X position.
|
||||
3. Join element text within a row (single spaces) for keyword/regex checks.
|
||||
4. Trim leading/trailing whitespace; collapse multiple internal spaces to one.
|
||||
5. Normalize decimal separators: treat `,` as the decimal point in any number
|
||||
matching `\d+[,.]\d{2}`. Convert to `.` internally for storage.
|
||||
6. Do NOT drop short rows outright (e.g. a lone `BROCCOLI` or a lone
|
||||
`2 X 0,95`) — both are meaningful in different receipt layouts.
|
||||
|
||||
## 3. Core line classification
|
||||
## 3. Core row classification
|
||||
|
||||
Classify every line into one of these categories, in this priority order
|
||||
(check top-down, first match wins):
|
||||
Classify every **row** (not flat OCR line) into one of these categories.
|
||||
Within a row, use **X-position column logic** alongside regex:
|
||||
|
||||
- Compute content width = rightmost − leftmost element across the receipt.
|
||||
- **Price column** = rightmost `price_column_width_fraction` (default **25%**)
|
||||
of that width. A row has a price when a token in the price column matches
|
||||
`\d+[,.]\d{2}`. Everything left of the price column is the candidate name.
|
||||
- A row with no price-column match is **name-only**.
|
||||
|
||||
Check categories in this priority order (first match wins):
|
||||
|
||||
### 3.0 QUANTITY_MODIFIER (checked first — unconditional)
|
||||
|
||||
- If the **entire row** matches (not a substring):
|
||||
`^\d+\s*[xX]\s*\d+[,.]\d{2}\s*(PER\s+STUK|PER\s+KG|ST\.?|STUKS?)?$`
|
||||
- Classify as QUANTITY_MODIFIER **unconditionally**. A pure `N X price` row
|
||||
can never be a standalone product, even if a price-column token is present.
|
||||
|
||||
### 3.1 STORE_HEADER
|
||||
- The first 1–5 lines of the receipt, before the first recognizable
|
||||
@@ -72,13 +95,13 @@ Classify every line into one of these categories, in this priority order
|
||||
- This is the "complete" case: name + price on the same line, quantity
|
||||
of 1 implied unless a QUANTITY_MODIFIER line follows (section 3.4).
|
||||
|
||||
### 3.4 QUANTITY_MODIFIER (sub-line, belongs to previous or next ITEM)
|
||||
- A line matching the pattern: `<int> X <price>` optionally followed by
|
||||
a unit word (e.g. `PER STUK`), e.g.:
|
||||
- `2 X 2,79`
|
||||
- `2 X 3,99 PER STUK`
|
||||
- This line does NOT introduce a new product — it modifies the item on
|
||||
an adjacent line.
|
||||
### 3.4 QUANTITY_MODIFIER (sub-row, belongs to previous or next ITEM)
|
||||
|
||||
See **§3.0** — pure `N X price` rows are detected before any other type.
|
||||
The rules below describe how they attach to items:
|
||||
|
||||
- This row does NOT introduce a new product — it modifies the item on
|
||||
an adjacent row.
|
||||
- Two observed variants — the parser must detect which applies:
|
||||
- **Variant A (Jumbo-style):** product name appears on its own line
|
||||
with NO price. The next line is `N X unit_price`. The line total
|
||||
@@ -154,66 +177,71 @@ Classify every line into one of these categories, in this priority order
|
||||
|
||||
## 4. Item construction algorithm (pseudocode)
|
||||
|
||||
Uses a **one-row lookahead buffer** — do not emit an item until the next
|
||||
row is inspected (or the receipt ends).
|
||||
|
||||
```
|
||||
items = []
|
||||
pending_item = None # name found, price/qty not yet resolved
|
||||
|
||||
for line in classified_lines:
|
||||
if line.type == ITEM:
|
||||
if pending_item:
|
||||
items.append(pending_item) # close out previous item first
|
||||
pending_item = { name: line.name, unit_price: line.price,
|
||||
quantity: 1, line_total: line.price,
|
||||
discount: 0 }
|
||||
|
||||
elif line.type == QUANTITY_MODIFIER:
|
||||
if pending_item and pending_item.unit_price is None:
|
||||
# Variant A: name-only line was pending, this line completes it
|
||||
pending_item.quantity = line.qty
|
||||
pending_item.unit_price = line.price
|
||||
pending_item.line_total = line.qty * line.price
|
||||
elif pending_item:
|
||||
# Variant B: name+price already set, this line refines/
|
||||
# confirms it (overwrite quantity/unit price, recompute total)
|
||||
pending_item.quantity = line.qty
|
||||
pending_item.unit_price = line.price
|
||||
pending_item.line_total = line.qty * line.price
|
||||
else:
|
||||
# Orphan quantity line with no preceding item — flag for
|
||||
# manual review rather than silently dropping
|
||||
flag_for_review(line)
|
||||
|
||||
elif line.type == DISCOUNT:
|
||||
if pending_item:
|
||||
pending_item.discount += line.amount # amount is negative
|
||||
pending_item.line_total += line.amount
|
||||
else:
|
||||
receipt.total_discount += line.amount
|
||||
|
||||
elif line.type in (SUBTOTAL, TOTAL):
|
||||
for row in classified_rows:
|
||||
if row.type == QUANTITY_MODIFIER:
|
||||
if pending_item:
|
||||
pending_item.quantity = row.qty
|
||||
pending_item.unit_price = row.price
|
||||
pending_item.line_total = row.qty * row.price
|
||||
merge bounding boxes from row into pending_item
|
||||
items.append(pending_item)
|
||||
pending_item = None
|
||||
if is_grand_total(line):
|
||||
receipt.total = line.amount
|
||||
break # stop item parsing here
|
||||
continue # orphan modifier without pending → ignore
|
||||
|
||||
# catch any trailing pending item if receipt ended oddly
|
||||
if pending_item:
|
||||
items.append(pending_item)
|
||||
if row.type == NAME_ONLY:
|
||||
if pending_item:
|
||||
items.append(pending_item with needs_review if no price)
|
||||
pending_item = { name: row.name, unit_price: None, quantity: 1,
|
||||
line_total: None, discount: 0, bounds: row.bounds }
|
||||
peek next row:
|
||||
if next.type == QUANTITY_MODIFIER → defer emit (handled above)
|
||||
else → append pending_item flagged needs_review, clear pending
|
||||
continue
|
||||
|
||||
if row.type == ITEM:
|
||||
if pending_item:
|
||||
items.append(pending_item with needs_review if no price)
|
||||
pending_item = { name: row.name, unit_price: row.price,
|
||||
quantity: 1, line_total: row.price,
|
||||
discount: 0, bounds: row.bounds }
|
||||
peek next row:
|
||||
if next.type == QUANTITY_MODIFIER → defer emit (modifier overrides qty/unit)
|
||||
else → append pending_item, clear pending
|
||||
continue
|
||||
|
||||
if row.type == DISCOUNT:
|
||||
... (unchanged: attach to pending_item or receipt.total_discount)
|
||||
|
||||
if row.type in (SUBTOTAL, TOTAL):
|
||||
flush pending_item
|
||||
if is_grand_total(row):
|
||||
receipt.total = row.amount
|
||||
break
|
||||
|
||||
flush any trailing pending_item
|
||||
```
|
||||
|
||||
Each emitted item keeps **source bounding box(es)** (normalized 0–1 on the
|
||||
receipt image) for highlight overlays in the review UI.
|
||||
|
||||
## 5. Validation pass (do this after parsing, before showing results to user)
|
||||
|
||||
1. Sum `line_total` across all parsed items.
|
||||
2. Add `receipt.total_discount` if it wasn't already folded into
|
||||
individual item discounts.
|
||||
3. Compare the result to `receipt.total` (the grand total line).
|
||||
4. If they match within 0.01 → high confidence, no flags needed.
|
||||
5. If they don't match → mark the WHOLE receipt as "needs review" and
|
||||
surface it clearly in the review/edit screen (don't just flag
|
||||
individual lines — a mismatch means something in the whole parse is
|
||||
probably off, e.g. a missed line or a misclassified discount).
|
||||
4. If the receipt prints an item count (e.g. `Aantal Artikel: N`), compare
|
||||
`N` to the number of parsed items.
|
||||
5. If totals match within 0.01 **and** item count matches (when present)
|
||||
**and** no item is flagged `needs_manual_review` → high confidence.
|
||||
6. If any check fails → mark the **whole receipt** as "needs review".
|
||||
|
||||
## 6. Known store-specific quirks to encode as configurable rules
|
||||
|
||||
|
||||
Reference in New Issue
Block a user