Invoice pipeline

Why

An invoice PDF, as submitted, tells a bank nothing usable. The pipeline turns this document into a structured, verified, and scored file — the condition for a financing request to even be submittable. It is the single entry point for any invoicing data into the rest of the platform.

How, in the application

Each pipeline transition is carried by a worker that consumes a dedicated Redis stream. The canonical macro-journey is UPLOADED → FILE_PREPROCESSED → OCR_DONE → FIELDS_EXTRACTED → VALIDATED → SCORED, but an invoice's actual state is more granular (about twenty statuses in total, including QUARANTINED and REJECTED), and each transition is checked by a state machine that refuses any unauthorized move.

• Antivirus — the first real step, before any processing of the content: the file is scanned (ClamAV); if infected, the invoice is quarantined and the pipeline stops; if clean, it moves to preprocessing.
• OCR — text is extracted from the document, with an integrity check of the file retrieved from object storage.
• Field extraction — structured data (amounts, dates, parties) is extracted, with a composite data-quality score (OCR confidence, extraction confidence, key-field completeness, arithmetic consistency: pre-tax + VAT = total).
• Validation and fraud detection — business rules are applied; a hard rejection occurs if the fraud score exceeds a threshold and a serious signal is present (supplier identical to buyer, due date earlier than issuance).
• Scoring — a risk grade is calculated and persisted: this is the terminal state SCORED, that of an "Analysée" (Analyzed) file, ready for review.

Safeguards

• A processing failure leaves the message pending, retried with backoff up to a limited number of attempts; beyond that, the message moves to an error queue rather than blocking the flow indefinitely.
• Each worker publishes a regular heartbeat — a stuck invoice first signals a stopped worker, not necessarily corrupted data.
• Every antivirus scan is audited and its result kept as evidence, independently of the rest of the processing.
• The SCORED state is today the pipeline's effective terminus: it makes the file reviewable for a financing request, but no longer produces, by itself, an automatic market listing.