AI Agent for Expense Report Processing

Expense report processing looks like a natural fit for an LLM agent — receipts are unstructured, policy rules are text, and the volume is high enough that manual review is genuinely tedious. It's also a domain where a confidently wrong extraction turns into a reimbursement for a fake or inflated receipt, so the design has to assume the model will misread something and build the checks around that assumption rather than around the happy path.

Receipt extraction: OCR plus structured output, not OCR alone

Raw OCR gives you text; it doesn't give you "this is the total, this is the merchant, this is the date." Feed the receipt image directly to a vision-capable model (Claude or GPT-4o's image input) with a structured output schema — merchant name, date, subtotal, tax, total, currency, line items where legible — rather than running a separate OCR pass and asking a text model to parse the output. Vision models handle skewed photos, thermal-paper fading, and handwritten tips far better than a classic OCR-then-regex pipeline, and you get one fewer place for information to get lost between steps. Always return a confidence signal per field, even if it's just the model's own self-reported certainty — you need it two sections down for the approval-routing decision.

Policy validation as deterministic rules, not model judgment

Per-diem limits, category caps, and receipt-required thresholds are exact numbers from a policy document — encode them as data (a rules table keyed by expense category, country, and role) and evaluate them in code, not by asking the LLM "does this comply with policy?" on every submission. The LLM's job stops at extraction: category, amount, currency, date. A deterministic rules engine then checks the extracted values against the policy table and produces a pass/fail plus reason per rule. This also means policy changes — a new per-diem rate, a new category — are a data update, not a prompt rewrite, and they take effect identically for every submission instead of depending on how the model happened to interpret that day's prompt.

def validate_expense(extracted: ExpenseFields, policy: PolicyTable) -> list[Violation]:
    violations = []
    limit = policy.per_diem_limit(extracted.category, extracted.country)
    if limit and extracted.amount > limit:
        violations.append(Violation(
            rule="per_diem_exceeded",
            detail=f"{extracted.amount} {extracted.currency} exceeds "
                    f"{limit} limit for {extracted.category}"
        ))
    if extracted.amount > policy.receipt_required_threshold and not extracted.has_receipt_image:
        violations.append(Violation(rule="receipt_missing"))
    if extracted.category == "alcohol" and not policy.allows_client_entertainment:
        violations.append(Violation(rule="disallowed_category"))
    return violations

Matching to corporate card feeds

Where a corporate card program exists, match each submitted receipt against the card transaction feed by amount, merchant name (fuzzy-matched — "SQ *COFFEE SHOP" vs. "Coffee Shop Nairobi" need normalization), and date within a small window (card postings often lag the purchase by a day or two). A confirmed match is strong evidence the expense is real and correctly amounted; an unmatched submission isn't necessarily fraudulent — cash payments and personal-card-then-reimburse are normal — but it's a signal to weight into the approval decision rather than ignore.

Flagging duplicates and anomalies

Duplicate detection should run on more than just an exact hash of the receipt image, since the same physical receipt gets photographed twice, or a scanned copy and a phone photo of the same purchase both get submitted. Check for same-merchant-same-amount-same-date across a submitter's recent history, and separately hash the image itself to catch literal re-uploads. For anomalies, a handful of simple rules go a long way before you need anything statistical: amounts just under an approval threshold submitted repeatedly, weekend dates on categories that are normally weekday-only, or a sudden jump in a submitter's average expense amount relative to their own trailing baseline. These are cheap, explainable checks — save a full anomaly-detection model for later if the simple rules aren't catching enough.

Auto-approval thresholds and routing to a human

Auto-approve only when every signal agrees: high extraction confidence on all required fields, zero policy violations, and (if applicable) a confirmed card-feed match, under a low dollar threshold tuned conservatively at launch and raised only after you've measured the false-approval rate on a review sample. Anything with a policy violation, a low-confidence extraction field, an unmatched high-value transaction, or an anomaly flag routes to a human approver's queue with the extracted data pre-filled and the specific reason for the flag shown — the reviewer's job becomes confirming or correcting one flagged issue, not re-keying the whole report from scratch.

Audit trail requirements

Every expense that goes through the agent needs a record of what the model extracted, its confidence per field, which policy rules ran and their results, whether a card-feed match was found, and — critically — whether the final action was an automatic approval or a human decision, with which human and when. Auto-approved expenses are exactly the ones that get pulled for audit later, so the log needs to stand on its own without anyone having to reconstruct "why did the agent approve this" from memory. Treat the audit log as a first-class output of the pipeline, written at the same time as the approval decision, not bolted on afterward.

Wrapping up

The pattern that holds up is the same one that holds up for any high-volume financial process: let the model do what it's actually good at — reading messy, unstructured input into structured fields — and keep every dollar-bearing decision in deterministic code that you can test, log, and explain to an auditor. The auto-approval threshold is the dial you tune over time; the audit trail and the policy engine are the parts you get right on day one.