Contract analysis is a document extraction problem with unusually high stakes: a missed auto-renewal clause or an unnoticed liability cap costs real money, so the agent's job isn't to "summarize the contract" in prose but to extract specific structured fields — parties, term dates, renewal terms, payment terms, liability caps, termination conditions — each traceable to the exact clause it came from.
Structured extraction, not free-text summary
A prose summary is pleasant to read and useless to build automation on top of. Define a schema of the fields that matter for your workflow (renewal date, auto-renewal flag, notice period, cap on liability, governing law) and have the model extract each one via a tool call, with a required citation — the exact sentence or clause the value came from. If the model can't point to a specific clause supporting a field, that field is left null and flagged for manual review rather than guessed.
{
"name": "extract_contract_terms",
"input_schema": {
"type": "object",
"properties": {
"renewal_date": {"type": ["string", "null"], "format": "date"},
"auto_renewal": {"type": "boolean"},
"notice_period_days": {"type": ["integer", "null"]},
"liability_cap": {"type": ["string", "null"]},
"citations": {
"type": "object",
"description": "field name -> exact source clause text",
"additionalProperties": {"type": "string"}
}
},
"required": ["auto_renewal", "citations"]
}
}Chunking long contracts without losing cross-references
Enterprise contracts run 20-80 pages, and clauses reference each other — a liability cap in section 9 might be modified by an amendment in an exhibit. Splitting the document into independent chunks for embedding and retrieval breaks those cross-references if you're not careful. A workable approach: chunk by logical section (defined by the document's own headings) rather than fixed token windows, retrieve the sections relevant to each field being extracted, and always include any amendment or exhibit sections regardless of the primary query, since they routinely override the base terms.
If an amendment exists, its terms take precedence — but a naive retrieval step can easily return only the original clause. Explicitly detect and prioritize amendment/addendum sections in the retrieval step rather than treating all chunks as equally authoritative.
A review gate for anything above a risk threshold
Not every extracted contract needs a lawyer's eyes, but some do — an unusually short termination notice period, an uncapped liability clause, or payment terms outside your standard range are worth flagging automatically. Define these as rule-based checks that run against the extracted structured fields (not a second LLM call asking "is this risky") so the flagging logic is deterministic and auditable, and route flagged contracts to legal review before they're approved in the workflow.
Extraction and flagging are the agent's job. Approving terms, signing, or advancing a contract to execution needs a human, every time — the cost of a bad contract term dwarfs the cost of a slower review cycle.
Evaluating extraction accuracy field by field
Build a golden set of contracts you've already had manually reviewed, with the correct field values recorded, and score the agent's extractions against them per field rather than as one blended accuracy number. Renewal dates might extract at 98% accuracy while liability cap language — often buried in dense boilerplate — extracts at 80%. That per-field breakdown tells you where to add stricter citation requirements or route more aggressively to human review, instead of treating the whole system as one pass/fail unit.
| Field type | Extraction approach |
|---|---|
| Dates, parties, dollar amounts | High-confidence extraction, spot-check sample |
| Liability/indemnification clauses | Extract + always route to human review |
| Amendment-modified terms | Explicit amendment-priority retrieval |
| Anything with no clear citation | Null field, flagged, never guessed |
Wrapping up
The value of a contract analysis agent is in turning unstructured legal text into fields your systems can act on, with every value traceable to its source clause. Keep the schema tight, treat amendments as authoritative, and route anything above a risk threshold to a human before it moves forward.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.