AI Agent for Lead Scoring in Acumatica CRM

Lead scoring is a narrow, well-bounded problem, which is exactly why it's a good first agent to ship: the input is a finite set of CRM fields, the output is a number and a reason, and you can validate both against what your sales team already believes about a lead. The mistake teams make is asking the LLM to "score the lead" end to end. The better design separates feature extraction, which the LLM is genuinely good at, from the scoring function, which should stay deterministic and auditable.

Why not just a prompt

Feeding a CRM record into an LLM and asking for a 0-100 score works in a demo and falls apart in production. The score isn't reproducible — ask twice, get two numbers, even at temperature 0 the model's rounding and phrasing drift enough to matter when a rep is comparing two leads side by side. It isn't explainable to a sales manager who wants to know why a lead dropped from 80 to 40 overnight. And it's impossible to tune: if the VP of sales says "we're underweighting company size," there's no lever to pull inside a single prompt that produces a scalar.

The fix is to split the job. The LLM's job is feature extraction: turning messy, unstructured CRM data — free-text notes, email threads, call transcripts, form fill responses — into a structured set of signals. A plain weighted scoring function, or a small trained classifier, turns those signals into the number. The LLM never sees "score this lead 0-100"; it sees "extract these twelve fields from this record."

Feature extraction with structured output

Use the provider's structured output mode rather than asking for JSON in the prompt and hoping. Anthropic's tool-use with a forced tool call, or OpenAI's response_format: json_schema with strict mode, both guarantee the shape comes back parseable — no more regex-stripping markdown fences out of a chat response. Define a schema with fields like budget_signal, urgency_signal, decision_maker_identified, competitor_mentioned, and a short evidence_quote per field so a human can spot-check the extraction against the source text.

from anthropic import Anthropic

client = Anthropic()

lead_schema = {
    "name": "extract_lead_signals",
    "description": "Extract qualification signals from CRM notes",
    "input_schema": {
        "type": "object",
        "properties": {
            "budget_signal": {"type": "string", "enum": ["confirmed", "implied", "none"]},
            "urgency_signal": {"type": "string", "enum": ["high", "medium", "low", "none"]},
            "decision_maker_identified": {"type": "boolean"},
            "company_size_estimate": {"type": "integer"},
            "evidence_quote": {"type": "string"}
        },
        "required": ["budget_signal", "urgency_signal", "decision_maker_identified"]
    }
}

resp = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=512,
    tools=[lead_schema],
    tool_choice={"type": "tool", "name": "extract_lead_signals"},
    messages=[{"role": "user", "content": crm_notes_text}]
)
signals = resp.content[0].input

That signals dict is the LLM's entire contribution. Everything downstream — weighting, thresholds, routing a lead to a rep — is ordinary code that a sales ops person can read and adjust without touching a prompt.

Scoring as a deterministic function, not a prompt

Once you have structured signals, scoring is a lookup table or a logistic regression, not another LLM call. This is the part teams skip because it feels like "less AI," but it's the part that makes the system trustworthy. A weighted sum over the extracted fields, with weights sales ops can tune in a spreadsheet, beats a black-box score every time a manager asks "why."

Handling sparse or contradictory CRM data

Real CRM records are inconsistent — half-filled forms, a call note that says "not interested" next to a follow-up email showing renewed interest. Don't let the model guess past what's actually there; force it to emit "none" or a null rather than inferring a signal it can't support, and make that an explicit instruction in the system prompt, not an assumption. Log every extraction alongside the source text so you can build an eval set from real disagreements between the model's read and what a rep later confirmed.

Evaluation and drift monitoring

Build a labeled set of 100-200 historical leads where you know the eventual outcome — closed-won, closed-lost, ghosted — and re-run extraction against it whenever you change the prompt or swap models. Track precision on the signals that matter most for routing, not just overall score correlation; a model that nails budget_signal but misses decision_maker_identified half the time will misroute leads even if its aggregate score looks fine. Re-score the eval set on every model version bump — a "minor" model update from a provider can shift extraction behavior enough to change routing outcomes for a meaningful slice of leads.

Wrapping up

Lead scoring agents earn trust by being boring where it counts: structured extraction from an LLM, arithmetic for the score, and a human in the loop for anything low-confidence or high-stakes. Resist the urge to let the model own the final number — the moment you do, you lose the ability to explain a score, tune a weight, or catch a prompt-injected note before it routes a bad lead to your best rep.