Fine-Tuning LLMs for ERP Use Cases

Fine-tuning gets reached for far too early on ERP projects, usually because the word sounds like the "serious" version of prompting. In practice, most of what people want from a fine-tuned model — knowing what a purchase order is, following your chart of accounts, citing the right screen names — is a retrieval and prompting problem, not a training problem. Fine-tuning earns its cost when the issue is style, format, or behavior a frontier model won't hold from prompting alone, not when the issue is facts.

What fine-tuning is actually for

A base or instruction-tuned model already knows what an AP bill is in general terms. It does not know your company's approval hierarchy, your vendor list, or last week's invoice status — and fine-tuning is a bad way to teach it those things. Facts injected via fine-tuning go stale the moment the underlying data changes, and there's no way to update a single fact without a retraining run. RAG solves that: put the current data in a vector store or query it live, retrieve what's relevant, and let the model reason over fresh context on every call.

Fine-tuning earns its keep on behavior that's hard to specify in a prompt and needs to be consistent across thousands of calls: always responding in a fixed JSON shape your downstream parser expects, matching the terse tone of your support team's actual replies, correctly using a niche internal vocabulary (SKU codes, internal ticket categories) that shows up too rarely in the base model's training data to be reliable from few-shot examples alone, or reproducing a classification behavior more cheaply than a long system prompt would at scale.

RAG vs. fine-tuning: the actual decision

If you're unsure, default to prompting a frontier model with good retrieval and few-shot examples first, and only fine-tune once you have evals showing prompting has plateaued below the bar you need.

Practical approaches: hosted API vs. LoRA/QLoRA

For most teams, the OpenAI fine-tuning API (or Anthropic's equivalent programs where available) is the pragmatic starting point: upload a JSONL dataset of input/output pairs, kick off a job, get back a model ID you call like any other. No infrastructure to run. The tradeoff is you're fine-tuning someone else's model on someone else's schedule, subject to their deprecation timeline.

For open-weight models (Llama, Mistral, Qwen) run on your own infrastructure, LoRA or QLoRA are the standard approach — instead of updating all the weights, you train a small set of low-rank adapter matrices and merge or load them at inference time. QLoRA adds quantization so this is feasible on a single consumer or mid-tier GPU rather than a cluster. This path costs more engineering time up front but gives you a model you control fully, with no vendor deprecation risk and predictable per-token cost at scale.

{"messages": [
  {"role": "system", "content": "You are an AP support assistant. Respond in the exact ticket-reply format used by the finance team."},
  {"role": "user", "content": "Vendor is asking why bill INV-88213 hasn't been paid yet."},
  {"role": "assistant", "content": "Hi [Vendor],\n\nThanks for checking in. INV-88213 is currently in the approval queue (2 of 3 approvals received) and is scheduled for the next Thursday payment run. You'll get a remittance email once it clears.\n\nBest,\nAP Team"}
]}

Building a training dataset from real interactions

The dataset is the actual work; the training run is a few lines of API call. Pull 300-1000+ real support tickets, AP email threads, or logged agent interactions, filter out the ones that were handled badly (don't teach the model your own mistakes), and normalize them into consistent input/output pairs. Deduplicate near-identical tickets so the model doesn't just memorize one phrasing. Have a second person spot-check a sample for quality before training — a dataset built from unreviewed historical tickets will faithfully reproduce whatever inconsistencies your support team had.

The maintenance burden nobody prices in upfront

A fine-tuned model is a dependency you now own. When the provider deprecates the base model version you tuned against — which happens on a predictable cadence with hosted fine-tuning APIs — you have to retrain against the new base, re-run your eval suite, and requalify before the old version is retired. That's recurring engineering cost that a prompted frontier-model call doesn't carry, since the vendor swaps the model behind the API for you. Budget for this before committing to fine-tuning as a long-term architecture, not just for the initial training run.

Wrapping up

Reach for RAG and better prompting first; they cover the majority of "the model doesn't know our ERP" complaints and stay current automatically. Reach for fine-tuning when the gap is measurably in style, format, or narrow-domain vocabulary that prompting can't hold consistently — and go in accepting that you're taking on a retraining and re-evaluation cycle every time the base model changes underneath you.