RAG for ERP Data — A Complete Guide

Retrieval-augmented generation over ERP data is a different problem than RAG over documents, and treating it the same way is the most common mistake. Acumatica records are structured, relational, and access-controlled at a granularity most vector-search tutorials never mention. Naively embedding rows and chunks and stuffing the top-k into a prompt gets you a demo that looks impressive and a production system that leaks data across branches or answers with stale numbers.

Structured data is not a document-chunking problem

A vendor bill isn't prose — it's a header, a set of line items, a GL distribution, and a status. Chunking it into 500-token blocks and embedding each chunk throws away the relational structure that makes the record meaningful in the first place. For anything with a clear schema (invoices, POs, GL transactions), skip the embedding step entirely and let the agent query the data directly through Acumatica's OData or contract-based REST endpoints, scoped by explicit filters. Reserve vector search for genuinely unstructured content: attached PDFs, email correspondence, free-text notes fields, knowledge-base articles.

Two retrieval paths, not one

The practical architecture has a router in front of two distinct retrieval mechanisms: structured queries against the live database for anything answerable by filtering and aggregation ("open AR balance for customer X"), and vector search over an indexed corpus of unstructured text for anything requiring semantic matching ("find correspondence mentioning a pricing dispute"). Trying to force both cases through embeddings means either bad aggregate answers (an LLM summing numbers from retrieved chunks, badly) or bad semantic search (SQL LIKE queries missing paraphrased matches).

Access control has to live below the retriever

Acumatica enforces row-level and branch-level security through its own access rights engine. A RAG layer that queries a replicated database or a search index without re-applying those same restrictions will happily surface another branch's AR data to a user who couldn't see it in the actual application. Retrieval must run as the requesting user, with the same restriction groups applied, not as a service account with blanket access. This is the single most common security gap in ERP-RAG builds, and it doesn't show up in testing unless someone specifically tests it with a restricted user.

def query_ar_balance(customer_id: str, requesting_user: str) -> dict:
    # Re-apply Acumatica's own restriction groups, not a service-account bypass
    allowed_branches = get_user_branch_access(requesting_user)
    if not allowed_branches:
        raise PermissionError("No branch access for this user")

    return odata_client.get(
        "ARBalances",
        filters={"CustomerID": customer_id, "BranchID": {"$in": allowed_branches}},
        auth=impersonate(requesting_user),
    )

Freshness beats recall for financial data

A vector index of ERP records is a snapshot the moment it's built, and financial data changes constantly — a stale cached balance is worse than no answer at all. For anything time-sensitive, query live rather than serving from an index, even if it's slower. Reserve indexing for content that changes slowly: item descriptions, historical closed transactions, policy documents. If you do index transactional data for semantic search, put a visible timestamp on every retrieved fact and re-index on a schedule tight enough that staleness can't silently produce a wrong answer.

Citations that actually point somewhere

Every answer synthesized from retrieved ERP data should carry a reference back to the source record — a document number and screen link the user can click to verify, not a vague "based on your records." This does double duty: it lets users catch mistakes before acting on them, and it forces the retrieval layer to track provenance instead of blending numbers from multiple documents into an unverifiable sentence.

Wrapping up

RAG over ERP data works when you stop treating every record as a document to embed and start treating structured data as a query problem and unstructured data as a search problem. Route by data type, re-apply Acumatica's own access control at retrieval time, keep financial aggregates out of the model's hands, and always cite back to the source record so a user can verify before they trust the answer.