Apache Airflow for ERP Workflows

ERP workflows are a good match for Airflow's execution model: they're batch-shaped, dependency-heavy, and need a retry history you can actually audit. Pulling sales orders, posting AP batches, or reconciling inventory between an ERP and a warehouse are all DAGs in the literal sense — a set of tasks with dependencies, not a stream. The failure modes that matter are different from a typical analytics DAG, though, because ERP APIs are slower, more rate-limited, and less tolerant of duplicate writes.

Modeling ERP syncs as DAGs

The natural unit of work is one DAG per entity type — orders, invoices, inventory adjustments — rather than one giant DAG that walks the whole ERP schema. Each entity sync typically breaks into extract (pull changed records since the last watermark), stage (land raw JSON or XML, unmodified, somewhere durable), and load (upsert into the warehouse or push transformed records back into the ERP). Keeping extract and load as separate tasks, rather than one fused step, means a transform bug doesn't force you to re-hit the ERP API to retry it.

Idempotency against ERP write APIs

Most ERP integration failures aren't extraction failures, they're double-posting failures from ambiguous retries. If a task pushes a batch of AP bills and times out waiting for a response, you don't know if the ERP received and processed it or not. Design the write task around an idempotency key — many ERP REST endpoints (Acumatica's contract-based API included) accept a client-supplied reference number that lets a retry either update the existing record or safely no-op instead of creating a duplicate.

from airflow.sdk import task
import hashlib

@task(retries=3, retry_delay=60)
def post_ap_batch(batch: dict):
    # deterministic key from batch contents, not a random uuid,
    # so a retried task reuses the same key as the first attempt
    idem_key = hashlib.sha256(
        f"{batch['vendor_id']}:{batch['period']}".encode()
    ).hexdigest()[:16]

    resp = erp_client.post(
        "/entity/Default/23.200.001/Bill",
        json=batch,
        headers={"Idempotency-Key": idem_key},
    )
    resp.raise_for_status()
    return resp.json()["id"]

Rate limits and connection pooling

ERP APIs commonly cap concurrent sessions far lower than a typical SaaS API — Acumatica's default license, for instance, limits concurrent API sessions per endpoint. Airflow's pools exist for exactly this: create a pool sized to the ERP's actual concurrency limit and assign every task that hits that ERP instance to it, so parallel task instances queue instead of tripping the ERP's own throttling or session limits.

Sensors versus polling for ERP batch windows

Some ERP processes — period close, batch posting jobs — run on the ERP's own schedule, not Airflow's. Rather than polling with a classic Sensor (which occupies a worker slot for the whole wait), use a deferrable sensor so the task suspends and frees the worker while it waits, or move to asset-based triggering if the ERP can push a webhook when the batch closes. Airflow 2.4+ handles both patterns natively; picking the wrong one is the usual reason a small Airflow deployment runs out of worker slots during a nightly ERP window.

Wrapping up

Airflow's value for ERP workflows is less about scheduling — cron could do that — and more about the retry, backfill, and observability semantics that come for free with a DAG run history. Model each entity as its own DAG, treat idempotency as a first-class design constraint rather than an afterthought, and size pools to the ERP's real limits, not Airflow's defaults.