Data / ML · Data-mesh

Data Mesh Architecture — A Field Guide

Data Mesh Architecture — A Field Guide is the work that turns raw data into decisions. The pipeline from "we have data" to "we have a model that runs in production" is the same in.

John Kihiu12 min read

Data mesh, as Zhamak Dehghani originally defined it, is an organisational answer to a scaling problem, not a new storage technology. A single central data team cannot keep up with data requests from a growing company; data mesh distributes ownership of data to the domain teams that generate it, while keeping enough shared standards that the result is still usable across the company.

The four principles, briefly

Domain-oriented ownership means the team that generates order data — not a central data engineering team — owns the pipeline that turns it into an analytics-ready dataset. Data as a product means that dataset gets treated like a product: documented, versioned, with an owner accountable for its quality and an SLA for freshness. Self-serve infrastructure means domain teams do not each build their own pipeline tooling from scratch; a platform team provides the paved road (ingestion, orchestration, warehouse access) that domain teams build on. Federated computational governance means standards — naming conventions, PII handling, interoperability — are enforced automatically rather than by a central team reviewing every dataset by hand.

What a "data product" actually looks like

In practice a data product is a small set of well-modeled tables (often built with dbt), a schema contract, ownership metadata, and documented SLAs, exposed through the same warehouse everyone already queries. It is not a separate database per team — that just recreates data silos with extra steps. The domain team owns the transformation logic that produces it; the platform team owns the infrastructure it runs on.

YAML · DATA PRODUCT CONTRACT
# schema.yml — orders domain data product
models:
  - name: orders_data_product
    description: "Order-level facts, owned by the Orders domain team."
    meta:
      owner: orders-team@company.com
      sla_freshness_hours: 4
      pii: false
    columns:
      - name: order_id
        tests: [unique, not_null]
      - name: customer_id
        tests:
          - relationships:
              to: ref('customers_data_product')
              field: customer_id
Data mesh is not free — it is a tradeoff

Distributing ownership means distributing the work of modeling, testing, and documenting data. A ten-person company adopting data mesh usually ends up with worse data quality than a small central team would have produced, because domain teams do not have the bandwidth or expertise to treat their pipeline as a product. Data mesh earns its cost at the scale where a central team has become the bottleneck, typically dozens of domain teams and a data platform team big enough to build real self-serve tooling.

The warehouse does not go away

A common misconception is that data mesh replaces the central data warehouse with fully distributed storage. In most real implementations the warehouse (Snowflake, BigQuery, Databricks) stays as the shared physical layer; what changes is who owns which schemas and models within it, and how discoverable those models are through a data catalog. Mesh is an ownership and governance model layered on largely the same infrastructure you already have.

Signs you are not actually ready for it

If your company has one data team and a handful of domains, you do not need data mesh — you need better modeling practices and a data catalog. If domain teams have no engineers who can own a dbt project, federating ownership just means the pipelines rot unattended. The clearest signal you are ready is a central data team that has become a queue: every new dataset request waits weeks because one team is the only one who can build it.

PrincipleWhat breaks without it
Domain ownershipCentral team stays the bottleneck
Data as a productDatasets have no owner, quality drifts silently
Self-serve platformEvery domain reinvents pipeline tooling
Federated governancePII and naming standards diverge per team

Wrapping up

Data mesh solves an organisational bottleneck, not a technical one — adopt it when a central team can no longer keep pace with domain data needs, and only after you have the platform tooling to make self-serve ownership realistic. For most companies, the warehouse stays the same; what changes is who is accountable for each table in it.

John Kihiu
Acumatica ERP Developer · Laravel Engineer

Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.