Acumatica · Lake

Data Lake vs Data Warehouse

Data Lake vs Data Warehouse is the work that turns raw data into decisions. The pipeline from "we have data" to "we have a model that runs in production" is the same in every.

John Kihiu12 min read

"Data lake or data warehouse" is usually the wrong question — most teams that ask it end up needing both, connected by a clear boundary. The useful question is which workloads belong on which side, and how data moves between them without turning into an unmaintainable tangle of one-off exports.

What each one is actually for

A data warehouse stores structured, modeled data — usually in star or Kimball-style schemas — optimised for known, repeatable queries: dashboards, finance reporting, anything a BI tool hits every day. Schema is enforced on write, so bad data mostly gets rejected before it lands. A data lake stores raw or semi-structured data — JSON events, Parquet files, logs, CSV drops — in cheap object storage (S3, ADLS, GCS) with schema applied on read. It is built for volume and flexibility, not for a business user typing SQL against it directly.

The tradeoff is upfront cost versus deferred cost. A warehouse costs more to load into because you model and validate first. A lake costs less to load into but pushes that cost onto every consumer, who now has to figure out the schema themselves. Teams that skip the warehouse entirely and let analysts query raw lake files usually end up reinventing a warehouse badly, one ad hoc notebook at a time.

The lakehouse is not a third option

Lakehouse formats — Delta Lake, Apache Iceberg, Apache Hudi — exist to close this gap by putting warehouse guarantees (ACID transactions, schema enforcement, time travel) on top of lake storage. That is genuinely useful, but it does not remove the modeling work; it just means the modeling can happen in the same storage layer instead of a separate warehouse. If you are evaluating a lakehouse, the real question is whether your query engine (Snowflake, Databricks, BigQuery, Trino) actually gets the performance benefits of the table format, or whether you are just storing lake files with extra metadata and calling it a warehouse.

Storage is cheap, re-processing is not

The reason lakes keep raw data around instead of deleting it after transformation is replay. If a transformation bug corrupts three months of warehouse tables, having the untouched raw source in the lake means you can rebuild from scratch. Deleting raw data to save storage costs is usually a false economy the first time you need to backfill.

A concrete architecture that avoids the mess

The pattern that holds up: land everything in the lake first, in its rawest usable form, partitioned by ingestion date. Transform from lake to warehouse with a tool that tracks lineage — dbt is the common choice — so every warehouse table has a traceable path back to a raw source. Nothing writes directly into the warehouse except the transformation layer; ad hoc scripts and manual inserts are how warehouses drift out of sync with their own documentation.

SQL · WAREHOUSE LOAD PATTERN
-- staging model reads raw lake-backed external table
CREATE OR REPLACE VIEW staging.stg_orders AS
SELECT
    order_id,
    customer_id,
    CAST(order_ts AS TIMESTAMP) AS order_ts,
    CAST(amount_cents AS BIGINT) / 100.0 AS amount
FROM raw.orders_external  -- points at s3://lake/orders/*.parquet
WHERE order_id IS NOT NULL;

-- warehouse fact table, only written by the transform job
CREATE TABLE warehouse.fct_orders AS
SELECT * FROM staging.stg_orders
WHERE order_ts >= DATEADD(year, -2, CURRENT_DATE);

Choosing for your actual workload

Pick a warehouse-first approach when most consumers are BI tools, finance, or anyone who needs consistent, governed numbers and cannot tolerate a schema changing under them. Pick a lake-first approach when the dominant workload is machine learning, ad hoc data science, or ingesting high-volume semi-structured data (clickstreams, IoT, logs) where you genuinely do not know the eventual schema yet. Most mid-size companies need both, which is why "lake vs warehouse" is really a question of sequencing and boundaries, not an either/or choice.

DimensionData warehouseData lake
SchemaEnforced on writeApplied on read
Primary usersAnalysts, BI toolsData scientists, engineers
Cost profileHigher compute, structured storageCheap storage, cost shifts to consumers
Failure modeRigid, slow to changeBecomes an unqueryable swamp without governance

Wrapping up

Land raw data in the lake, model it into the warehouse through a tracked transformation layer, and resist the urge to let either side become the only source of truth. The architecture that survives three years of schema changes is the one where raw data is replayable and the warehouse tables have a clear, auditable lineage back to it.

John Kihiu
Acumatica ERP Developer · Laravel Engineer

Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.