The lakehouse pitch is simple: keep data in cheap object storage the way a data lake does, but get the transactional guarantees, schema enforcement, and query performance of a warehouse on top of it. Delta Lake, Apache Iceberg, and Apache Hudi are the three table formats that made this actually work, by adding a transaction log and metadata layer over plain Parquet files sitting in S3, GCS, or Azure Blob.
What a plain data lake was missing
A traditional data lake is just files in object storage — Parquet, ORC, or CSV, organized into directories, queried by whatever engine you point at them. That's cheap and flexible, but it has no concept of a transaction: two jobs writing to the same table at once can corrupt it, there's no schema enforcement so a bad write can silently introduce a column-type mismatch, and there's no way to see a consistent snapshot of a table while it's being updated. Warehouses solved all of this decades ago with ACID transactions, but at the cost of proprietary storage you can't cheaply query with arbitrary engines. The lakehouse closes that gap by adding a metadata/transaction layer on top of the same open file formats the lake already used.
The transaction log is the actual mechanism
Delta Lake, Iceberg, and Hudi all solve this the same fundamental way: a table isn't just "the files in this directory," it's a versioned log of which files belong to which version of the table. Delta Lake keeps a JSON+Parquet transaction log (_delta_log) of every add/remove file operation. Iceberg tracks table state through a chain of metadata and manifest files, with no dependency on a specific engine's log format. Hudi does something similar with its own timeline and indexing structure, and additionally supports upsert-optimized storage layouts (copy-on-write vs. merge-on-read) more explicitly than the other two. The practical effect for all three: readers get a consistent snapshot, writers get atomic commits, and you get time travel — querying the table as it looked at a previous version or timestamp.
-- Read the table as of a previous version
SELECT * FROM orders VERSION AS OF 42;
-- Or as of a timestamp
SELECT * FROM orders TIMESTAMP AS OF '2026-07-01 00:00:00';
-- MERGE for upserts, not possible on plain Parquet without this layer
MERGE INTO orders t
USING staged_orders s
ON t.order_id = s.order_id
WHEN MATCHED THEN UPDATE SET *
WHEN NOT MATCHED THEN INSERT *;Schema enforcement and evolution
All three formats enforce schema on write by default, rejecting a write that doesn't match the table's expected columns and types — the thing a raw Parquet-files-in-a-folder lake has no way to do. They also support schema evolution: adding a nullable column, widening a type, or renaming a field, tracked in the metadata rather than requiring a full table rewrite. This is what makes a lakehouse usable as the single source of truth for both ad hoc analytics and production ETL, instead of needing a separate curated warehouse layer just for the schema guarantees.
Delta Lake has the deepest integration with Databricks/Spark; Iceberg has the widest engine support (Spark, Trino, Flink, Snowflake, BigQuery all read Iceberg natively) and is the closest thing to a vendor-neutral standard; Hudi leans hardest into incremental/streaming upsert workloads. If you're not on Databricks and want to avoid lock-in, Iceberg is usually the safer default choice today.
Why separating compute from storage is the real win
The other half of the lakehouse pitch is that storage (cheap object storage) is decoupled from compute (Spark, Trino, Flink, or a warehouse's own engine querying the same files). A traditional warehouse bundles the two, so you pay warehouse-storage prices for cold data nobody's actively querying, and scaling compute independently of storage isn't always straightforward. With a lakehouse, the data sits in S3 at object storage prices regardless of which engine is reading it, and you can point three different query engines at the same Iceberg table without duplicating the data three times.
Streaming or frequent small writes to a lakehouse table produce many small Parquet files, which hurts read performance regardless of which table format you're using. All three formats support compaction (Delta's OPTIMIZE, Iceberg's rewrite procedures, Hudi's clustering), but none of them do it for you automatically — you need a scheduled compaction job, or read performance degrades over months without anyone noticing until a query that used to take seconds takes minutes.
When a plain warehouse is still the simpler answer
A lakehouse is the right call when you have large volumes of semi-structured or unstructured data, need multiple engines querying the same data, or are already paying warehouse-storage costs for data that's mostly cold. If your workload is a few terabytes of clean structured data queried almost exclusively by one BI tool, a managed warehouse (BigQuery, Snowflake, Redshift) with none of the table-format decisions is often genuinely less operational overhead — the lakehouse buys you flexibility and cost control at the price of managing compaction, file layout, and catalog infrastructure yourself.
Wrapping up
The lakehouse works because Delta Lake, Iceberg, and Hudi all solve the same problem the same way: a transaction log and metadata layer over open file formats, giving object storage the ACID guarantees and schema enforcement that used to require a proprietary warehouse. The trade-off is that compaction, catalog management, and format choice become your responsibility instead of a vendor's — worth it once you're running enough scale or enough different query engines that the flexibility pays for the extra operational surface.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.