Beyond the format war
Architecture, not debate, settled the lake versus warehouse argument: open table formats put ACID semantics, schema evolution, and time travel on object storage, which is what the original lakehouse paper proposed in 2021. The consequential question now is a different one: where does that context come from, when an agent needs enterprise context to make a decision, and what makes it safe to use?
This does not respond to access a model and a vector index. If a passage matches a query, it is a retrieval hit. It doesn’t say anything about whether the underlying record is current, whether the requesting identity is entitled to see it, which upstream system produced it, or what happens if the agent acts on it.
Where the OS Metaphor Works and Where it Doesn’t
Four operating-system responsibilities align directly with a mature data platform. Resource discovery: a catalogue that tells you what is, who owns it and what it means. Permissions: Central authorisation evaluation, not per engine. State: durable, versioned, transactionally consistent data with snapshot isolation Observability: lineage and audit of what was read and what changed
Three don’t. An OS schedules and isolates execution. A lakehouse doesn’t run your agent. An OS provides low-latency syscalls; lakehouse queries are seconds-scale, so they are out of a real-time decision path. And an OS has its own resources. The lakehouse has an analytical copy. The systems of record still live in Postgres, the ERP and the payment processor. The lakehouse as a kernel overpromises on all three counts. It's closer to a filesystem + a permission model + a package manager for context.
What the lakehouse actually provides
Precision matters here because vendor narratives blur it. The Iceberg specification and Delta Lake give you atomic commits, snapshot isolation, schema evolution and time travel at single-table scope. Multi-table transactions are not part of the model, as the comparative analysis of lakehouse storage systems makes explicit. A REST catalog such as Apache Polaris adds discovery and a governance boundary independent of the query engine.
What it does not provide: vector search (a separate index, whatever your platform bundles), stream processing (that is Flink or Kafka Streams), sub-second point lookups, workflow orchestration, model serving, or transactional writes to operational systems. An agent architecture that assumes otherwise will discover the gap under load.
The real product is trusted context
Usually, context, not reasoning, leads to a confidently wrong agent decision. Three failure modes reappear: Stale data (a claim status that is refreshed daily, read at 3pm as if it were fresh), semantic mismatch (three definitions of "active customer" in three marts, and the agent chooses whichever it pulls up), and duplication without provenance.
The mechanisms for dealing with these are unglamorous and already exist. Data contracts enforce schema and semantics as versioned agreements, not tribal knowledge. OpenLineage as the open standard for column-level lineage allows tracing an agent’s output to source systems. Machine-readable metadata should not be exposed by dashboards, but by freshness and quality checks. An agent needs to know when a table’s watermark is four hours old, and a well-designed retrieval layer will return that with the rows. A semantic layer provides you with the single definition of a metric that the agent is not allowed to redefine.
Design rule: every fact retrieved has provenance – source, snapshot id, freshness, sensitivity class. Context without provenance is good-latency rumour.
Reading is not doing
Most architectures are underspecified between reading data and acting on data. Three examples make the point.
The claims investigator is well-read: policy history, prior claims and adjuster notes, PII masked by policy, every conclusion citing snapshot-pinned rows. Batch freshness is fine, output is a recommendation for a human, and the lakehouse is really the center of that workload.
A fraud agent responding to a suspicious transaction is not . Features are needed in milliseconds for the decision, so the serving path is an online store fed by CDC and stream processing, while the lakehouse provides the historical baselines, training sets and post-hoc investigation, not the hot path. And blocking a card is a transaction on the payment system. It needs an idempotency key, an authorisation check on the agent’s own identity and a threshold above which a human decides.
Between the two is a supply-chain agent that responds to inventory changes. CDC from the ERP gives minutes-fresh state, but to create a purchase order you go through the API of the ERP and a workflow engine. The key distinction to remember is that an agent writing to a Delta table is mutating an analytical copy . An agent placing an order is spending money . Only the first is an operating data platform.
An architecture reference
Operational systems publish changes through CDC, streaming and batch pipelines, into governed open-format tables. A catalogue contains ownership, contracts, lineage, freshness and classification. It’s got a retrieval layer over it: structured questions through an engine like Trino, a retrieval layer for documents with vector or hybrid search, and an online store for low-latency features — three different paths that are selected based on the query’s latency and precision requirements, not all in one “RAG” box. Through this layer only the agent runtime consumes. Only under its own identity.
Actions go a very different route: a tool gateway that enforces schemas, per-call authorisation and idempotency, then APIs, workflow engines or transactional systems. The model doesn’t alter core data, and outcomes go back through CDC to close the loop. Inside: Deterministic infrastructure, probabilistic decision element.
The bottleneck risk and governance
An agent should not inherit the table access of a data scientist nor the session rights of a user. It needs to have a distinct identity, delegated and time-bounded authority, row- and column-level policy enforced in the catalogue and engine, and purpose limitation – entitlement to read claims for adjudication is not entitlement to read them for marketing. Policy enforcement should happen at the boundary, in a policy engine such as OPA, not in a system prompt; and since retrieved content re-enters the reasoning loop, prompt injection via documents is a data governance problem as much as a model one.
The real danger is decentralisation. Agents query much more often than analysts, with unpredictable concurrency, and each retrieval costs money. An agent's beliefs can silently change with a schema-evolution-tolerant dashboard. If each agent request has to go through the backlog of one platform team, the lakehouse becomes the bottleneck the data lake was supposed to eliminate.
The defensible version of the thesis is that the lakehouse is not becoming an OS. It is the governed, discoverable, constantly updated context layer that agents pull trusted information from and, when properly authorised, reach into real workflows via systems built for the purpose. The strategic part was never storage. Trust is.
