For over a decade, running Kafka meant running ZooKeeper too — a separate ensemble whose sole job was storing cluster metadata and electing a controller. KRaft (Kafka Raft) removed that dependency by moving metadata management into Kafka itself, using the same Raft consensus algorithm Kafka already used for partition replication. As of Kafka 3.3+ KRaft is production-ready, and as of 4.0 ZooKeeper support was removed entirely — so understanding KRaft isn't optional anymore for anyone running a current Kafka version.
Why ZooKeeper was the thing to remove
ZooKeeper worked, but it meant operating two distinct distributed systems with different failure modes, different tuning knobs, and different operational runbooks, just to run one Kafka cluster. Metadata changes (topic creation, partition reassignment, ACL updates) went through ZooKeeper and then had to propagate to the controller and out to every broker — a path with real latency, especially at cluster sizes with thousands of partitions, where controller failover after a crash could take tens of seconds while the new controller reloaded full metadata state from ZooKeeper.
KRaft doesn't remove the concept of a controller — it removes ZooKeeper as the place that metadata lives. A small set of nodes run in the "controller" process role (forming a Raft quorum that stores the metadata log), while broker nodes handle client traffic. In smaller deployments, a node can run both roles combined.
What actually changed operationally
With KRaft, cluster metadata lives in an internal Kafka topic (__cluster_metadata) replicated via Raft across the controller quorum, the same replication mechanism Kafka already uses for regular topics. Controller failover is now a Raft leader election among nodes that already have the metadata log locally, instead of a full reload from an external system — which is why failover times dropped from tens of seconds to sub-second in most reported benchmarks. Operationally, this means one system to run, one set of nodes to secure and monitor, and one fewer moving part that can fall out of sync with Kafka's own view of the world.
process.roles=broker,controller
node.id=1
controller.quorum.voters=1@controller1:9093,2@controller2:9093,3@controller3:9093
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
controller.listener.names=CONTROLLER
inter.broker.listener.name=PLAINTEXTMigrating an existing ZooKeeper cluster
Kafka provided an online migration path (available from 3.4 onward) that lets a running ZooKeeper-based cluster move to KRaft without downtime: stand up a KRaft controller quorum, put it into migration mode so it reads from ZooKeeper as the source of truth temporarily, then cut brokers over one at a time to KRaft-managed metadata, and finally decommission ZooKeeper once every broker has migrated. The realistic risk isn't the mechanism, which is well-tested at this point — it's doing this on a cluster with a large number of partitions and topics without a rollback plan tested in staging first, since a partial migration state is harder to reason about than either endpoint.
Some older admin tooling, monitoring integrations, and third-party Kafka management UIs assumed ZooKeeper's existence for cluster introspection. Before migrating a production cluster, verify every tool in your operational stack — not just the Kafka clients — has KRaft support, since a monitoring gap post-migration is a worse surprise than the migration itself.
New clusters should just start on KRaft
For any cluster being stood up new, there's no argument for starting on ZooKeeper mode at this point — KRaft has been the default and the actively developed path for several major versions, and ZooKeeper mode is deprecated and removed outright in Kafka 4.0. The only reason to still be thinking about the migration path at all is an existing cluster that predates KRaft's production readiness.
Wrapping up
KRaft's real contribution isn't a new feature so much as the removal of an entire second distributed system Kafka operators had to run and reason about. Faster controller failover and simpler operations are the visible benefits; the bigger one is fewer places for cluster state to disagree with itself. If you're still running ZooKeeper mode, the online migration path is well-supported — the main risk is under-testing it on a cluster with real production scale before cutting over.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.