Apache Kafka

Apache Kafka is an open source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. Producers write streams of events to topics and consumers read them, in real time or replayed later.

Events are stored durably as an ordered, append-only log, partitioned and replicated across a cluster of brokers for scale and fault tolerance. On top of this, Kafka Streams offers a library for stateful stream processing, and Kafka Connect provides ready-made connectors to move data in and out of external systems.

Written in Java and Scala, Kafka runs as a self-hosted cluster and is governed by the Apache Software Foundation under the Apache License 2.0.