Kafka Lab: Interactive Simulations & Lessons
Logs, partitions, keys, consumer groups and ordering. A hands-on kafka lab with 3 interactive simulations and 5 lessons.
Logs, partitions, keys, consumer groups and ordering.
Kafka is a distributed append-only log. Almost every surprise people hit with it — out-of-order events, duplicates, stuck consumers, uneven load — comes from four ideas: partitions, keys, offsets and consumer groups. This lab lets you break each one.
Simulations
- Kafka Partitions & Keys — Produce keyed messages, change partition count, and see how keys map to partitions and what that does to ordering.
- Consumer Groups & Rebalancing — Add and remove consumers, crash one mid-batch, and watch partition assignment, offsets, lag and redelivery.
- Idempotency Keys & Duplicate Payments — Click “Pay” on a flaky network, retry, and see duplicate charges disappear once idempotency keys are enabled.
Lessons
- Logs & Kafka: Partitions, Offsets and Ordering — Level 8 · Data Systems
- Consumer Groups & Delivery Semantics — Level 8 · Data Systems
- Batch vs Stream Processing & Data Pipelines — Level 8 · Data Systems
- Event Sourcing & CQRS — Level 8 · Data Systems
- Enterprise Integration Patterns: Messaging Between Systems — Level 5 · Enterprise Architecture
Also explore
- Build a Mini Message Broker — Build it yourself
- Competing Consumers — Pattern
- Idempotent Receiver — Pattern
- Dead Letter Channel — Pattern
Other labs
- Database Internals Lab — Storage engines, indexes, replication and partitioning.
- Transactions Lab — Isolation anomalies, lost updates and distributed transactions.
- Caching Lab — Hit rates, TTLs, invalidation, stampedes and CDNs.
- Microservices Lab — Failures, retries, circuit breakers, sagas and idempotency.
- Architecture Lab — Evolution, decisions, characteristics and fitness functions.