Logs & Kafka: Partitions, Offsets and Ordering
An append-only log as the backbone of data flow. Lesson 12 of Level 8: Data-Intensive Systems. Based on DDIA.
Level 8 · Data-Intensive Systems · Lesson 12 of 15
An append-only log as the backbone of data flow.
Continue learning
- Consistency Models, CAP & Consensus — Previous lesson
- Consumer Groups & Delivery Semantics — Next lesson
- Level 8: Data-Intensive Systems — Back to the level
Try it hands-on
- Kafka Partitions & Keys — Produce keyed messages, change partition count, and see how keys map to partitions and what that does to ordering.
- Kafka Lab — Logs, partitions, keys, consumer groups and ordering.
All lessons in Data Systems
- Storage Engines: B-Trees vs LSM-Trees
- Indexes: Fast Reads, Slower Writes
- Caching: Speed at the Cost of Truth
- Replication: Leaders, Followers & Failover
- Replication Lag: When Your Own Write Disappears
- Partitioning: Hash, Range, Hotspots & Rebalancing
- Transactions & What ACID Actually Promises
- Isolation Levels: Dirty Reads to Serializable
- Lost Updates: Optimistic vs Pessimistic Locking
- The Trouble With Distributed Systems
- Consistency Models, CAP & Consensus
- Logs & Kafka: Partitions, Offsets and Ordering
- Consumer Groups & Delivery Semantics
- Batch vs Stream Processing & Data Pipelines
- Event Sourcing & CQRS