Kafka interview questionsQuestion 1 of 2
Kafka interview question · Question 1 of 2
Explain Kafka partitions and consumer groups.
Short answer
A Kafka topic is split into partitions, each an ordered log; ordering is guaranteed only within a partition, and records with the same key go to the same partition. A consumer group is a set of consumers sharing a group id, and Kafka assigns each partition to exactly one consumer in the group, so partitions are divided among them. This means the partition count caps how many consumers in one group can work in parallel.
Detailed explanation
- Partitions are the unit of both ordering and parallelism. Each has its own sequence of offsets.
- Keys route records: the same key always maps to the same partition while the partition count is unchanged.
- Consumer groups split partitions among members. Two different groups each receive every record independently.
Example
A topic orders has 6 partitions.
Consumers in group billing |
Partitions per consumer |
|---|---|
| 1 | 6 |
| 3 | 2 each |
| 6 | 1 each |
| 8 | 6 consumers get 1, 2 are idle |
A second group analytics reads all 6 partitions on its own, unaffected by billing.
Failure and rebalancing
If a consumer crashes or stops sending heartbeats, the group rebalances and its partitions move to other members, which resume from the last committed offsets. Records processed but not yet committed are processed again, which is why at-least-once consumers need idempotent processing.
Common mistakes
- Claiming Kafka guarantees ordering across a whole topic.
- Adding consumers beyond the partition count to increase throughput.
- Picking a skewed key so one partition receives most of the traffic.
Progress is saved in this browser only. No account needed.