Kafka Partition & Storage Calculator
Plan your Apache Kafka topics with confidence. Enter your workload parameters to calculate the optimal partition count, total storage requirements, daily ingestion rate, and network bandwidth needed.
Workload Parameters
Total producer throughput across all partitions
Include key + value + headers overhead
168 hours = 7 days (Kafka default)
Cluster Configuration
Max parallelism = partition count
Number of copies of each partition
Typically 10-30 MB/s depending on message processing
Recommended Partitions
--
partitions per topic
Total Storage Needed
--
GB
Storage Per Broker
--
GB (assuming 3 brokers)
Daily Ingestion
--
GB/day
Network In (producers)
--
MB/s
Network Out (replication + consumers)
--
MB/s
Frequently Asked Questions
How many Kafka partitions should I use?▼
The number of partitions should be at least equal to the number of consumers in your consumer group for maximum parallelism. Beyond that, consider your target throughput: divide total required throughput by the throughput you can achieve per partition (typically 10-30 MB/s depending on message size and processing logic). Start with fewer partitions and scale up, as increasing partitions is easy but decreasing them requires topic recreation.
How does the replication factor affect Kafka storage?▼
The replication factor directly multiplies your storage needs. With a replication factor of 3 (the recommended production setting), every message is stored on 3 different brokers, tripling your total storage requirement. It also increases network bandwidth usage since data must be replicated across brokers. However, a replication factor of 3 is strongly recommended for production to survive broker failures without data loss.
How do I calculate Kafka storage requirements?▼
Kafka storage is calculated as: messages_per_second x average_message_size x retention_period_in_seconds x replication_factor. For example, 1000 messages/sec at 1 KB each with 7-day retention and replication factor 3 requires: 1000 x 1024 x 604800 x 3 = approximately 1.73 TB total. Add 10-20% overhead for indexes and log segment metadata.
What is a good Kafka retention period?▼
The default Kafka retention is 7 days (168 hours), which works well for most use cases. For event sourcing or audit logs, you may need longer retention (30-90 days or even unlimited with log compaction). For real-time streaming with fast consumers, 24-48 hours may suffice. Balance retention against storage costs: longer retention means more disk space. You can also set retention by size (bytes) in addition to time.