KAFKA recommendation and High level understanding of kafka
Introduction Below document briefs about the Kafka best practices and recommendation for optimum performance. Document also touches configuration basics and concept of Kafka. Kafka services disk recommendations Recommend is using multiple drives to get good throughput and not sharing the same drives used for Kafka data with application logs or other OS filesystem activity to ensure good latency. With the available disks either RAID these drives together into a single volume or format and mount each drive as its own directory. Since Kafka has replication the redundancy provided by RAID can also be provided at the application level. This choice has several tradeoffs. a) Multiple disk and mount points With this option If you configure multiple data directories partitions will be assigned round-robin to data directories. Each partition will be entirely in one of the data directories. If data is not well balanced among partitions this can lead to...