Kafka disk access while reading and writing

167 Views Asked by At

According to Kafka doc https://kafka.apache.org/documentation/#persistence, Kafka utilities squential read/write performace of spnning disks and hence have high throughput. But how can this high throughput be maintained when node1 with disk1 is sequentially writing in paritition but a list of consumers also reading from that parition and disk1 have to move disk head to read partition.

Shouldnt this parallel reads and write in a spinning disk get into way of achieving high throughput?

1

There are 1 best solutions below

5
OneCricketeer On

Kafka uses sendfile system calls, pagecache, and zero-copy transfer for consumers, as explained in the efficiency section of the docs.

Also, logs.dir config can use multiple volumes (i.e. multiple physically mounted disks). It is true that having one disk is expected to be slower than many, in parallel.