Categories

Tools

RabbitMQ 4.3 streams vs quorum queues, measured

A RabbitMQ quorum queue deletes each message once it is acknowledged and fsyncs before accepting it; a stream keeps it for rereading. On RabbitMQ 4.3.6, with 1 KB messages, the quorum queue moved 45,156 msg/s and the stream, over its own protocol, 281,330 msg/s, using 9 KB of memory per million messages against 20.5 MB.

Architecture

Kafka without ZooKeeper: KRaft in production

To remove ZooKeeper from Kafka you have to go through 3.9. I migrated a three-broker Kafka 3.9.2 ZooKeeper cluster to KRaft in Docker and then upgraded it to Kafka 4.4.0: five phases with rolling restarts, the config and commands for each one, and all 100,000 messages intact at the end.