Scaling Sidekiq is challenging when paired with Redis. The hurdle arises from the complexity of Redis’ horizontal scaling model:

Redis is mostly single-threaded, unable to fully utilize modern multi-core servers.
On the other hand, it is discouraged to use Redis Cluster for Sidekiq, as detailed here.

This dilemma often leads to throughput bottlenecks when using Sidekiq with Redis. We have optimized Dragonfly, a modern multi-threaded drop-in replacement for Redis that can be used as a backend data store for Sidekiq, to address these scaling challenges. Without further ado, let’s dig into how we did this.