Recently, I was connecting multiple NodeJs SocketIO applications through a Redis adapter using the Redis PUB/SUB mechanism, everything worked fine until the number of client connections in my application reached around 4000+ then everything started to slow down. After researching, I found a passage that says there is a limit to using Redis as an adapter.
However, there were only a few resources on using RabbitMQ as the adapter, even the package provided by the SocketIO official docs seems outdated. I then tried with the socket.io-amqp0 instead (which seems to support socket io v4); while there are errors on some channel operations which may shut down the application sometimes.
After doing more research, the @socket.io-redis-adapter package seems to support the redis v7 redis sharded pub/sub mechanism refer to this issue which seems can solve the scalarble issue perfectly.
The application then seems able to connect to the Redis cluster successfully without any error, but the message never successfully distributed across nodes through the Redis cluster, any thought on this?
sharded-redis.ts
// env contain REDIS_SHARD_HOSTS=host1.com,host2.com,host3.com
const REDIS_SHARD_HOSTS = process.env.REDIS_SHARD_HOSTS?.split(',')
const REDIS_SHARD_PORT = 6379
const hosts = REDIS_SHARD_HOSTS!.map((host) => {
const options = {
host,
port: REDIS_SHARD_PORT,
}
return options
})
export const shardedRedisPubClient = new Redis.Cluster(redisOptions)
export const shardedRedisSubClient = shardedRedisPubClient.duplicate()
app.ts
const io = new Server<ISocketRoute, ISocketData>({
cors: {
origin: '*',
},
})
const shardedAdapter = createShardedAdapter(shardedRedisPubClient, shardedRedisSubClient)
io.adapter(shardedAdapter)
//...
Packages
"@socket.io/redis-adapter": "8.2.1",
"ioredis": "^5.3.1",
"socket.io": "^4.6.1"