For system_auth default Class should be SimpleStrategy or NetworkTopologyStrategy

Question

In our Prod cluster, I see some of the system* keyspaces with SimpleStrategy. As we are adding new DCs in our cluster we need to be on NetworkTopologyStrategy otherwise while I am running nodetool rebuild, It got failed.

I alter all keyspaces from SimpleStrategy to NetworkTopologyStrategy & rebuild works well.

During ALTER it gives warning also that you are altering system keyspace.

Another question: Opscenter Keyspace(rollup* tables) is having a hell lot of data, why I should replicate that, it should be enough to put on only 1 DC??

My question what should be the ideal strategy for system* keyspaces??

Answer 1

My question what should be the ideal strategy for system* keyspaces?

A little warning on this one. system and system_schema have a special replication strategy of LocalStrategy, and they should stay that way.

The other keyspaces of system_auth, system_distributed, and system_traces however, are a different story. Setting those to use NetworkTopologyStrategy with a RF 3 (assuming each DC has at least 3 nodes) for each DC is perfectly acceptable. Setting that for system_distributed, and system_traces isn't as important, but it shouldn't hurt anything.

On the other hand, system_auth should definitely be set to use NetworkTopologyStrategy with a RF specified for each DC. The reason, is that SimpleStrategy could potentially put all of its replicas in a single DC, or even zero replicas in one DC. That could cause high latency or even timeouts for auth checks, as that would result in cross-DC network traffic.

Also, if your applications use DC-specific load balancing policies, you will need to specify a RF for each DC in system_auth. As mentioned above, using SimpleStrategy could result in a DC not getting any replicas for a user. And than would prevent DC-specific applications from connecting.