Too many GRPC calls for create snapshot

Question

In our CSI Driver, we recently included snapshot feature support. Using v6.0.1 external-snapshotter. I see that when we do a create snapshot action, too many GRPC calls are seen in the logs for csi.v1.Controller/CreateSnapshot

cat driver.log | grep 'csi.v1.Controller/CreateSnapshot' 
I0927 17:47:14.105028       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:17.243862       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:18.155425       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:19.026359       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:19.816381       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:20.553471       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:21.265841       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:22.090815       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:22.851513       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:23.555432       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:24.269332       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:25.051528       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:25.764960       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:26.682565       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:27.359656       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:28.120403       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:28.790861       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:29.509234       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:30.199635       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:30.905746       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:31.605362       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:32.418714       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:33.306218       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:34.021675       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:34.754203       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:35.593932       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:38.163890       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:38.958784       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:39.901797       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:40.594249       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:41.351820       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:42.071174       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot
I0927 17:47:42.765785       1 server.go:151] GRPC call: /csi.v1.Controller/CreateSnapshot

What could possibly be the reason. Any pointers on where I should start looking into.

Answer 1

Perhaps you could log the addresses of the clients. In Go, you can get their addresses with peer.FromContext. This would help you determine whether you have too many callers operating simultaneously or if you have a single misconfigured caller.