I post the question here as recommended in tf-serving github repo
I trained a RegNetX008 model using tensorflow 2.12 and now I struggle with serving it. I use tensorflow/serving-2.12.1-gpu.
When calling the network, it raises the following error:
{'error': '2 root error(s) found.\n (0) UNIMPLEMENTED: Could not find compiler for platform CUDA: NOT_FOUND: could not find registered compiler for platform CUDA -- was support for that platform linked in?\n\t [[{{function_node __inference__wrapped_model_13303}}{{node model/regnetx008_Stage_0_XBlock_0_conv_3x3/PartitionedCall}}]]\n\t [[StatefulPartitionedCall/_587]]\n (1) UNIMPLEMENTED: Could not find compiler for platform CUDA: NOT_FOUND: could not find registered compiler for platform CUDA -- was support for that platform linked in?\n\t [[{{function_node __inference__wrapped_model_13303}}{{node model/regnetx008_Stage_0_XBlock_0_conv_3x3/PartitionedCall}}]]\n0 successful operations.\n0 derived errors ignored.'}
Are there any recommendations on how to serve the model? Or maybe even recommendations on changing the versions - I am ready to retrain the network and serve it using another tensorflow/tf-serving versions, but I'm not sure which ones will work if any.
The error persists on 4 different machines - one with cuda-11.8 and cudnn 8.6 as recommended in tensorflow-gpu installation guide, and the rest having 11.7, 12.0 and 12.1 cuda versions.
I've tried to initialize a RegNetX008, save it without changes and serve it - no help, the error persists.
I've also tried tensorflow serving nightly and 2.9-2.11 versions.