How to use Tensorflow Serving for the tensorflow transfomer example?

Question

I'm learning Tensorflow serving and got simple one working. But can not get the transfomer model working. https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/transformer.ipynb#scrollTo=eY_uXsOhSmbb

I trained the model, saved it and served it using below commands:

nohup tensorflow_model_server --rest_api_port=8502 --model_name=translator --model_base_path="/xxx".

Running command

saved_model_cli show --dir /xxxx/1 --tag_set serve --signature_def serving_default

gives the message below:

The given SavedModel SignatureDef contains the following input(s):
    inputs['sentence'] tensor_info:
      dtype: DT_STRING
      shape: ()
      name: serving_default_sentence:0
      The given SavedModel SignatureDef contains the following output(s):
    outputs['output_0'] tensor_info:
      dtype: DT_STRING
      shape: ()
      name: StatefulPartitionedCall_2:0
    Method name is: tensorflow/serving/predict

Then call it using

curl -d '{  "signature_name": "serving_default",   "instances": [{"sentence": "swab skin chest mcs"}] }' -H "Content-Type: application/json" -X POST http://localhost:8502/v1/models/translator:predict

But got below error:

{"error": "Trying to modify element 0 in a list with 0 elements.\n\t [[{{function_node __inference___call___366516}}{{node TensorArrayV2Write/TensorListSetItem}}]]"}

Can anyone help me to figure out what's wrong with what I have done?

I expect it return another string. Have tried to save it using below code, still same error:

from tensorflow.python.saved_model import signature_constants
from tensorflow.python.saved_model import tag_constants
class ExportTranslator(tf.Module):
    def __init__(self, translator):
        self.translator = translator

    @tf.function(input_signature=[tf.TensorSpec(shape=[], dtype=tf.string)])
    def __call__(self, sentence):
        (result, tokens, attention_weights) = self.translator(sentence, max_length=MAX_TOKENS)
        return result
    def get_config(self):
        # Define the configuration for your model, including any non-serializable objects.
        return {
            'translator': self.translator  # You may need to adapt this depending on your setup.
        }
translator = ExportTranslator(translator)
serving_signature = {
    'serving_default': translator.__call__.get_concrete_function()
}

tf.saved_model.save(
    translator, export_dir='panel',
    signatures=serving_signature,
    options=tf.saved_model.SaveOptions(namespace_whitelist=["serving_default"])
)