Concatenate ragged inputs in Keras

Question

The following code creates a dummy model that concatenate 2 inputs. One input is used with an Embedding layer with output size of 5, while the second input is just merged with the output of the Embedding layer:

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Input, Embedding, Concatenate, Dense
from tensorflow.keras.models import Model
import keras

x = np.random.randint(0 ,50, size = (10,5,1))
y = np.random.randint(0 ,1, size = (10,1) )

def get_model():
  input1 = Input( shape =(None,11), name='timeseries_input' )
  input2 = Input( shape = (None,1) ,name='embedding_input')
  emb = Embedding(input_dim= len(np.unique(x)) , output_dim= 5)(input2)
  emb = keras.layers.Reshape( target_shape=( -1,5) )(emb)
  merged = Concatenate(axis =2 )([emb,input1])
  out = Dense(1)(merged)
  model = Model([input1,input2],out)
  model.summary()
  return model


m = get_model()
tf.keras.utils.plot_model(
    m,
    show_shapes=True,
    show_dtype=True,
    show_layer_names=True,
    rankdir="TB",
)

The code works and produce the following structure:

__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
embedding_input (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
embedding_17 (Embedding)        (None, None, 1, 5)   155         embedding_input[0][0]            
__________________________________________________________________________________________________
tf.compat.v1.shape_15 (TFOpLamb (4,)                 0           embedding_17[0][0]               
__________________________________________________________________________________________________
tf.__operators__.getitem_12 (Sl ()                   0           tf.compat.v1.shape_15[0][0]      
__________________________________________________________________________________________________
tf.reshape_12 (TFOpLambda)      (None, None, 5)      0           embedding_17[0][0]               
                                                                 tf.__operators__.getitem_12[0][0]
__________________________________________________________________________________________________
timeseries_input (InputLayer)   [(None, None, 11)]   0                                            
__________________________________________________________________________________________________
concatenate_11 (Concatenate)    (None, None, 16)     0           tf.reshape_12[0][0]              
                                                                 timeseries_input[0][0]           
__________________________________________________________________________________________________
dense_10 (Dense)                (None, None, 1)      17          concatenate_11[0][0]             
==================================================================================================
Total params: 172
Trainable params: 172
Non-trainable params: 0
__________________________________________________________________________________________________

However , when adding ragged=True to my inputs:

  input1 = Input( shape =(None,11), name='timeseries_input',ragged=True )
  input2 = Input( shape = (None,1) ,name='embedding_input',ragged=True)

the code breaks with the following error:

TypeError: Failed to convert object of type <class 'tensorflow.python.ops.ragged.ragged_tensor.RaggedTensor'> to Tensor. Contents: tf.RaggedTensor(values=Tensor("Placeholder:0", shape=(None, 1, 5), dtype=float32), row_splits=Tensor("Placeholder_1:0", shape=(None,), dtype=int64)). Consider casting elements to a supported type.

How can one concatenate ragged inputs? What am I missing?

Answer 1

Using tf.concat instead of tf.keras.layers.Concatenate resolves the issue, because Ragged Tensor is not supported by Concatenate Layer but it is supported by tf.concat.

Please find the working code below:

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Input, Embedding, Concatenate, Dense
from tensorflow.keras.models import Model
import keras

x = np.random.randint(0 ,50, size = (10,5,1))
y = np.random.randint(0 ,1, size = (10,1) )

def get_model():
  input1 = Input( shape =(None,11), name='timeseries_input', ragged = True )
  input2 = Input( shape = (None,1) ,name='embedding_input', ragged = True)
  emb = Embedding(input_dim= len(np.unique(x)) , output_dim= 5)(input2)
  emb = keras.layers.Reshape( target_shape=( -1,5) )(emb.to_tensor())
  merged = tf.concat([emb,input1], axis = 2)
  out = Dense(1)(merged.to_tensor())
  model = Model([input1,input2],out)
  model.summary()
  return model


m = get_model()
tf.keras.utils.plot_model(
    m,
    show_shapes=True,
    show_dtype=True,
    show_layer_names=True,
    rankdir="TB",
)