ValueError: Input 0 of layer lstm_17 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 128]

Question

Here is the code:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, RepeatVector, Dense, Reshape

Model = Sequential([
          Embedding(vocab_size, 256, input_length=49),
          LSTM(256, return_sequences=True),
          LSTM(128, return_sequences=False),
          LSTM(128),
          Reshape((128, 1)),
          Dense(vocab_size, activation='softmax')
])

And this is the error message:

ValueError: Input 0 of layer lstm_11 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 128]

I am using tensorflow 1.15.0 and running it on Google Colab. How can I fix it.

Answer 1

As also said by Marco in the comments, the decoder expects 3d but it gets 2d, so applying a RepeatVector layer before the decoder worked. The corrected Model:

Model = Sequential([
      Embedding(vocab_size, 256, input_length=49),
      LSTM(256, return_sequences=True),
      LSTM(128, return_sequences=False),
      RepeatVector(1),
      LSTM(128),
      Dense(vocab_size, activation='softmax')
])

I added RepeatVector layer to make the output shape 3D, and removed the Reshape layer since now it has no use.

Thanks Marco for the help!