I am applying a bert layer to my dataset. After that i need to use that trained model and apply a multihead attention layer. But I am having lot of trouble in fitting my model with the attention layer.
I tried to get the shape of the sample input i am passing , tokenized to get the shape and pass that shape to the bert model. After that I have defined a custom multihead attention layer but it does not seem to be fitting in the model.