How to implement softmax loss with variable length of choice

Question

I'm trying to implement the drqa model on tensorflow where I'm going to predict begin and end position of the answer substring. So there is [batch_size, max_time] output of probabilities, corresponding context sequences. Now I want to apply softmax cross enthropy loss, but sequences are zero padded. So I don't want padding to be counted in loss. Is there some advice how to implement this? Thank you.

Answer 1

Use weights.

tf.losses.softmax_cross_entropy(
  labels,
  logits,
  weights=[1.]* actual_length + [0.]* (max_length - actual_length)
)