Add 4D tensor of shape (16, 8, 8, 5) and 2D tensor of shape (16, 16)

Question

I want to add a noise of shape (16,16) to a 4D tensor of shape (16,8,8,5). I have tried several broadcasting approaches but still get errors but I get singleton mismatch errors. How can I add these two tensors without any broadcasting errors.

Answer 1

I assume you have dimensions like this (batch size, width, height, channels) or similar.

There is not enough noise for each element in the tensor, so some elements will have to share noise. I guess that the channel dimension will have to share their noises. Lets say the channels were RGB (colors). Then there will be a unique noise for every image, at every location, but just one for each color channel.

However, your noise does not fit to this (nor any other way).

If I described your case correctly, simply use noise of the correct shape (16,8,8)

If you had that kind of noise, it will work:

import torch

tensor1 = torch.ones((16,8,8,5))
tensor2 = torch.ones((16,8,8))

# you will need to reshape noise from 16x16 to 16x8x8
result = tensor1 * tensor2.reshape(16,8,8,1)   # add a last dimension for the channels

If I wasn't correct with my guessing, please elaborate