how to calculate loss between output of first convolutional block and output of second convolutional block?

Question

x1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)
x1= MaxPooling2D((2, 2), padding='same')(x1)                                                         
x2 = Conv2D(64, (3, 3), activation='relu', padding='same')(x1)
x2= MaxPooling2D((2, 2), padding='same')(x2)
x3= Conv2D(128, (3, 3), activation='relu', padding='same')(x2)
x3= MaxPooling2D((2, 2), padding='same')(x3)

how do we calculate l2 loss between x1 and x3 layers in tensorflow? The problem is that x1 and x3 outputs differ in size.Direct reshaping throws errors.I tried to flatten the outputs and reshape but the sizes are still not comparable