This is the code snippet for the datasets
train_ds= tf.keras.preprocessing.image_dataset_from_directory( "./tmp/Alzheimer_s_Dataset/train/", validation_split=0.2, subset="training", shuffle=True, seed=1337, image_size=(IMAGE_WIDTH,IMAGE_HEIGHT), batch_size=BATCH_SIZE )
val_ds = tf.keras.preprocessing.image_dataset_from_directory( "./tmp/Alzheimer_s_Dataset/train/", validation_split=0.2, subset="validation", shuffle=False, seed=1337, image_size=(IMAGE_WIDTH,IMAGE_HEIGHT), batch_size=BATCH_SIZE )
test_ds= tf.keras.preprocessing.image_dataset_from_directory( "./tmp/Alzheimer_s_Dataset/test/", shuffle=False, seed=1337, image_size=(IMAGE_WIDTH,IMAGE_HEIGHT), batch_size=BATCH_SIZE )
train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=tf.data.AUTOTUNE)
val_ds = val_ds.cache().prefetch(buffer_size=tf.data.AUTOTUNE)
test_ds = test_ds.cache().prefetch(buffer_size=tf.data.AUTOTUNE)`
Even though the training accuracy and validation accuracy are around 94%, the model doesn't perform well on test data. I want to see the F1 score and other metrics. How to calculate the F1 score and other metrics for a tf dataset which is batched and prefetched.
I have tried to do custom F1 score calculation and also have tried to use tensorflow-addons for the metrics calculation. Custom f1 score doesn't work properly and when i compile like this
model2.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.00001), loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=[tf.keras.metrics.Accuracy(), tf.keras.metrics.Precision(), tf.keras.metrics.Recall(), tfa.metrics.F1Score(num_classes=4, average='macro', threshold=0.5)])
and try to train the model, it always throw an error
ValueError: Shapes (None, 1) and (None, 4) are incompatible.