I've been training a swin Transformer model for a classification task, and I'm encountering a perplexing issue. Despite adjusting various hyperparameters, such as learning rate and network architecture, the accuracy of my model seems to remain stagnant throughout the training process. Here's a snippet of the training log:
Epoch: 1 | train_loss: 2.0857 | train_acc: 0.3321 | test_loss: 1.9843 | test_acc: 0.2143
Epoch: 2 | train_loss: 1.4990 | train_acc: 0.3879 | test_loss: 1.5551 | test_acc: 0.3929
Epoch: 3 | train_loss: 1.4910 | train_acc: 0.3854 | test_loss: 1.4985 | test_acc: 0.3929
Epoch: 4 | train_loss: 1.4396 | train_acc: 0.4148 | test_loss: 1.5062 | test_acc: 0.3929
Epoch: 5 | train_loss: 1.4558 | train_acc: 0.3811 | test_loss: 1.4954 | test_acc: 0.3929
Epoch: 6 | train_loss: 1.4412 | train_acc: 0.4105 | test_loss: 1.5001 | test_acc: 0.3929
As you can see, the training and test accuracies seem to plateau around 0.39. I'm using the Swin Transformer architecture for this task. Could anyone provide insights or suggestions on what might be causing this issue and how to address it? Any help would be greatly appreciated.
In attempts to improve the swin Transformer model's performance. The accuracy remains stagnant around 0.39, contrary to expected improvements.