Validation loss doesn't react to hyperparameter tuning

Question

I built a regressor through neural network. The plot of my training losses changes together with the adjustment of the hyperparameters (lr, dropout rate, activation fn). However, my val losses remain the same. The val loss is decreasing as it should. I want to make it lower, but it starts to plateau at the same loss value whatever the hyperparameters are.

What should I try doing next?

Answer 1

firstly you need overfit on train dataset. it means the value of the validation loss goes down and goes up. After that you can regularize your data