I have used VertexAI AutoML tabular to train a regression model.
In the dataset I have provided the TRAIN, VALIDATE & TEST split manually using a column indicating which set the row belongs to.
When I train the model, it provides me with the evaluations, but I'm not sure on which set these evaluations are calculated (Logically it should be on the TEST set).
In the image (Img 1) the metrics generated after training the model can be beed (marked in red box).
My doubt arises when I create evaluations separately for TRAIN, TEST and VALIDATION sets, the evaluation for neither of these sets match with the evaluation provided after model training.
It can be refered in the image (Img 1) that none of the evaluations ("Eval-test-set", "Eval-validation-set", "Eval-train-set") match with the original evaluation (Untitled...)
Has someone faced this on VertexAI AutoML tabular (regression)? or could someone provide me an understanding as to why this descripency in evaluations?
Thanks in advance!
I have also checked the evalation outputs for the separately created evaluations ("Eval-test-set", "Eval-validation-set", "Eval-train-set"), there are no errors in any of the rows.