I am using CEEDMAN to decompose a time series with the aim of improving its forecast. However, I have questions about how to evaluate the results using the Multilayer Perceptron model?
I'm doing it this way, is that correct?
imfs_prediction = []
i = 1
array_MAPE = []
array_MAE = []
array_MSE =[]
array_RMSE = []
for imf in imfs:
print('-'*45)
print('This is ' + str(i) + ' time(s)')
print('*'*45)
X1_train, Y1_train, X1_test, Y1_test = data_split(imf_data(imf,1), 1700, 10)
##########################################Modelo##################################
model2 = MLPRegressor(activation= 'tanh', alpha=0.01,batch_size= 64, hidden_layer_sizes= (100,100), learning_rate = 'constant',max_iter= 300, solver= 'adam')
multioutput = MultiOutputRegressor(model2,n_jobs=-1).fit(X1_train,Y1_train)
prediction_Y = multioutput.predict(X1_test)
imfs_prediction.append(prediction_Y)
i+= 1
# =================================================================
MSE=np.mean(( prediction_Y- Y_Test_Full)**2)
print("MSE: ", MSE)
print('RMSE: ',MSE**0.5)
MAE=np.mean(np.abs( prediction_Y- Y_Test_Full))
print('MAE: ',MAE)
MAPE=mean_absolute_percentage_error(Y_Test_Full,prediction_Y)
print('MAPE: ',MAPE)
array_MAPE.append(MAPE)
array_MAE.append(MAE)
array_MSE.append(MSE)
array_RMSE.append(MSE**0.5)
final_prediction = np.sum(imfs_prediction, axis=0)
I am averaging the results obtained from MFIs
print("MAPE", np.mean(array_MAPE))
print("MAE", np.mean(array_MAE))
print("MSE", np.mean(array_MSE))
print("RMSE", np.mean(array_RMSE))
Or add the prediction of the final results and compare with the real time series
mse = mean_squared_error(Test, final_prediction)
In short, my question is how to evaluate CEEMDAN together with Multilayer Perceptron. Should I use the average of the results obtained from the MFIs, or should I add the predicted results of the MFIs and then compare it with the actual time series?