Stacking with neural network as baseline model: 'History' object has no attribute 'predict'

Question

I have a problem including a neural network as one of three baseline models in my stacking algorithm.

rf = RandomForestRegressor
xgb = XGBRegressor
nn = Sequential([ Dense(neurons, input_dim=input_dim, use_bias=False),
                  Dense(neurons),
                  Dense(1)
                ])
                 
# STACKING CODE# 
# generate dummy data
X = train_x.values
y = train_y.values

# initialize 3 models to be stacked
model_1 = rf
model_2 = xgb
model_3 = nn


# generate cross-val-prediction with rf and xgb and nn using TimeSeriesSplit
cross_val_predict = np.row_stack([
    np.column_stack([
        model_1.fit(X[id_train], y[id_train]).predict(X[id_test]),
        model_2.fit(X[id_train], y[id_train]).predict(X[id_test]),
        model_3.fit(X[id_train], y[id_train]).predict(X[id_test]),
        y[id_test]  # we add in the last position the corresponding fold labels
    ])
    for id_train,id_test in TimeSeriesSplit(n_splits=3).split(X)
])  # (test_size*n_splits, n_models_to_stack+1)

After I run the model for the part on neural network it says that 'History' object has no attribute 'predict'.

AttributeError                            Traceback (most recent call last)
<ipython-input-76-5c269db6c559> in <module>
     12 
     13 # generate cross-val-prediction with rf and gb using TimeSeriesSplit
---> 14 cross_val_predict = np.row_stack([
     15     np.column_stack([
     16         model_1.fit(X[id_train], y[id_train]).predict(X[id_test]),

<ipython-input-76-5c269db6c559> in <listcomp>(.0)
     16         model_1.fit(X[id_train], y[id_train]).predict(X[id_test]),
     17         model_2.fit(X[id_train], y[id_train]).predict(X[id_test]),
---> 18         model_3.fit(X[id_train], y[id_train]).predict(X[id_test]),
     19         y[id_test]  # we add in the last position the corresponding fold labels
     20     ])

AttributeError: 'History' object has no attribute 'predict'

I get that you can't call predict straight after fitting the neural model as fitting the model calls out a history object rather than a model object, so i changed the code to:

cross_val_predict = np.row_stack([
    np.column_stack([
        model_1.fit(X[id_train], y[id_train]),
        model_1.predict(X[id_test]),
        model_2.fit(X[id_train], y[id_train]),
        model_2.predict(X[id_test]),
        model_3.fit(X[id_train], y[id_train]),
        model_3.predict(X[id_test]),
        y[id_test]  # we add in the last position the corresponding fold labels
    ])
    for id_train,id_test in TimeSeriesSplit(n_splits=3).split(X)
])  # (test_size*n_splits, n_models_to_stack+1)

However, now i get a new error....

1/1 [==============================] - 2s 2s/step - loss: 64.2233 - mse: 64.2233 - mae: 1.5694
1/1 [==============================] - 0s 187ms/step
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-98-cd6a2aea778e> in <module>
     12 
     13 # generate cross-val-prediction with rf and gb using TimeSeriesSplit
---> 14 cross_val_predict = np.row_stack([
     15     np.column_stack([
     16         model_1.fit(X[id_train], y[id_train]), model_1.predict(X[id_test]),

<ipython-input-98-cd6a2aea778e> in <listcomp>(.0)
     13 # generate cross-val-prediction with rf and gb using TimeSeriesSplit
     14 cross_val_predict = np.row_stack([
---> 15     np.column_stack([
     16         model_1.fit(X[id_train], y[id_train]), model_1.predict(X[id_test]),
     17         model_2.fit(X[id_train], y[id_train]), model_2.predict(X[id_test]),

<__array_function__ internals> in column_stack(*args, **kwargs)

~\Anaconda3\lib\site-packages\numpy\lib\shape_base.py in column_stack(tup)
    654             arr = array(arr, copy=False, subok=True, ndmin=2).T
    655         arrays.append(arr)
--> 656     return _nx.concatenate(arrays, 1)
    657 
    658 

<__array_function__ internals> in concatenate(*args, **kwargs)

ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 1850 and the array at index 1 has size 75

It seems like somehow i have to preserve the original sequence of the code while calling fit and predict on the neural network at the same time....anyone can help please?

Answer 1

As stated in the error,

all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 1850 and the array at index 1 has size 75

Numpy is essentially telling you that the shapes of the concatenated matrices/arrays should be aligned. For example, we can concatenate a 4x5 matrix with a 4x4 matrix/array to create a 4x9 matrix/array.

The error here is reporting that the axes are not aligned. We cannot attempt to concatenate a 4x5 matrix with a 10x10 matrix because the shapes are not aligned. You use the np.reshape function to modify the shape of one of the matrices/arrays so that they may be concatenated.

Kindly refer to this for more information. Thank you!