Can CNN and RF be trained together

54 Views Asked by At

I am working on two different inputs, one is to CNN and the other data (categorical & numerical variables) is to the Random Forest model and concatenate and give it to a regression layer. I guess this is possible in ensemble stacking where you train CNN and RF separately. But, I am trying to concatenate the CNN and RF model and give it to the FC layer to get regression output. Here's the code:

rf_input = tf.keras.Input(shape=(60,))
cnn_output = tf.keras.layers.Conv2D(filters=16, kernel_size=(12, 125), activation='relu')(cnn_input)
cnn_output = tf.keras.layers.Conv2D(filters=32, kernel_size=(1, 40), activation='relu')(cnn_output)
cnn_output = tf.keras.layers.Flatten()(cnn_output)

rf_output = tfdf.keras.RandomForestModel(
    num_trees=fixed_hyperparameters['rf_num_trees'],
    max_depth=fixed_hyperparameters['rf_max_depth'],
    min_examples=fixed_hyperparameters['min_examples'],
    compute_oob_variable_importances=True
)(rf_input)

combined_output = tf.keras.layers.concatenate([cnn_output, rf_output])
fc_output = tf.keras.layers.Dense(32, activation='relu')(combined_output)
output = tf.keras.layers.Dense(1, activation='relu')(fc_output)

model = tf.keras.Model(inputs=[cnn_input, rf_input], outputs=output)`

optimizer = tf.keras.optimizers.Adam(learning_rate=fixed_hyperparameters['learning_rate'])
model.compile(optimizer=optimizer, loss='mse')

Does the model train both CNN and RF together when I call model.fit()? Also, I get an output for the above model. However, I am unsure if it does get trained together. When i try to attempt to get feature importance from randomforest object from one of the layers of the combined model, it throws me an error:

TypeError: in user code:

File "/home/hybrid/anaconda3/lib/python3.11/site-packages/tensorflow_decision_forests/keras/core_inference.py", line 436, in yggdrasil_model_path_tensor  *
    if multitask_model_index >= len(self._models):

TypeError: object of type 'NoneType' has no len()

I would like to know if CNN and RF trained together in the above code, OR is there any way I can combine trained CNN model with RF model(RF to be trained)?

0

There are 0 best solutions below