I am working on a hybrid model that uses CNN (time series input) and H2oRandom forest (tabular data input) combined at the fully connected layer to solve regression problems. I would like to optimize the hyperparameters of CNN and RF. The only way I figured out is to optimize two models separately and combine them at the FC layer to obtain the output. I use RandomSeach Keras Tuner for CNN and Grid RandomSearch for H2O Random Forest hyperparameters.
I am uncertain if optimizing the models separately is the most effective way to enhance the model's performance. Is there a different approach to optimizing hybrid models? Please let me know if there is a better approach.
Check this page: https://www.tensorflow.org/decision_forests/tutorials/model_composition_colab
It combines CNN and decision forest just like what you are trying to do.
If they do not have gridsearch, you will just have to run the chained models multiple times. Each time you will use a different hyperparameter settings.