first of all: I'm not a data scientist, so I might use wrong terms here and there. My apologies upfront :).
First some details of my goal: I have a very complex algorithm for which I want to optimize the execution time. For my tests, I have quite a few binary switches that enable/disable a certain code path. Other parameters (e.g. integer-valued in a certain range [a,b]) are only active if a binary switch is active. Hence, if the corresponding switch is off, it doesn't make sense to permutate over the dependent parameter, as this would result in the identical code being executed.
Optimization Script:
I plan to use one or more optimizers available in Python. Since the problem is complex, execution times are long and there might be significant local extrema caused by certain parameter values, I started with the Bayesian optimizer implemented by gp_minimize() in the scikit.optimize package.
For contraining allowed parameter sets, I see (roughly) the following two options:
- Provide some additional info or a callback to the optimizer that tells it, if a parameter set is valid or not. Or in different terms, which values a certain parameter can take given the currently selected value of the other binary switch parameters.
- Penalize invalid parameter sets in the objective function, e.g. by returning a huge value. However, I'm concerned that this would push the optimizer in a wrong direction (e.g. in case that it alters several parameters at once it wouldn't know that only a single parameter lead to the penalty)
Question:
Could someone give me some hints how to achieve this? I guess there are packages that allow providing some callbacks, but gp_minimize() doesn't seem to be one of these.
Are my concerns correct with option (2)?
Many thanks in advance