Access OptimizeResult in scipy.optimize.minimize callback function

Question

I am trying to define a callback function to be used when optimizing. I am only interested in the current objective function's value for each iteration.

The documentation suggests, that most optimize methods, and specifically BFGS (my method used), support a callback function with the signature: callback(OptimizeResult: intermediate_result). But trying to access the intermediate_result's fun attribute throws AttributeError.

Using the signature callback(xk) works fine, accessing only the parameter of the current iteration. But to get the value of my objective at each iteration, I would need to re-evaluate the objective, which is not desirable.

How can I get the objective value at each iteration to be returned in my callback without redundant function evaluations of my objective?

Answer 1

If you read the documentation you linked closely it says:

Note that the name of the parameter must be intermediate_result for the callback to be passed an OptimizeResult.

Meaning, you cannot leave the variable as xk and have it be a positional argument, it must be called intermediate_result and must be a keyword argument. In Python 3 you can force a function argument to be a keyword argument by putting * for the first argument (relevant Stack Overflow question).

from scipy.optimize import minimize, rosen

def callback(*, intermediate_result):
    print(intermediate_result.fun)

x0 = [1.3, 0.7, 0.8, 1.9, 1.2]
res = minimize(rosen, x0, method="Nelder-Mead", tol=1e-6, callback=callback)