How to preallocate using JacobianConfig for "Hessian of vector valued function" double Jacobian in Julia ForwardDiff package

24 Views Asked by At

I have a vector valued function that is fixed in input and output size, but is called in a loop and I would like to calculate the gradient and hessian of all output entries.

In addition, each call slightly changes additional inputs to that function (that is an index is changed but I am not interested in the gradient wrt to that index and that input does not change the function in a meaningful way wrt gradient calculations).

This is what I am currently doing (as described here: https://juliadiff.org/ForwardDiff.jl/stable/user/advanced/#Hessian-of-a-vector-valued-function)

Outside Loop

jcfg = ForwardDiff.JacobianConfig(y->f(y,param,k),
       x[1:LenIndex], 
       ForwardDiff.Chunk(zeros(LenIndex)))
hcfg = ForwardDiff.JacobianConfig(
       z->ForwardDiff.jacobian(y->f(y,param,k),z),
       x[1:LenIndex], 
       ForwardDiff.Chunk(zeros(LenIndex)))

Inside loop:

H = ForwardDiff.jacobian(y -> f(y,param,k),x[Index],jcfg,Val{false}())::Matrix{Float64}    
HH = reshape(
         ForwardDiff.jacobian(z -> 
           ForwardDiff.jacobian(y -> f(y,param,k), z),
           x[Index],
           hcfg,
           Val{false}())::Matrix{Float64},
     4,LenIndex,LenIndex)

This works and removing tag checks and using the configs noticably improves speed. However, I assume that there is more to be gained by also using a config for the "inner" Jacobian in the Hessian. Of course, I cannot specify the same things because the dual number magic will be the input. I tried some things like zeros(ForwardDiff.Dual, LenIndex) but that did not work and I honestly have no idea what I am doing. I tried finding a guide on how to use the dual number type, but what I find confuses me more than it aids me, because suddenly there is something like Dual{S,V,M} and I have no clue what S and V should be or if I need to define tags and whatnot.

Anyways, how would one configure the inner jacobian? Can I disable tag checking in there or is that exactly the point where things go bananas? And can I use the DiffResults thing to get function, gradient and hessian in one swoop also here?

0

There are 0 best solutions below