I am trying to implement basic newton method using pytorch. This is my code basically
hessian_inverse = torch.inverse(hessian)
step = -(hessian_inverse @ gradient)
new_point = curr_point + step
however, what should I do when the hessian is singular? I tried to run my code but I got this error
The diagonal element 1 is zero, the inversion could not be completed because the input matrix is singular.
Which tells me the hessian couldn't be inverted. How does newton method handle this in practice?