PyTorch methods that can only be called from `forward`

Question

In the docs for extending torch.autograd:

...please refer to the docs of Function to find descriptions of useful methods that can be called only from forward().

But following that link, it's not clear to me what they're referring to. Any ideas?

Answer 1

They are referring to the methods only accessible in the forward method. In the scope of the forward method, self is conventionally named ctx and is provided as an object mixin. There are four special functions available on that mixin: mark_dirty, mark_non_differentiable, save_for_backward, and set_materialize_grads.