ITMO Builds a Tool That Shows What Your Optimizer Misses About Curvature

If you have ever tried to understand why second-order optimizers work well on some networks and fall apart on others, you have probably hit the same wall: modern autodiff frameworks give you the Hessian as one giant opaque matrix. A paper from researchers at ITMO University proposes an analytical decomposition that indexes the Hessian by the computation graph itself, turning that monolithic object into a set of interpretable inter-layer blocks with quantitative diagnostics.

The blind spot in every autodiff framework

Modern autodiff frameworks such as JAX and PyTorch return the Hessian of the loss as a single tensor, without exposing the internal structure of inter-layer interactions. The paper presents an analytical formalism that decomposes the full matrix into blocks indexed by the DAG of an arbitrary architecture. A lot of what we care about, including saddle points, curvature coupling between layers, and why skip connections stabilize training, lives in the cross-terms between layers rather than on the diagonal.

Existing tools work around this rather than solving it. K-FAC and Shampoo approximate block-diagonal pieces. Pearlmutter-style HVP gives you exact Hessian-vector products but reveals nothing about which layers are interacting. Per-layer spectral analyses look at diagonal blocks in isolation. None of these expose the off-diagonal structure between arbitrary node pairs in a DAG.

Splitting curvature into convex and residual parts

The core contribution is a canonical decomposition: H = H^GN + H^T. This separates the Gauss-Newton component, the convex part, from the tensor component, which is the residual curvature responsible for saddle points. The GN component always sits inside the positive semi-definite cone, which is exactly why optimizers built on it, with K-FAC as the poster child, are structurally blind to negative curvature.