HND Keeps Hypergraph Neural Networks Stable at 40 Layers Deep
A new framework treats hypergraph message passing as a nonlinear diffusion PDE, staying stable at 40 layers while topping 11 benchmarks.
ProTakeaways
- HND recasts hypergraph message passing as a nonlinear PDE with learnable, structure-aware diffusion coefficients per hyperedge-node pair.
- Provable energy dissipation, discrete maximum principle, and unconditional stability under implicit Euler discretization.
- Supports Euler, Runge-Kutta, Adams-Bashforth/Moulton, and adaptive-step solvers, each producing a valid layer variant.
- Ranks first overall on six real-world hypergraph benchmarks and top-two on five academic ones against 15+ baselines.
- Accuracy stays flat from 2 to 40 layers on Cora while most baselines degrade sharply past four layers.
- Code is available at gitee.com/zmy-ovo/hnd; per-layer complexity is O(cmd).
Hypergraphs let a single edge connect three or more nodes at once, which fits how real data actually looks: papers with multiple co-authors, molecules with functional groups, purchases that bundle several items. The trouble is that hypergraph neural networks (HGNNs) tend to fall apart when you stack more than a few layers. A new paper called Hypergraph Neural Diffusion (HND) rebuilds message passing from the ground up as a nonlinear partial differential equation, and the resulting architecture stays stable at 40 layers while outperforming a dozen baselines.
Why deep HGNNs collapse
The authors frame the issue bluntly: existing HGNNs suffer from shallow propagation, oversmoothing, and limited adaptability to complex hypergraph structures. Oversmoothing is the well-known failure mode where node features become nearly identical after too many rounds of neighbor aggregation, destroying the discriminative signal a classifier needs.
Prior fixes came from two camps. Spectral methods like HGNN and HyperGCN approximate hypergraph convolution via clique or star expansion, which distorts the true combinatorial semantics of hyperedges. Message-passing models like AllSetTransformer and ED-HNN preserve more structure but tend to remain shallow. What was missing was a principled continuous-time formulation that could tell you, mathematically, why depth would or would not blow up.
Message passing as heat flow
HND's core move is defining gradient and divergence operators directly on hypergraphs. The gradient of a node function at a hyperedge-node pair (e, v) measures how far node v deviates from the average of the hyperedge it belongs to. Divergence aggregates those deviations back to nodes. Composing the two yields a hypergraph Laplacian, and the authors show it algebraically recovers the classical normalized Laplacian from Zhou et al. 2006, derived here from variational principles rather than combinatorial expansion.