Østfold Researchers Show Tiny 9,800-Parameter AI Shares Hidden Features Across Classes
A tiny 9,800-parameter Neural Cellular Automaton beats larger recurrent and feed-forward models on few-shot MNIST by reusing hidden-state features across classes.
- New paper shows NCA hidden channels learn general, scale-invariant shape features rather than class-specific templates.
- A ~9,800-parameter NCA beats recurrent and feed-forward baselines on few-shot and scale-variant MNIST.
- Novel transfer method injects a teacher's hidden states into a student NCA during early training.
- Student trained on digits 6-9 succeeds using features from a teacher that only saw 0-5.
- Hidden channels converge to mutually orthogonal states, absorbing morphology while readout channels vote uniformly.
- Work by Guichard and Nichele, to be presented at WIVACE 2026 at the University of Cambridge.
Neural cellular automata reuse hidden features across digit classes
Etienne Guichard and Stefano Nichele report that the hidden channels inside a Neural Cellular Automaton, or NCA, learn structural features that transfer across digit classes. Their experiments turn those features into a few-shot training method: a pretrained teacher supplies hidden states to a small student early in training, then the student continues independently.
The researchers, both at Østfold University of Applied Sciences, describe the work in Neural Cellular Automata Learn General Features in their Hidden Channels (paper). It is scheduled for presentation at WIVACE 2026 at the University of Cambridge.
A 9,800-parameter test bed
Few-shot classification exposes a common weakness in large neural networks: with one or two labeled examples per class, they can memorize the training set without learning features that generalize. Conventional transfer learning can reduce that risk, although its pretrained backbone may carry millions of parameters into deployment.
An NCA offers a smaller, iterative architecture. Each pixel acts as a cell containing a state vector. A shared neural network reads a cell and its local neighborhood, updates the state, and repeats that operation across the grid for several steps. Repetition lets information travel beyond the initial neighborhood while keeping the learned update rule compact.
| Component | Role |
|---|---|
| Visible channels | Hold observable values and produce the spatial class readout. |
| Hidden channels | Store intermediate features, memory, and coordination signals. |
| Shared update network | Applies the same local rule at every cell and time step. |
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
