Unconventional AI's Un-0 Gets Better After Deleting 93% of Its Connections

Unconventional AI just published a result that runs counter to almost every instinct in deep learning: they deleted 93.25% of the connections in their Un-0 image generation model and the model got measurably better. On ImageNet 64x64, the sparse version hits an FID of 7.15 , roughly 1.9 points ahead of the dense baseline at matched size. Lower FID is better, so this is a genuine quality improvement, not a rounding error.

A quick primer on what Un-0 actually is

Un-0 is an image-generation model built on Kuramoto dynamics: it generates an image by integrating the phase dynamics of a population of coupled oscillators , no diffusion schedule, no adversary, no iterative denoising. Think of it as thousands of tiny pendulums, each spinning at its own natural frequency, nudging each other through learned coupling weights. The system evolves over time, and a small conventional decoder reads out the final oscillator states and renders them into pixels.

The company is building an oscillator-based computer architecture that abandons the digital logic underpinning virtually all modern computing. Instead of processing data through transistors performing binary operations, Unconventional's approach uses coupled ring oscillators in a fabric network, encoding and processing information through the physics of the oscillators themselves. The long-term bet: Un-0 aims to demonstrate a path toward dramatically reducing AI's energy consumption , potentially by up to 1000x compared to current GPU-based digital systems.

The model is software-only for now (no physical chip yet) but demonstrates that the company's radically different computing approach can produce real AI results. With Naveen Rao (former Databricks AI chief, Nervana founder) leading and $475M raised at a $4.5B seed valuation, Unconventional AI is one of the most consequential AI hardware bets of the current cycle.

The scaling problem that forced the sparsity question

In the original Un-0, every oscillator is coupled to every other oscillator. That sounds powerful, but it creates a brutal quadratic scaling problem. The number of connections grows as N², so a network with 1,000 oscillators needs 1 million couplings; a million oscillators would need a trillion. Such architectures are challenging to implement at scale in hardware. The team needed a way to cut connections without cutting quality.

They tested two strategies:

• Random sparsity: Before training, randomly delete each potential coupling with probability s. A network with 99% sparsity keeps only 1% of the original all-to-all connections. The mask is fixed for the entire training run.
• Modular sparsity: Partition oscillators into groups (modules). Oscillators connect all-to-all within a module, but modules are completely isolated from each other. With 16 modules, only ~6% of couplings survive.

Why cutting connections made things better

The improvement is not accidental , it has a clean dynamical explanation. In the dense all-to-all network, every oscillator is pulling on every other one simultaneously. Under strong enough coupling, the whole system can fall into catastrophic synchronization: all oscillators lock into the same phase, gradients go flat, and learning stalls. It is the oscillator equivalent of a neural network collapsing into a trivial solution.

Sparse and modular systems tolerate higher learning rates, indicating greater stability and less hyperparameter sensitivity. Dense models easily fall into catastrophic synchronization into a globally locked, zero-gradient state. Sparsity breaks this all-to-all coupling, allowing the network to maintain high-dimensional dynamics via chimera-like states (coexisting coherent and incoherent activity) and preserving the dimensionality of the dynamics even as the system scales up.

A chimera state , a term from physics , is when part of a network synchronizes while the rest stays chaotic and incoherent. It turns out this mixed regime is exactly what you want for learning: some structure, some diversity. Dense coupling kills it. Sparsity preserves it.

The numbers across tasks

The team validated sparsity across three different tasks, not just ImageNet:

• ImageNet 64x64 generation: Sparse Un-0 (16 modules) achieves FID 7.15 vs. the dense baseline's ~9.05 , a ~1.9 FID improvement at matched model size.
• CIFAR-10 generation: Both random and modular sparsity improve FID over the dense 1024-oscillator baseline, with improvements holding across a wide range of sparsity levels.
• MNIST1D classification: Sparsity recovers and exceeds dense performance as system size grows, with the optimal sparsity level scaling with the number of oscillators.

One important caveat: there is a critical sparsity threshold. Push past it and performance degrades, because oscillators become so disconnected that information can no longer propagate from input to output. Connectivity is a control knob with an optimal setting, not a dial you turn all the way up.

Modularity unlocks interpretability

A surprising bonus of the modular architecture is that it becomes mechanistically inspectable. Because modules are isolated from each other, you can swap individual modules between two generation runs , a technique called activation patching , and watch what changes. The team found modules have distinct functions in the task. Some modules appear to target background, others target car orientation, and others yet control vehicle color. This kind of functional specialization is reminiscent of what you see in disentangled VAEs and mixture-of-experts models, but it emerges here from the oscillator dynamics alone.

The diversity connection

The team also ran a Stacked MNIST benchmark , a test specifically designed to measure whether a generative model covers the full diversity of possible outputs, not just the most common ones. Each image combines three independently chosen MNIST digits across color channels, creating 1,000 possible combinations. Large, strongly coupled systems lose diversity: their generated distributions concentrate on fewer modes. Removing connections or dividing the dynamics into modules restores broader coverage.

Crucially, the FID improvements correlate almost entirely with recall (diversity), not precision. The correlation between FID improvement and recall is 0.924; with precision it is essentially zero (-0.114). Dense coupling is not just bad for training stability , it actively suppresses the model's ability to generate varied outputs.

What this means beyond Un-0

The broader implication is that connectivity density is a first-class design parameter for dynamical systems used as compute substrates , not just a hardware constraint to minimize. Sparsity is not simply a powerful tool for driving efficiency of implementation, but also a tool for shaping the trainability and dynamics of unconventional dynamical systems.

For the hardware roadmap, this result is doubly good news. Sparse connectivity is not only easier to implement on physical chips (fewer wires, less data movement), it also produces better models. The two goals , hardware feasibility and model quality , are now pointing in the same direction.

The Un-0 codebase is open-source on GitHub, with weights and training code available for both CIFAR-10 and ImageNet-64 variants. If you want to experiment with sparsity levels yourself, the repo is a plain-PyTorch implementation with no exotic dependencies.