AILFM Keeps 3D-Stacked CPUs Cool While Running LLMs 31% Faster
A new scheduling framework called AILFM uses imitation learning to keep 3D-stacked CPUs cool during transformer inference, delivering up to 31 percent faster execution.
ProTakeaways
- AILFM schedules transformer kernels on 3D-stacked S-NUCA CPUs to prevent overheating during inference: arXiv
- Uses per-kernel Gaussian process Oracle plus a small neural network with Monte Carlo Dropout for uncertainty.
- Self-attention kernels lose over 35 percent IPS on distant cores; FFN loses under 6.5 percent.
- Delivers up to 31 percent higher performance versus baselines with under 5 percent runtime overhead.
- Reaches 96.7 percent Oracle accuracy at 3.2 percent overhead versus Oracle's 24.6 percent.
- Scales from 64-core to 216-core 3D stacks, hitting 1.64x speedup over 2D planar S-NUCA.
Running large transformer models on CPUs is no longer a fringe idea. With GPU supply tight and datacenter operators looking to squeeze more from existing silicon, general-purpose processors are becoming a real inference substrate. A new paper accepted at DAC 2026 pushes that idea onto exotic 3D-stacked CPU designs, which promise huge bandwidth gains but overheat aggressively under transformer workloads.
The work introduces AILFM, an active imitation learning scheduler that decides where to run each transformer kernel on a 3D-stacked chip so the silicon stays under a temperature threshold while still hitting near-peak throughput. The authors report that it outperforms state-of-the-art baselines and generalizes well across diverse LFM workloads.
Why 3D CPUs melt under transformers
The target hardware is a 3D-stacked Static Non-Uniform Cache Architecture, or 3D S-NUCA. These systems stack High-Bandwidth Memory atop many-core processors to boost bandwidth and locality, but they suffer from severe thermal challenges and uneven cache latencies due to their 3D Networks-on-Chip. Two nasty problems fall out of that geometry.
Stacking silicon multiplies power density without adding surface area to dissipate heat, so the chip runs hot and has to throttle via dynamic voltage and frequency scaling. Cores are also not equal even when architecturally identical. The paper measures this using Average Manhattan Distance (AMD), the mean hop count from one core to every other core through the on-chip network. Cores near the middle of the stack have lower AMD and lower cache access latency, but they also cook faster because they get less airflow and absorb heat from neighbors.