Skip to main content
Back

Phasor Transformer Beats Attention With 100x Fewer Parameters on Time-Series

A new arxiv preprint proposes swapping self-attention for phase-shifts and a Fourier transform, cutting parameters by orders of magnitude on synthetic time-series.

Pro

Paper, Full Access
  • Sigdel's Phasor Transformer replaces self-attention with a fixed DFT plus trainable phase-shifts.
  • Token states are constrained to the unit circle, giving O(N log N) global mixing without attention maps.
  • Stacked blocks form the Large Phasor Model, a parameter-light backbone for autoregressive time-series.
  • On synthetic multi-frequency signals it beats persistence and stays competitive with attention baselines.
  • Uses roughly two orders of magnitude fewer parameters than a comparable self-attention encoder.
  • Primitives are open source via the PhasorFlow library from the same author.

A new single-author preprint on arxiv proposes a different way to build a Transformer for long time-series: throw out the dot-product attention matrix entirely and replace it with a fixed Discrete Fourier Transform plus a handful of trainable phase rotations. The Phasor Transformer, by Dibakar Sigdel, introduces a block that represents each token as a point on the unit circle and mixes tokens with an O(N log N) Fourier operation instead of an O(N²) attention map.

Stacked into a deep network, the author calls it the Large Phasor Model, or LPM. The pitch is that it lands in the same accuracy ballpark as attention while using roughly two orders of magnitude fewer parameters, which reframes what a competitive time-series backbone can look like.

The quadratic wall, again

Every additional token you push into a Transformer's context makes the attention matrix grow by N², which becomes painful for sensor streams, financial ticks, or physiological recordings where you want thousands of steps of history. The motivation here is that familiar bottleneck.

The Phasor Transformer sidesteps the cost by giving up learned token-to-token similarity in favor of a deterministic global mixer. Each block combines lightweight trainable phase shifts with parameter-free DFT token coupling, achieving global O(N log N) mixing without explicit attention maps. Google's FNet did essentially the same swap for language models; this paper pushes further by constraining all token states to live on the unit circle.