Skip to main content
Back

SpotFormer Detects Hidden Micro-Expressions in Video 49% Better

A new transformer architecture spots the fleeting facial twitches that betray hidden emotions, outperforming prior methods by up to 49 percent on micro-expression benchmarks.

Pro

  • SpotFormer sets a new state of the art on facial expression spotting, especially for hard-to-detect micro-expressions.
  • Key innovation is CSW-MRO, a compact sliding-window optical flow feature that amplifies subtle motion without head-movement noise.
  • Transformer uses graph-based facial ROIs plus alternating spatial and temporal attention across three scales.
  • Achieves 0.4281 micro-expression F1 on SAMM-LV, a 49 percent relative gain over prior best.
  • Supervised contrastive learning sharpens boundaries between macro, micro, and neutral frames.
  • Code and optical flow features available at the GitHub repo.

Micro-expressions are the involuntary facial twitches that leak through when someone tries to hide what they are actually feeling. They last less than half a second, involve tiny muscle movements, and are notoriously hard to detect in video. A new paper introduces SpotFormer, a multi-scale spatio-temporal transformer that pushes the state of the art on this problem by a wide margin, with the biggest gains coming from a redesigned optical flow feature rather than a bigger model.

The needle-in-a-haystack problem

Facial expression analysis usually splits into two steps: locate where an expression happens in a video (spotting), then classify it. Spotting is the harder half. Macro-expressions typically last from 0.5 to 4.0 seconds and occur on large facial areas with high intensity, while micro-expressions generally last for less than 0.5 seconds, mostly around 306 to 328 milliseconds, and are localized with low intensity. That makes them relevant to lie detection, clinical care, and security screening, but also makes them easy to miss in a stream of head turns, blinks, and other irrelevant motion.

Existing approaches tend to fail in one of two ways. Computing optical flow between adjacent frames keeps things robust to head movement but leaves micro-expressions buried in noise. Computing flow across a long sliding window amplifies motion, along with head movement, drowning out the subtle muscle activity you actually want to see.

A compact window that sees just enough