KiToke retains 89.2% of baseline performance with 1% of video tokens

Long video inputs can produce tens of thousands of visual tokens. Transformer self-attention compares token pairs, so its compute and memory costs rise roughly with the square of the sequence length. A new training-free method called KiToke selects and merges projected visual tokens before language-model prefill, reducing the sequence without changing model weights. Tests at 1% retention show smaller performance losses than competing token-compression methods.

Why 1% retention exposes weak selectors

A 1% retention budget discards 99 of every 100 tokens, leaving little room for duplicate information. Near-identical views of a static background can consume much of the remaining budget and displace a brief action, small object, or unusual frame. Temporal mistakes add another failure mode: merging tokens across a scene cut combines evidence from unrelated shots.

Existing training-free methods address parts of the problem, but their selection rules become less effective as the budget shrinks:

• PruneVID treats temporal and spatial redundancy separately, which can miss duplicates spanning both dimensions and perform poorly on dynamic motion.
• VidCom2 measures token uniqueness against an averaged prototype. Dynamic footage can make that prototype unstable, and the score does not directly control redundancy among retained tokens.
• FastVID uses density-based ranking within temporal intervals. Rare, informative tokens can receive low priority, while similar tokens still consume the limited budget.

Global density finds scarce evidence

KiToke treats selection as a global diversity problem across the full video. For each visual-token embedding, it sums Gaussian-kernel similarity against every other token. Repeated frames, static backgrounds, and slowly changing regions produce high-density scores because many nearby embeddings look alike. Unusual tokens have fewer close neighbors and receive lower density scores.