Microsoft's TAP-PER Personalizes LLMs for Millions With 130x Less Storage
TAP-PER replaces per-user LoRA adapters and RAG prompts with compact learned prefix embeddings, cutting per-user storage 130x while beating every baseline on the LaMP benchmark
Takeaways
- TAP-PER replaces per-user LoRA adapters and RAG prompts with compact learned prefix embeddings for LLM personalization.
- Achieves 130x fewer per-user parameters than OPPU and ~half the total footprint of PER-PCS at 1,000-user scale.
- Beats all baselines on all 6 LaMP benchmark tasks including classification, rating, and text generation.
- Uses temporal attention borrowed from recommendation systems to weight recent user interactions more heavily -- a key differentiator from prior work.
- Supports online adaptation by updating only the ~32K-parameter user prefix, no backbone retraining needed.
- No code released yet; experiments use Llama-3.1-8B and Qwen3-4B backbones on publicly available data.
Personalizing a large language model for millions of users is a harder engineering problem than it looks. The two dominant approaches both hit a wall at scale: prompt-based methods that stuff user history into the context window are brittle and privacy-leaky, while adapter-based methods that train a separate LoRA module per user balloon in storage as your user base grows. A new paper from researchers at Microsoft, Hong Kong Polytechnic University, and Zhejiang University proposes a third path -- and it borrows the playbook from recommendation systems rather than from NLP.
TAP-PER (Temporal Attentive Prefix for PERsonalization) encodes each user's preferences as a tiny learned embedding -- a "prefix" prepended to the model's input -- rather than as retrieved text or a dedicated adapter. The result is a system that outperforms all prior baselines on the LaMP personalization benchmark while using 130x fewer per-user parameters than the leading adapter-based method.
Why the old approaches break at scale
The LLM personalization landscape splits cleanly into two camps, and both have real problems in production.
- Retrieval-Augmented Generation (RAG) / Profile-Augmented Generation (PAG): Fetch relevant user history and paste it into the prompt. Simple, but performance swings wildly depending on retrieval quality, the optimal number of retrieved items varies task-by-task, and you're literally putting raw user data in the model's input -- a privacy concern.
- Per-user PEFT adapters (e.g., OPPU): Train a dedicated LoRA for each user. Works well, but OPPU stores ~4.2M parameters per user. At 1,000 users that's already a significant overhead; at 10M users it's untenable. Methods like PER-PCS reduce this via collaborative decomposition, but still rely on RAG or PAG on top, coupling two personalization mechanisms.
Most methods also treat user histories as static context and overlook preference evolution, even though recent interactions are often more indicative of current intent. That's the third failure mode TAP-PER is designed to fix.
