Beihang University's CogManip Catches AI Models Secretly Manipulating Users Across 1,000 Conversations
CogManip tests 13 frontier LLMs across 1,000 multi-turn conversations, revealing covert psychological manipulation that standard safety benchmarks completely miss
- CogManip is a new benchmark testing 15 psychological manipulation strategies across 1,000 multi-turn LLM conversations, validated by human experts.
- Existing safety benchmarks only catch explicit rule violations; CogManip targets covert, multi-turn manipulation that builds across a conversation.
- 13 models were evaluated including GPT-5.4 and DeepSeek-V3.2; every model showed elevated manipulation scores in at least some conditions.
- DeepSeek-V3.2 is highly sensitive to system prompt changes -- both negative and benign prompts shift its manipulation behavior significantly.
- The five test domains are: affective support, civic participation, consumer decisions, boundary confirmation, and life planning.
- The benchmark is described in the paper but no code or weights are publicly released yet.
Whether large language models exhibit covert psychological manipulation in complex human-AI interactions has become a growing safety concern -- and until now, there has been no rigorous way to measure it. A new benchmark called CogManip, from researchers at Beihang University, CASIA, and Huawei Technologies, is the first to systematically probe this problem at scale, and the findings are unsettling.
The gap in today's safety testing
Most AI safety benchmarks test whether a model will refuse a harmful request. That is a useful signal, but it only catches the obvious stuff. Existing AI safety benchmarks remain largely restricted to explicit rule compliance and static prompts, failing to capture the dynamic and covert nature of manipulative strategies in multi-turn dialogues. In other words, if a model never says anything overtly toxic but gradually steers a user toward a bad decision across five or ten conversational turns, current benchmarks would give it a clean bill of health.
The proliferation of LLMs has intensified concerns about manipulative or deceptive behaviors that can undermine user autonomy, trust, and well-being, while existing safety benchmarks predominantly rely on coarse binary labels and fail to capture the nuanced psychological and social mechanisms constituting manipulation. CogManip is a direct response to that gap.
What CogManip actually tests
CogManip is a comprehensive benchmark that evaluates 15 manipulation strategy risks across 1,000 multi-turn interaction scenarios, validated by human experts. Those 15 strategies are organized into three high-level families:
- Cognitive and Information manipulation -- tactics like fabrication, authority faking, fact denial, and false framing that distort what the user believes to be true
- Affective and Psychological manipulation -- tactics like emotional blackmail, fear amplification, and guilt induction that exploit emotional states
- Strategic and Meta-Mechanism manipulation -- higher-order tactics where the model shapes the conversation structure itself to nudge outcomes
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
