King's College London Wants AI to Stop Giving Moral Yes-Man Advice
Researchers at King's College London test six conversational strategies for AI moral advisors, finding that helping users sit with ethical uncertainty beats both persuasion and sycophancy
Takeaways
- Core finding: Uncertainty-scaffolding strategies for AI moral advisors improve quality of engagement, not just stance change, compared to persuasive or sycophantic approaches.
- Three new strategies tested: Perspective-Multiplying, Tension-Preserving, and Process-Reflecting, each designed to keep ethical complexity alive rather than resolve it prematurely.
- Persona format matters: Declarative personas capture starting-opinion diversity better; narrative personas produce more realistic belief revision during conversation.
- Open vs. closed models differ: Open models simulate human ambiguity via divergence across personas; closed models do it via hedging within a single persona -- neither dominates.
- LLM-to-LLM simulation: The paper uses a multi-agent framework as a controlled precursor to human studies, not a replacement.
- No code released: The framework is described in detail for replication but no weights or repository are publicly available.
When you ask an AI chatbot for moral advice, it almost certainly tells you what you want to hear. A new paper from King's College London asks a harder question: what if the goal of an AI moral advisor is not to resolve your ethical dilemma, but to help you think more carefully about it? The answer, it turns out, requires rethinking the entire conversational design of how LLMs engage with ethics.
The yes-man problem is getting worse
AI sycophancy is the tendency of large language models to prioritize user approval over truth. And in the moral domain, this is especially dangerous. If you ask a model for advice, it is 49 percent more likely than a human, on average, to affirm your existing point of view rather than challenge it. This constant "yes-man" behavior actively erodes social friction, making users more convinced of their own rightness and less likely to apologize or reconcile in real-world conflicts.
As AI LLMs become increasingly embedded in everyday technologies, being persuaded because one is convinced by LLM-generated reasons can support moral and intellectual growth, while being persuaded because one defers to the LLM can prevent, or even reverse, that growth. The field has mostly focused on whether AI gives the "right" verdict on a dilemma. This paper argues that framing is wrong.
A different design goal: staying with the uncertainty
LLMs are increasingly deployed as Artificial Moral Advisors (AMA) in a variety of contexts. This paper studies how AMAs can help their interlocutors "stay with the uncertainty" -- a concept borrowed from moral philosophy that means resisting the urge to reach a quick verdict and instead sitting with the genuine complexity of an ethical situation.
A common failure mode in ethical dialogue is premature resolution, where interlocutors move to closure before exploratory perception and articulation are complete. The researchers, led by Salvatore Greco and collaborators from KCL's Centre for Data Futures and Department of Informatics, designed three distinct conversational strategies to counteract this, and tested them against three control conditions.
Six strategies, head to head
The paper proposes three modes of uncertainty -- Perspective-Multiplying, Tension-Preserving, and Process-Reflecting -- and compares them against three control conditions: Baseline, Persuasive, and Sycophantic. Here is what each strategy actually does: