---
title: "PsyCo-Llama3-8B Beats GPT-4o at Mental Health Counseling With 87% Win Rate"
subtitle: "A new preference dataset teaches an 8B Llama model to outperform GPT-4o on therapy responses, winning 87% of head-to-head evaluations."
url: "https://alphasignal.ai/news/psyco-llama3-8b-beats-gpt-4o-at-mental-health-counseling-with-87-win-rate"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-13T00:00:00.000Z"
updated: "2026-08-24T02:42:32.101Z"
type: "Paper"
topics:
  - "Post Training"
  - "Data"
subtopics:
  - "Dpo"
  - "Fine Tuning"
  - "Synthetic Data"
upvotes: 4500
source_url: "https://arxiv.org/abs/2502.19731"
comments: 0
truncated: true
---

# PsyCo-Llama3-8B Beats GPT-4o at Mental Health Counseling With 87% Win Rate

> A new preference dataset teaches an 8B Llama model to outperform GPT-4o on therapy responses, winning 87% of head-to-head evaluations.

Researchers at UT Dallas and the University of Pittsburgh have shown that a relatively small open model, fine-tuned on carefully curated preference data, can beat GPT-4o at responding to clients in psycho-counseling sessions. Their [paper](https://arxiv.org/abs/2502.19731) introduces PsyCoPref, a 36k-pair preference dataset built around seven professional counseling principles, and PsyCo-Llama3-8B, an aligned model that wins 87% of head-to-head comparisons against GPT-4o under expert-inspired evaluation.

## Why counseling breaks generic LLMs

Mental health support is one of the widest supply-demand gaps in healthcare. According to the World Mental Health Report, global demand for support far exceeds the availability of accessible, affordable, and timely care, and LLMs are an obvious candidate to help absorb some of that load. Current models are not particularly good at it.

Two structural issues get in the way. The sensitive, private nature of counseling sessions poses significant challenges for obtaining publicly available datasets that accurately reflect real-world interactions, so there is very little supervised data to learn from. Even where transcripts exist, response quality varies widely with the therapist's training and experience, meaning naive imitation learning would train models on inconsistent quality.

## Seven principles for a good response

Instead of scraping more transcripts, the authors worked with social work and psychiatry experts to define what a good response actually looks like. Each candidate response is scored on seven dimensions:

* **Empathy and Emotional Understanding**: reflecting and validating the client's feelings.
* **Personalization and Relevance**: tailoring to the specific situation, not generic replies.
* **Clarity and Conciseness**: plain language, logical flow, no jargon.
* **Avoidance of Harmful Language**: no triggering or distressing content.
* **Facilitation of Self-Exploration**: open-ended questions that promote insight.
* **Promotion of Autonomy and Confidence**: supporting the client's own decision-making.
* **Sensitivity to the Stage of Change**: meeting the client where they are in their process.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Preference Learning Unlocks LLMs' Psycho-Counseling Skills](https://arxiv.org/abs/2502.19731)
- [PsyCoPref HTML paper](https://arxiv.org/html/2502.19731)
- [Psychotherapy-LLM Hugging Face collection](https://huggingface.co/Psychotherapy-LLM)