---
title: "Jadavpur University Finds Fine-Tuned PEGASUS Beats GPT-3.5 at Writing Research Titles"
subtitle: "A fine-tuned PEGASUS model beats GPT-3.5 and LLaMA-3-8B at generating research paper titles from abstracts, with a new social sciences dataset to boot"
url: "https://alphasignal.ai/news/jadavpur-university-finds-fine-tuned-pegasus-beats-gpt-3-5-at-writing-research"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-06-03T00:00:00.000Z"
updated: "2026-06-11T03:00:01.137Z"
type: "Paper"
topics:
  - "Llms"
  - "Data"
subtopics:
  - "Small Models"
  - "Datasets"
upvotes: 3797
source_url: "https://arxiv.org/abs/2606.05085"
comments: 0
truncated: true
---

# Jadavpur University Finds Fine-Tuned PEGASUS Beats GPT-3.5 at Writing Research Titles

> A fine-tuned PEGASUS model beats GPT-3.5 and LLaMA-3-8B at generating research paper titles from abstracts, with a new social sciences dataset to boot

Picking a title for a research paper is deceptively hard. It must succinctly convey the main contribution of a paper while remaining brief and appealing to a broad audience. Most researchers spend hours agonizing over it. A new paper published in the [International Journal on Digital Libraries](https://link.springer.com/journal/799) asks: what if a language model could do it for you?

Automatic title generation can be viewed as a form of abstractive text summarization, which aims to capture and represent the most salient aspects of a document in a compact form. The team from Jadavpur University and the Indian Association for the Cultivation of Science framed the problem exactly that way, and ran a systematic comparison of models ranging from compact fine-tuned PLMs (pre-trained language models) to GPT-3.5-turbo.

## The Bigger Models Don't Always Win

Fine-tuned PEGASUS-large outperforms other models, including fine-tuned LLaMA-3-8B and zero-shot GPT-3.5-turbo, across most metrics. That's a striking result. PEGASUS is orders of magnitude smaller than either of those two, yet it consistently produces titles closer to what human authors actually write.

The reason comes down to how PEGASUS was originally trained. In PEGASUS, important sentences are removed and masked from an input document and are generated together as one output sequence from the remaining sentences, similar to an extractive summary. That pre-training objective is essentially a rehearsal for exactly this task: compress a document into its most important sentence. Fine-tuning on abstract-title pairs then sharpens that skill further.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Automatic Generation of Titles for Research Papers Using Language Models (arXiv)](https://arxiv.org/abs/2606.05085)
- [Can pre-trained language models generate titles for research papers? (arXiv HTML)](https://arxiv.org/html/2409.14602v1)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (arXiv)](https://arxiv.org/abs/1912.08777)
- [Can Pre-trained Language Models Generate Titles for Research Papers? (Springer)](https://link.springer.com/chapter/10.1007/978-981-96-0865-2_13)
- [MoDeST: A dataset for Multi Domain Scientific Title Generation (ScienceDirect)](https://www.sciencedirect.com/science/article/pii/S0950705125006033)