---
title: "EPFL's FlexMerge Recovers 13.5% Accuracy by Rethinking AI Model Merging"
subtitle: "FlexMerge lets you dial the merged model size from 1x to full, revealing 13.5% accuracy jumps just by doubling storage."
url: "https://alphasignal.ai/news/epfl-s-flexmerge-recovers-13-5-accuracy-by-rethinking-ai-model-merging"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "AlphaSignal"
company_url: "https://alphasignal.ai/company/alphasignal"
published: "2026-04-14T00:00:00.000Z"
updated: "2026-08-23T22:07:53.858Z"
type: "Paper"
topics:
  - "Post Training"
  - "Llms"
subtopics:
  - "Distillation"
  - "Fine Tuning"
  - "Small Models"
upvotes: 3410
source_url: "https://arxiv.org/abs/2505.23209"
comments: 0
truncated: true
---

# EPFL's FlexMerge Recovers 13.5% Accuracy by Rethinking AI Model Merging

> FlexMerge lets you dial the merged model size from 1x to full, revealing 13.5% accuracy jumps just by doubling storage.

Model merging has become a popular way to combine many single-task fine-tuned checkpoints into one multi-task model without retraining. The catch is that squeezing everything into a single set of weights leaves a noticeable accuracy gap compared to the original fine-tuned models, and that gap widens as more tasks are added. A new framework called [FlexMerge](https://arxiv.org/abs/2505.23209) argues the field has been optimizing the wrong variable, and that a bit of extra storage buys a surprising amount of accuracy back.

## The one-model bottleneck

Merging combines multiple fine-tuned checkpoints into a single model without access to training data, sidestepping the compute cost and privacy headaches of joint multi-task training. A single set of weights, though, often cannot resolve parameter conflicts between tasks, and the resulting accuracy gap grows as more models are folded in.

The obvious workaround, keeping every checkpoint on disk, blows up storage linearly with the number of tasks. Some methods narrow the accuracy gap by pulling in extra validation data, but that dependency is hard to satisfy under privacy constraints or proprietary restrictions. Data-free merging, where the accuracy hit is still very real, is what remains.

## Merging as a spectrum, not a switch

The authors from EPFL and Carnegie Mellon reframe the problem by treating model size as a continuous knob rather than a choice between one fused model and M full checkpoints. At one extreme, retaining every fine-tuned model achieves maximal accuracy at the cost of larger overall size; at the other, fully merging into a single model minimizes storage but sacrifices accuracy. FlexMerge lets you stop anywhere in between, including at fractional sizes like 1.75x or 2.34x.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [Navigating the Accuracy-Size Trade-Off with Flexible Model Merging](https://arxiv.org/abs/2505.23209)
- [FlexMerge HTML paper](https://arxiv.org/html/2505.23209)
- [FlexMerge GitHub Repository](https://github.com/sacs-epfl/flexmerge)