---
title: "Cohere Labs' Aya-23-8B Beats Mistral and Gemma Across 23 Languages"
subtitle: "Cohere Labs' Aya-23-8B brings instruction-following quality to 23 languages, beating Mistral and Gemma in multilingual benchmarks with an open-weights 8B model"
url: "https://alphasignal.ai/news/cohere-labs-aya-23-8b-beats-mistral-and-gemma-across-23-languages"
author: "AlphaSignal Newsroom"
author_url: "https://alphasignal.ai/editorial-team"
company: "Cohere"
company_url: "https://alphasignal.ai/company/cohere"
published: "2026-07-09T12:56:52.000Z"
updated: "2026-07-09T13:03:50.625Z"
type: "Model"
topics:
  - "Llms"
  - "Open Source"
subtopics:
  - "Translation"
  - "Small Models"
upvotes: 3237
source_url: "https://huggingface.co/CohereLabs/aya-23-8B"
comments: 0
truncated: true
---

# Cohere Labs' Aya-23-8B Beats Mistral and Gemma Across 23 Languages

> Cohere Labs' Aya-23-8B brings instruction-following quality to 23 languages, beating Mistral and Gemma in multilingual benchmarks with an open-weights 8B model

[Aya-23-8B](https://huggingface.co/CohereLabs/aya-23-8B) is an open-weights, instruction-tuned language model from Cohere Labs built to serve 23 languages at a quality level that most multilingual models have never reached. It sits at 8 billion parameters, runs on consumer-grade hardware, and consistently outperforms same-size models like Mistral-7B and Gemma-1.1-7B across translation, summarization, reasoning, and open-ended generation tasks in languages far beyond English.

## The problem it actually solves

Most LLM progress has been English-first. Models like Mistral and Gemma are heavily used by multilingual communities, but they were never explicitly trained for it. The result is a steep performance cliff the moment you leave high-resource European languages. Multilingual efforts like Aya 101, BLOOMZ, and mT0 made strides in expanding access to NLP for the world, but significant room for improvement remained relative to first-class citizen languages like English and Chinese.

The previous Aya model (Aya 101) tried to solve this with sheer breadth, covering 101 languages. But breadth came at a cost. Aya 23 is an experiment in depth vs breadth, exploring the impact of allocating more capacity to fewer languages that are included during pre-training. The bet paid off.

## What's under the hood

Aya-23-8B is a multilingual language model that supports 23 languages and builds on the Command family architecture, integrating the Aya Collection dataset. The base model is Cohere's Command R, which was pre-trained on data spanning all 23 target languages, giving the model a strong multilingual foundation before any fine-tuning begins.

The architecture uses several modern design choices from the [Aya 23 technical report](https://arxiv.org/html/2405.15032v1):

* **Grouped Query Attention (GQA):** Multiple query heads share a single key-value head, cutting inference-time memory usage significantly.
* **SwiGLU activations:** A gating mechanism in the feed-forward layers that consistently improves downstream task performance over standard ReLU.
* **Rotary Positional Embeddings (RoPE):** Enables better generalization to longer contexts than alternatives like ALiBi.

_Preview only. Full text requires an AlphaSignal Pro account._

## Sources

- [CohereLabs/aya-23-8B · Hugging Face](https://huggingface.co/CohereLabs/aya-23-8B)
- [Aya 23: Open Weight Releases to Further Multilingual Progress (arXiv)](https://arxiv.org/html/2405.15032v1)
- [Papers Explained 151: Aya 23 – Medium](https://ritvik19.medium.com/papers-explained-151-aya-23-d01605c3ee80)
- [A Deepdive into Aya Expanse – Hugging Face Blog](https://huggingface.co/blog/aya-expanse)
- [aya-23-8B model overview – AI Models FYI](https://www.aimodels.fyi/models/huggingFace/aya-23-8b-coherelabs)