Skip to main content
Back

McGill NLP's AfriqueQwen3.5 Beats Larger Models Across 50 African Languages

McGill NLP's post-trained AfriqueQwen3.5-4B covers 50 African languages and beats larger open models on the AfroBench-Lite suite.

Model, Full Access
  • McGill NLP released AfriqueQwen3.5-4B-50Langs-Instruct-v1, a Qwen 3.5 based model covering 50 African languages.
  • Post-trained with SFT, GRPO, and OPD on top of continued pretraining over 25B+ tokens.
  • Scores 61.6 overall on AfroBench-Lite, beating Gemma-3-4B, tiny-aya-earth, and Sunflower-Qwen3.5-9B.
  • Released under CC-BY-4.0; served with vLLM using the --language-model-only flag.
  • Part of the broader AfriqueLLM collection including 8B, 9B, and 14B variants.
  • Strong on translation, NLI, and math reasoning in African languages; weaker on knowledge-heavy MMLU-style QA.

McGill NLP has released a 50-language checkpoint built from Qwen 3.5 4B Base. AfriqueQwen3.5-4B-50Langs-Instruct-v1 underwent continued pretraining on African-language data, followed by supervised fine-tuning, reinforcement learning, and preference optimization. The model carries a CC BY 4.0 license.

Open multilingual models often provide uneven support for African languages because their training corpora contain limited text from those languages. The resulting gaps appear in translation, reading comprehension, mathematical reasoning, and intent classification. McGill NLP’s broader AfriqueLLM collection addresses those tasks with 4B, 8B, 9B, and 14B variants.

The 4B checkpoint tops the reported average

McGill NLP reports an average score of 61.6 for the 4B model on AfroBench-Lite, narrowly exceeding larger AfriqueLLM variants and several outside models in the same evaluation. These results come from the project team and have not been independently replicated here.

AfriqueQwen3.5 4B results on AfroBench-Lite
Benchmark Score
AfriMGSM54.2
AfriMMLU45.4
AfriXNLI62.2
Belebele66.9
Injongo58.1
SIB-20081.5
FLORES62.9
Overall average61.6
Selected overall scores from the same evaluation
Model Overall score
AfriqueQwen3.5 4B61.6
Sunflower-Qwen3.5 9B52.9
tiny-aya-earth42.8
Gemma 3 4B IT40.3

The suite combines reasoning, knowledge, language understanding, classification, and translation tasks:

  • AfriMGSM evaluates eight-shot mathematical reasoning across 64 languages.
  • AfriMMLU measures five-shot, multiple-choice knowledge answering across 64 languages.
  • AfriXNLI tests whether a premise entails, contradicts, or remains neutral toward another statement across languages.
  • Belebele evaluates reading comprehension across 122 language variants.
  • SIB-200 and Injongo measure topic classification and intent recognition in low-resource languages.