Masakhane's AfrIFact Exposes AI Fact-Checking Failures Across 10 African Languages
A new multilingual benchmark exposes how embedding models and LLMs fail at fact-checking across 10 African languages, and shows what fixes work.
- AfrIFact benchmarks IR, evidence extraction and fact-checking across 10 African languages with 18,000+ claims.
- Covers two domains: health (WHO-sourced) and culture-news (Wikipedia and XL-Sum).
- Embedding models lose 50-70% performance when retrieving across languages, revealing weak cross-lingual alignment.
- Few-shot prompting improves AfriqueQwen-14B by 43 points, from near-random to 77% accuracy.
- QLoRA 8-bit fine-tuning adds another 26 points and teaches models to actually use evidence spans.
- Dataset on HuggingFace, code on GitHub, accepted at EMNLP 2026.
Online fact-checking gets technically difficult when a Wolof claim concerns a disease outbreak and the strongest source is written in English or another African language. The AfrIFact benchmark measures that multilingual chain, revealing sharp losses in cross-language retrieval and near-chance zero-shot verification for many open models.
Researchers from Masakhane NLP, Saarland University, McGill University, the University of Waterloo, and collaborating institutions built more than 18,000 human-written claims with annotated source documents and evidence spans. The collection covers English and 10 African languages: Amharic, Hausa, Igbo, Oromo, Shona, Swahili, Twi, Wolof, Yoruba, and isiZulu.
Three tasks, one pipeline
AfrIFact separates automated fact-checking into three stages, allowing researchers to locate failures instead of reducing the entire process to one classification score:
- Information retrieval: Find relevant source documents for a claim in a large multilingual corpus.
- Evidence extraction: Select the sentences that support or contradict the claim.
- Claim verification: Assign
SUPPORTS,REFUTES, orNOT_ENOUGH_INFORMATION. The third label applies when the available evidence cannot settle the claim.
The benchmark spans two domains with distinct sourcing methods. AfrIFact-Health uses World Health Organization articles translated from English, with claims and annotations transferred across language versions. AfrIFact-Culture-News draws from native-language Wikipedia articles about African political leaders, food, religion, sport, and currencies, supplemented by XL-Sum news articles.
This story is for Pro members
You've reached the end of the free preview. Upgrade to AlphaSignal Pro to read the full article - and everything else behind the paywall.
Comments
