IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
IrokoBench is a human-translated evaluation benchmark covering 17 typologically diverse low-resource African languages across three tasks: natural language inference (AfriXNLI), mathematical reasoning (AfriMGSM) and knowledge-based multiple-choice QA (AfriMMLU). The paper evaluates open and proprietary LLMs and documents a large gap between high-resource languages and African languages, with the best open model reaching about 63 percent of GPT-4o performance. It was published at NAACL 2025.
- Category
- Research
- Pricing
- Free / open
- Country
- 🌍 Pan-African
- Last verified
- 10 Oct 2026
{ "name": "IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models", "slug": "irokobench-a-new-benchmark-for-african-languages-in-the-age-of-large-language-models", "category": "RESEARCH", "country": "Pan-African", "docs_status": "LIVE", "licensing_required": "NONE", "verified": false, "last_verified": "2026-10-10", "website": "https://arxiv.org/abs/2406.03368", "documentation_url": null}Verification history
- 10 Oct 2026 · live
- 7 Oct 2026 · live
- 4 Oct 2026 · live
- 1 Oct 2026 · live
- 28 Sept 2026 · live
- 25 Sept 2026 · live
- 22 Sept 2026 · live
- 22 Sept 2026 · live
Automated checks run every few days. See all recent status changes
30-day uptime
Tags
Compare IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models
Side-by-side, verified specs against its closest nlp benchmark alternatives.
Related in Research
AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages
AfriQA is the first cross-lingual open-retrieval question answering benchmark for African languages, with more than 12,000 XOR-QA examples across 10 African languages. The paper shows that current automatic translation and multilingual retrieval methods perform poorly for these languages, where in-language digital content is scarce.
AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages
AfriSenti is a sentiment analysis benchmark of more than 110,000 tweets in 14 African languages spanning four language families, annotated by native speakers. It underpinned SemEval-2023 Task 12, a shared task that attracted more than 200 participants, and documents data collection, annotation and baseline methods for low-resource languages.
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition
MasakhaNER 2.0 introduces the largest human-annotated named entity recognition dataset for 20 African languages and studies Africa-centric cross-lingual transfer learning. The paper reports that choosing the best transfer language improves zero-shot F1 by an average of 14 points across the 20 languages compared with transferring from English.