ModelsDocumentationResearchEcosystem
Research & Methodology

The Science Behind
OribAI 1.0

A deep dive into our fine-tuning approach, training data curation, evaluation methodology, and honest performance findings.

open_in_newView Model Card

Key Numbers

27,498

Training Pairs

Hausa + Yoruba curated data

3.22

Yoruba Perplexity (AYA)

Strong — fluent, coherent

14B

Model Parameters

Qwen2.5-14B-Instruct base

3

Training Epochs

LR 2e-4 · LoRA rank 32

Architecture

Qwen2.5-14B + LoRA Fine-tuning

OribAI-14B extends Qwen2.5-14B-Instruct — Alibaba's state-of-the-art multilingual model — with parameter-efficient LoRA adapters trained specifically on Hausa and Yoruba conversational data using the Unsloth framework for optimized throughput.

  • chevron_rightBase: Qwen2.5-14B-Instruct (Apache 2.0)
  • chevron_rightLoRA rank 32, alpha 64 — targets attention layers
  • chevron_right4-bit NF4 quantization during training via bitsandbytes
  • chevron_rightTRL SFTTrainer · 3 epochs · LR 2e-4
  • chevron_rightChat template: Qwen2.5 format with system prompt
LoRA Adapter Layers

Training Methodology

dataset
01

Data Curation

Training data sourced from CohereForAI/aya_dataset (Hausa + Yoruba multilingual instructions), CohereForAI/aya_evaluation_suite (human-annotated evaluation pairs), mangaphd/hausa_aug_lex (14,663 Hausa lexical pairs), and llama-lang-adapt/AfriInstruct-Data (African instruction data). Total: 27,498 unique conversational pairs.

model_training
02

Base Model Selection

OribAI-14B is built on Qwen2.5-14B-Instruct — a state-of-the-art multilingual instruction-following model with strong cross-lingual transfer capabilities, providing a robust foundation for African language fine-tuning.

tune
03

LoRA Fine-tuning

Parameter-efficient fine-tuning via LoRA (rank 32, alpha 64) using Unsloth and TRL SFTTrainer. Training ran for 3 epochs with a learning rate of 2e-4 and 4-bit NF4 quantization to reduce memory overhead during training.

analytics
04

Evaluation

Perplexity measured on 50 held-out samples from the CohereForAI/aya_dataset train split. Yoruba achieves 3.22 PPL (strong, production-ready). Hausa achieves 62.54 PPL — functional for short factual Q&A but unreliable for open-ended generation. Hausa improvement is planned for v2.

warning
05

Known Limitations

Hausa open-ended generation may hallucinate or go off-topic. Coverage of Yoruba dialects may be uneven. The model has not been evaluated on formal, legal, or medical language use cases. Responses may mix languages occasionally (code-switching).

Data Sources & Citation

open_in_new

OribAI: Hausa and Yoruba Language Model

Alkamal01

HuggingFace · 2026

oribai-14b-v1open_in_new

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Singh, S., et al. (CohereForAI)

arXiv · 2024

aya-datasetopen_in_new

AfriInstruct: Instruction Tuning of African Languages

llama-lang-adapt

HuggingFace · 2024

AfriInstruct-Data

BibTeX

citation.bib
@misc{oribai2026,
  author    = {Alkamal01},
  title     = {OribAI: Hausa and Yoruba Language Model},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Alkamal01/oribai-14b-hausa-yoruba-v1}
}