Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2/README.md

---
library_name: transformers
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
language:
- ln
tags:
- lingala
- congo
- qlora
- lora
- fine-tuned
- text-generation
- conversational
license: llama3.1
pipeline_tag: text-generation
model-index:
- name: Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Lingala Instruction Corpus (internal, 964-example test set)
      type: custom
    metrics:
    - type: accuracy
      name: Mean Token Accuracy
      value: 0.7120
    - type: perplexity
      name: Perplexity
      value: 17.7211
    - type: rouge
      name: ROUGE-L F1
      value: 0.2379
    - type: bleu
      name: BLEU
      value: 0.0459
    - type: rouge
      name: ROUGE-1
      value: 0.2939
    - type: rouge
      name: ROUGE-2
      value: 0.0789
---

# 🇨🇬 Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2

A **Meta-Llama-3.1-8B-Instruct** model adapted to **Lingala** through supervised fine-tuning (QLoRA/LoRA), developed by **Congo Digital Services (CDS)**. This model is intended for conversation, text generation, summarization, translation, and classification in Lingala.

> 🔗 Sister model: QLoRA adapters only are available at [`Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters`](https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters)

## Model Details

### Model Description

- **Developed by:** Congo Digital Services (CDS SARL) — [congo-digital.com](https://www.congo-digital.com/)
- **Base model:** [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
- **Adaptation method:** Supervised fine-tuning via QLoRA/LoRA
- **Language:** Lingala (ln)
- **Model type:** Causal language model, 8B parameters, merged
- **License:** Llama 3.1 Community License
- **Functional objectives:** Conversation, text generation, summarization, translation, classification, and conversational responses tailored to Lingala

### Model Sources

- **Organization:** [Congo-digital-service on Hugging Face](https://huggingface.co/Congo-digital-service)
- **Contact:** [info@congo-digital.com](mailto:info@congo-digital.com)

## Uses

### Direct Use

This model can be used directly for conversational text generation in Lingala via the standard `transformers` API, or deployed behind a compatible inference server (see Deployment Infrastructure below).

### Out-of-Scope Use

This model is not intended for use cases requiring critical factual accuracy (medical, legal domains) without human oversight, nor for certified professional translation tasks.

## How to Get Started with the Model

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": "Loba na ngai na lingala : ndenge nini okoki kosala mombongo na Kinshasa ?"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Training Details

### Training Data

A unified corpus of **4,818 examples** across five stylistic categories:

| Category | Examples |
|---|---|
| Urban | 1,425 |
| Educational | 1,063 |
| Summarization | 1,041 |
| Formal | 889 |
| Translation | 400 |

**Split:** stratified 80/20 split → 3,854 initial training examples / 964 test examples (unaugmented, held out for independent evaluation).

**Augmentation:** applied only to the training set to balance categories at 1,140 examples each, resulting in a **final training set of 5,700 examples**.

### Training Procedure

- **Environment:** Google Colab, with checkpoints and run logs saved to Google Drive

#### Training Hyperparameters

| Parameter | Value |
|---|---|
| Epochs | 3 |
| Batch size | 2 |
| Gradient accumulation steps | 8 (effective batch = 16) |
| Initial learning rate | 2 × 10⁻⁵ |
| Scheduler | Cosine |
| Intermediate evaluations | Every 200 steps |

#### Training Convergence

| Step | Training Loss | Validation Loss | Entropy | Mean Token Accuracy |
|---|---|---|---|---|
| 200 | 1.5638 | 1.5747 | 1.6141 | 67.15% |
| 400 | 1.3008 | 1.3514 | 1.3425 | 69.81% |
| 600 | 1.2714 | 1.2901 | 1.3146 | 70.83% |
| 800 | 1.2072 | 1.2741 | 1.2669 | 71.14% |
| 1000 | 1.2211 | 1.2720 | 1.2690 | 71.15% |
| 1071 | 1.1956 | 1.2720 | 1.2689 | 71.19% |

Validation loss decreases steadily throughout training with no sign of overfitting.

## Evaluation

Evaluation performed on the independent test set (964 examples), held out during training.

### Final Results

| Metric | Base Model | Fine-tuned Model | Improvement |
|---|---|---|---|
| **ROUGE-L F1** (primary metric) | 0.1521 | **0.2379** | **+56.4%** |
| Mean Token Accuracy | — | **71.20%** | — |
| Token F1 (lexical) | 0.1721 | — | — |
| Perplexity | — | **17.7211** | — |
| ROUGE-1 | — | 0.2939 | — |
| ROUGE-2 | — | 0.0789 | — |
| BLEU (SacreBLEU for base) | 2.3280 | 0.0459 | not representative* |
| Exact match | 0.00% | not computed | — |

\* *BLEU proves unrepresentative for open-ended generation tasks; progress is primarily measured via ROUGE-L.*

### Summary

The QLoRA adaptation delivers a robust, measurable performance gain: the fine-tuned model handles Lingala's linguistic structures significantly better than the base model, with a +56.4% relative gain in ROUGE-L F1.

### Human Evaluation

- **Operational target:** ≥ 85% of responses rated acceptable (per the project's Terms of Reference)
- **Dimensions evaluated:** Lingala correctness, coherence and meaning, instruction adherence, business relevance

## Target Deployment Infrastructure

| Component | Detail |
|---|---|
| Inference server | vLLM deployed on GPU pods |
| Gateway / auth | LiteLLM v1.87.0 + PostgreSQL 16 |
| Endpoint | `POST ${VLLM_BASE_URL}/v1/chat/completions` |
| Format | OpenAI Chat Completions compatible (`choices[0].message.content`) |
| Access interface | LoBAI (LibreChat-based) with Keycloak / OpenID Connect |

## Traceability

- **Merged model:** `Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2`
- **QLoRA adapters:** [`Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters`](https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters)
- **Version manifest and checksums (SHA-256):** centralized in the project's delivery GitHub repository

## Bias, Risks, and Limitations

This model was trained on a moderately sized corpus (5,700 augmented examples) covering five stylistic registers. Performance may vary outside these registers, particularly on regional Lingala dialects not represented in the corpus, or on specialized technical domains absent from the training set. Users should validate model outputs before any high-stakes use.

## Environmental Impact

Training was performed on Google Colab infrastructure. Carbon emissions can be estimated using the [ML Impact calculator](https://mlco2.github.io/impact#compute) ([Lacoste et al., 2019](https://arxiv.org/abs/1910.09700)).

## Citation

**BibTeX:**

```bibtex
@misc{cds2026lingala,
  title={Llama-3.1-8B-Instruct-Lingala-QLoRA},
  author={Congo Digital Services},
  year={2026},
  howpublished={\url{https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2}}
}
```

## Contact

- **Organization:** Congo Digital Services (CDS SARL)
- **Website:** [congo-digital.com](https://www.congo-digital.com/)
- **Email:** [info@congo-digital.com](mailto:info@congo-digital.com)