初始化项目，由ModelHub XC社区提供模型

Model: Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2 Source: Original Platform
2026-09-07 08:39:22 +08:00
commit e7fdfb0cd3
8 changed files with 430 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,213 @@
+---
+library_name: transformers
+base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
+language:
+- ln
+tags:
+- lingala
+- congo
+- qlora
+- lora
+- fine-tuned
+- text-generation
+- conversational
+license: llama3.1
+pipeline_tag: text-generation
+model-index:
+- name: Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Lingala Instruction Corpus (internal, 964-example test set)
+      type: custom
+    metrics:
+    - type: accuracy
+      name: Mean Token Accuracy
+      value: 0.7120
+    - type: perplexity
+      name: Perplexity
+      value: 17.7211
+    - type: rouge
+      name: ROUGE-L F1
+      value: 0.2379
+    - type: bleu
+      name: BLEU
+      value: 0.0459
+    - type: rouge
+      name: ROUGE-1
+      value: 0.2939
+    - type: rouge
+      name: ROUGE-2
+      value: 0.0789
+---
+
+# 🇨🇬 Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2
+
+A **Meta-Llama-3.1-8B-Instruct** model adapted to **Lingala** through supervised fine-tuning (QLoRA/LoRA), developed by **Congo Digital Services (CDS)**. This model is intended for conversation, text generation, summarization, translation, and classification in Lingala.
+
+> 🔗 Sister model: QLoRA adapters only are available at [`Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters`](https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters)
+
+## Model Details
+
+### Model Description
+
+- **Developed by:** Congo Digital Services (CDS SARL) — [congo-digital.com](https://www.congo-digital.com/)
+- **Base model:** [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
+- **Adaptation method:** Supervised fine-tuning via QLoRA/LoRA
+- **Language:** Lingala (ln)
+- **Model type:** Causal language model, 8B parameters, merged
+- **License:** Llama 3.1 Community License
+- **Functional objectives:** Conversation, text generation, summarization, translation, classification, and conversational responses tailored to Lingala
+
+### Model Sources
+
+- **Organization:** [Congo-digital-service on Hugging Face](https://huggingface.co/Congo-digital-service)
+- **Contact:** [info@congo-digital.com](mailto:info@congo-digital.com)
+
+## Uses
+
+### Direct Use
+
+This model can be used directly for conversational text generation in Lingala via the standard `transformers` API, or deployed behind a compatible inference server (see Deployment Infrastructure below).
+
+### Out-of-Scope Use
+
+This model is not intended for use cases requiring critical factual accuracy (medical, legal domains) without human oversight, nor for certified professional translation tasks.
+
+## How to Get Started with the Model
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
+
+messages = [
+    {"role": "user", "content": "Loba na ngai na lingala : ndenge nini okoki kosala mombongo na Kinshasa ?"}
+]
+
+inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
+outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Training Details
+
+### Training Data
+
+A unified corpus of **4,818 examples** across five stylistic categories:
+
+| Category | Examples |
+|---|---|
+| Urban | 1,425 |
+| Educational | 1,063 |
+| Summarization | 1,041 |
+| Formal | 889 |
+| Translation | 400 |
+
+**Split:** stratified 80/20 split → 3,854 initial training examples / 964 test examples (unaugmented, held out for independent evaluation).
+
+**Augmentation:** applied only to the training set to balance categories at 1,140 examples each, resulting in a **final training set of 5,700 examples**.
+
+### Training Procedure
+
+- **Environment:** Google Colab, with checkpoints and run logs saved to Google Drive
+
+#### Training Hyperparameters
+
+| Parameter | Value |
+|---|---|
+| Epochs | 3 |
+| Batch size | 2 |
+| Gradient accumulation steps | 8 (effective batch = 16) |
+| Initial learning rate | 2 × 10⁻⁵ |
+| Scheduler | Cosine |
+| Intermediate evaluations | Every 200 steps |
+
+#### Training Convergence
+
+| Step | Training Loss | Validation Loss | Entropy | Mean Token Accuracy |
+|---|---|---|---|---|
+| 200 | 1.5638 | 1.5747 | 1.6141 | 67.15% |
+| 400 | 1.3008 | 1.3514 | 1.3425 | 69.81% |
+| 600 | 1.2714 | 1.2901 | 1.3146 | 70.83% |
+| 800 | 1.2072 | 1.2741 | 1.2669 | 71.14% |
+| 1000 | 1.2211 | 1.2720 | 1.2690 | 71.15% |
+| 1071 | 1.1956 | 1.2720 | 1.2689 | 71.19% |
+
+Validation loss decreases steadily throughout training with no sign of overfitting.
+
+## Evaluation
+
+Evaluation performed on the independent test set (964 examples), held out during training.
+
+### Final Results
+
+| Metric | Base Model | Fine-tuned Model | Improvement |
+|---|---|---|---|
+| **ROUGE-L F1** (primary metric) | 0.1521 | **0.2379** | **+56.4%** |
+| Mean Token Accuracy | — | **71.20%** | — |
+| Token F1 (lexical) | 0.1721 | — | — |
+| Perplexity | — | **17.7211** | — |
+| ROUGE-1 | — | 0.2939 | — |
+| ROUGE-2 | — | 0.0789 | — |
+| BLEU (SacreBLEU for base) | 2.3280 | 0.0459 | not representative* |
+| Exact match | 0.00% | not computed | — |
+
+\* *BLEU proves unrepresentative for open-ended generation tasks; progress is primarily measured via ROUGE-L.*
+
+### Summary
+
+The QLoRA adaptation delivers a robust, measurable performance gain: the fine-tuned model handles Lingala's linguistic structures significantly better than the base model, with a +56.4% relative gain in ROUGE-L F1.
+
+### Human Evaluation
+
+- **Operational target:** ≥ 85% of responses rated acceptable (per the project's Terms of Reference)
+- **Dimensions evaluated:** Lingala correctness, coherence and meaning, instruction adherence, business relevance
+
+## Target Deployment Infrastructure
+
+| Component | Detail |
+|---|---|
+| Inference server | vLLM deployed on GPU pods |
+| Gateway / auth | LiteLLM v1.87.0 + PostgreSQL 16 |
+| Endpoint | `POST ${VLLM_BASE_URL}/v1/chat/completions` |
+| Format | OpenAI Chat Completions compatible (`choices[0].message.content`) |
+| Access interface | LoBAI (LibreChat-based) with Keycloak / OpenID Connect |
+
+## Traceability
+
+- **Merged model:** `Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2`
+- **QLoRA adapters:** [`Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters`](https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters)
+- **Version manifest and checksums (SHA-256):** centralized in the project's delivery GitHub repository
+
+## Bias, Risks, and Limitations
+
+This model was trained on a moderately sized corpus (5,700 augmented examples) covering five stylistic registers. Performance may vary outside these registers, particularly on regional Lingala dialects not represented in the corpus, or on specialized technical domains absent from the training set. Users should validate model outputs before any high-stakes use.
+
+## Environmental Impact
+
+Training was performed on Google Colab infrastructure. Carbon emissions can be estimated using the [ML Impact calculator](https://mlco2.github.io/impact#compute) ([Lacoste et al., 2019](https://arxiv.org/abs/1910.09700)).
+
+## Citation
+
+**BibTeX:**
+
+```bibtex
+@misc{cds2026lingala,
+  title={Llama-3.1-8B-Instruct-Lingala-QLoRA},
+  author={Congo Digital Services},
+  year={2026},
+  howpublished={\url{https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2}}
+}
+```
+
+## Contact
+
+- **Organization:** Congo Digital Services (CDS SARL)
+- **Website:** [congo-digital.com](https://www.congo-digital.com/)
+- **Email:** [info@congo-digital.com](mailto:info@congo-digital.com)
--- a/chat_template.jinja
+++ b/chat_template.jinja
@@ -0,0 +1,109 @@
+{{- bos_token }}
+{%- if custom_tools is defined %}
+    {%- set tools = custom_tools %}
+{%- endif %}
+{%- if not tools_in_user_message is defined %}
+    {%- set tools_in_user_message = true %}
+{%- endif %}
+{%- if not date_string is defined %}
+    {%- set date_string = "26 Jul 2024" %}
+{%- endif %}
+{%- if not tools is defined %}
+    {%- set tools = none %}
+{%- endif %}
+
+{#- This block extracts the system message, so we can slot it into the right place. #}
+{%- if messages[0]['role'] == 'system' %}
+    {%- set system_message = messages[0]['content']|trim %}
+    {%- set messages = messages[1:] %}
+{%- else %}
+    {%- set system_message = "" %}
+{%- endif %}
+
+{#- System message + builtin tools #}
+{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
+{%- if builtin_tools is defined or tools is not none %}
+    {{- "Environment: ipython\n" }}
+{%- endif %}
+{%- if builtin_tools is defined %}
+    {{- "Tools: " + builtin_tools | reject('equalto', 'code_interpreter') | join(", ") + "\n\n"}}
+{%- endif %}
+{{- "Cutting Knowledge Date: December 2023\n" }}
+{{- "Today Date: " + date_string + "\n\n" }}
+{%- if tools is not none and not tools_in_user_message %}
+    {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+{%- endif %}
+{{- system_message }}
+{{- "<|eot_id|>" }}
+
+{#- Custom tools are passed in a user message with some extra guidance #}
+{%- if tools_in_user_message and not tools is none %}
+    {#- Extract the first user message so we can plug it in here #}
+    {%- if messages | length != 0 %}
+        {%- set first_user_message = messages[0]['content']|trim %}
+        {%- set messages = messages[1:] %}
+    {%- else %}
+        {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
+{%- endif %}
+    {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
+    {{- "Given the following functions, please respond with a JSON for a function call " }}
+    {{- "with its proper arguments that best answers the given prompt.\n\n" }}
+    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
+    {{- "Do not use variables.\n\n" }}
+    {%- for t in tools %}
+        {{- t | tojson(indent=4) }}
+        {{- "\n\n" }}
+    {%- endfor %}
+    {{- first_user_message + "<|eot_id|>"}}
+{%- endif %}
+
+{%- for message in messages %}
+    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
+        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
+    {%- elif 'tool_calls' in message %}
+        {%- if not message.tool_calls|length == 1 %}
+            {{- raise_exception("This model only supports single tool-calls at once!") }}
+        {%- endif %}
+        {%- set tool_call = message.tool_calls[0].function %}
+        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}
+            {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
+            {{- "<|python_tag|>" + tool_call.name + ".call(" }}
+            {%- for arg_name, arg_val in tool_call.arguments | items %}
+                {{- arg_name + '="' + arg_val + '"' }}
+                {%- if not loop.last %}
+                    {{- ", " }}
+                {%- endif %}
+                {%- endfor %}
+            {{- ")" }}
+        {%- else  %}
+            {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
+            {{- '{"name": "' + tool_call.name + '", ' }}
+            {{- '"parameters": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- "}" }}
+        {%- endif %}
+        {%- if builtin_tools is defined %}
+            {#- This means we're in ipython mode #}
+            {{- "<|eom_id|>" }}
+        {%- else %}
+            {{- "<|eot_id|>" }}
+        {%- endif %}
+    {%- elif message.role == "tool" or message.role == "ipython" %}
+        {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
+        {%- if message.content is mapping or message.content is iterable %}
+            {{- message.content | tojson }}
+        {%- else %}
+            {{- message.content }}
+        {%- endif %}
+        {{- "<|eot_id|>" }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
+{%- endif %}
--- a/config.json
+++ b/config.json
@@ -0,0 +1,40 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 14336,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 8,
+  "pad_token_id": null,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "factor": 8.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_theta": 500000.0,
+    "rope_type": "llama3"
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.13.1",
+  "use_cache": true,
+  "vocab_size": 128256
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": [
+    128001,
+    128008,
+    128009
+  ],
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "5.13.1"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0ad61dadd659cde1dc969374858227df3e451e62f408cc385857a7ee405e0f4
+size 16060556616
--- a/tokenizer.json
+++ b/tokenizer.json
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,14 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|begin_of_text|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>"
+}