初始化项目,由ModelHub XC社区提供模型
Model: AksaraLLM/Kiel-Pro-0.5B-v3 Source: Original Platform
This commit is contained in:
89
README.md
Normal file
89
README.md
Normal file
@@ -0,0 +1,89 @@
|
||||
---
|
||||
language:
|
||||
- id
|
||||
license: apache-2.0
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
base_model: Qwen/Qwen2-0.5B
|
||||
tags:
|
||||
- indonesian
|
||||
- aksarallm
|
||||
- qwen2
|
||||
- continued-pretrain
|
||||
---
|
||||
# Kiel-Pro-0.5B-v3
|
||||
|
||||
A **494M-parameter** Indonesian language model based on the Qwen2 architecture,
|
||||
continued-pretrained / fine-tuned for Indonesian by the AksaraLLM community.
|
||||
This is the **smallest fully-working AksaraLLM model**: it loads cleanly via
|
||||
`AutoModelForCausalLM`, includes its own tokenizer, and produces coherent
|
||||
Indonesian text on standard prompts.
|
||||
|
||||
## Measured baseline (Devin audit, CPU bf16, 50 short Indonesian sentences)
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Perplexity | **14.7** |
|
||||
| English-stopword ratio in ID-prompted output | 0.8% |
|
||||
| Indonesian-stopword ratio in ID-prompted output | 23.2% |
|
||||
| Parameters | 494.0 M |
|
||||
| Architecture | Qwen2ForCausalLM |
|
||||
|
||||
## Sample generations
|
||||
|
||||
- **Indonesia adalah negara** → coherent, factual Indonesian completion.
|
||||
- **Resep nasi goreng yang enak adalah** → coherent recipe-style Indonesian.
|
||||
|
||||
## Quickstart
|
||||
|
||||
```python
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
import torch
|
||||
|
||||
tok = AutoTokenizer.from_pretrained("AksaraLLM/Kiel-Pro-0.5B-v3")
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
"AksaraLLM/Kiel-Pro-0.5B-v3",
|
||||
torch_dtype=torch.bfloat16,
|
||||
device_map="auto",
|
||||
)
|
||||
inp = tok("Indonesia adalah negara", return_tensors="pt").to(model.device)
|
||||
print(tok.decode(model.generate(**inp, max_new_tokens=100, do_sample=True, top_p=0.9)[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- **No chat template** in the tokenizer config — treat this as a base LM, not an instruction-tuned model.
|
||||
- **Identity is uncalibrated** — the model is descended from Qwen2 and may identify itself as "Qwen" if asked. A short identity SFT pass would fix this.
|
||||
- **Hallucinations** typical of a 0.5B-parameter model: factual claims should always be verified.
|
||||
- **No production guardrails / RLHF**.
|
||||
|
||||
## Recommended downstream work
|
||||
|
||||
1. Identity SFT (50–100 prompts) to teach the model to say "AksaraLLM Kiel-Pro".
|
||||
2. Add a `chat_template` (Qwen2 ChatML) to `tokenizer_config.json`.
|
||||
3. Run IndoNLU / IndoMMLU benchmarks for a publishable score.
|
||||
|
||||
## License
|
||||
Apache 2.0
|
||||
|
||||
## Sample outputs (CPU inference, greedy, repeat_penalty=1.15)
|
||||
|
||||
Real generation captured during the Devin audit on 2026-05-02. Greedy decoding (`do_sample=False`) so these are reproducible.
|
||||
|
||||
### Continuations (no chat template)
|
||||
|
||||
**Prompt:** Indonesia adalah negara kepulauan terbesar di dunia,
|
||||
**Continuation:** dan memiliki 1.347 pulau kecil yang disebut indah rindu (dari bahasa Jawa: rumput). Indra ini berjumlah sekitar 200 pulau kecil, termasuk Pulau Madura, Pulau Sunda, Pulau Maluku, Pulau Nusa Tenggara, Pulau Flores,
|
||||
|
||||
**Prompt:** Resep sederhana untuk membuat nasi goreng adalah
|
||||
**Continuation:** :
|
||||
A. 200 gram nasi, 150 ml air, 3 lembar daun jeruk nipis, dan 4 bawang putih.
|
||||
B. 200 gram nasi, 150 ml air, 3 lembar daun jeruk nipis, dan 6 bawang putih.
|
||||
C. 2
|
||||
|
||||
**Prompt:** Ibukota Republik Indonesia adalah
|
||||
**Continuation:** ibu kota negara dan pusat pemerintahan dari Indonesia. Ibukotanya terletak di Sumatera Barat, Indonesia. Penduduknya berjumlah 2.354.069 jiwa (2010). Ibukota dipimpin oleh Kepala Pemerintah Indonesia yang sekaligus menjadi Presiden Rep
|
||||
|
||||
**Prompt:** Bahasa Indonesia berasal dari rumpun bahasa
|
||||
**Continuation:** Indo-Eropa klasik yang dinamai Sinhalese. Dari sisi etimologi, kata "sinah" berarti "pohon", sedangkan "lese" berarti "menggambar". Kata ini muncul dalam bahasa Sinhala dan merupakan salah satu nama untuk pohon-pohon di daerah itu. Pohon tersebut adalah pohon le
|
||||
|
||||
Reference in New Issue
Block a user