78 lines
2.8 KiB
Markdown
78 lines
2.8 KiB
Markdown
|
---
|
||
|
|
license: apache-2.0
|
||
|
|
language:
|
||
|
|
- km
|
||
|
|
- en
|
||
|
|
base_model:
|
||
|
|
- Qwen/Qwen3-0.6B
|
||
|
|
library_name: transformers
|
||
|
|
pipeline_tag: text-generation
|
||
|
|
tags:
|
||
|
|
- khmer
|
||
|
|
- qwen3
|
||
|
|
- rag
|
||
|
|
- on-device
|
||
|
|
- small-language-model
|
||
|
|
---
|
||
|
|
|
||
|
|
# Qwen3-0.6B-khm-ft3 — a small Khmer model for on-device RAG
|
||
|
|
|
||
|
|
A **0.6B** Khmer-focused language model fine-tuned from **Qwen3-0.6B**, built to
|
||
|
|
run **on a phone, offline** — it's the answering model behind
|
||
|
|
[iAny](https://github.com/sengtha/iAny) and runs on a 2019 Galaxy S10.
|
||
|
|
|
||
|
|
The key trick: the base Qwen3 vocabulary (~150k tokens) is **trimmed to ~32k
|
||
|
|
Khmer-focused tokens**, which shrinks the output logits buffer enough to fit
|
||
|
|
weak devices while keeping Khmer coverage. It is then continued-pretrained and
|
||
|
|
instruction-tuned on Khmer.
|
||
|
|
|
||
|
|
## Lineage
|
||
|
|
|
||
|
|
| Stage | What | Data |
|
||
|
|
| --- | --- | --- |
|
||
|
|
| Base | Qwen3-0.6B | — |
|
||
|
|
| Vocab trim | ~32k Khmer-focused vocabulary (smaller logits → fits low-RAM phones) | via [alphaedge-ai](https://huggingface.co/alphaedge-ai) |
|
||
|
|
| CPT | Continued pre-training for Khmer fluency | FineWeb-2 (Khmer) + ParaCrawl |
|
||
|
|
| SFT (ft3) | Instruction / Q&A tuning for correct, fuller answers | [`sengtha/khmer-qa`](https://huggingface.co/datasets/sengtha/khmer-qa) |
|
||
|
|
|
||
|
|
`ft3` is the current release — retrained on a richer `khmer-qa` for fuller
|
||
|
|
answers than the earlier `ft`/`ft2` checkpoints.
|
||
|
|
|
||
|
|
## Intended use
|
||
|
|
|
||
|
|
Grounded **Khmer question-answering / RAG** on-device: the app retrieves
|
||
|
|
context from the user's own documents and this model writes the answer in
|
||
|
|
Khmer. Small and Khmer-first — not a general-purpose assistant, and it can be
|
||
|
|
wrong or hallucinate, especially without retrieved context.
|
||
|
|
|
||
|
|
## Usage (transformers)
|
||
|
|
|
||
|
|
```python
|
||
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
|
|
||
|
|
m = "sengtha/Qwen3-0.6B-khm-ft3"
|
||
|
|
tok = AutoTokenizer.from_pretrained(m)
|
||
|
|
model = AutoModelForCausalLM.from_pretrained(m)
|
||
|
|
|
||
|
|
msgs = [{"role": "user", "content": "តើភ្នំពេញជាអ្វី?"}]
|
||
|
|
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
|
||
|
|
out = model.generate(inputs, max_new_tokens=256)
|
||
|
|
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
|
||
|
|
```
|
||
|
|
|
||
|
|
Uses the standard **Qwen3 chat template** (ChatML). Append `/no_think` to a
|
||
|
|
message to skip the reasoning block for faster, direct answers.
|
||
|
|
|
||
|
|
## On-device (GGUF)
|
||
|
|
|
||
|
|
Quantized **GGUF** builds (Q4_K_M + Q8_0) for `llama.cpp` / `llama.rn` live in
|
||
|
|
**[`sengtha/Qwen3-0.6B-khm-ft3-Q8_0-GGUF`](https://huggingface.co/sengtha/Qwen3-0.6B-khm-ft3-Q8_0-GGUF)**.
|
||
|
|
Q4 (~0.4 GB) fits weak phones; Q8 is higher quality.
|
||
|
|
|
||
|
|
## License & attribution
|
||
|
|
|
||
|
|
**Apache-2.0**, inherited from the base **[Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B)**
|
||
|
|
(© Alibaba, Apache-2.0). 32k-vocab base via **alphaedge-ai**. Training corpora:
|
||
|
|
**FineWeb-2**, **ParaCrawl**, and **`sengtha/khmer-qa`**. Please keep this
|
||
|
|
attribution in derivatives.
|