---
license: apache-2.0
language:
  - km
  - en
base_model:
  - Qwen/Qwen3-0.6B
library_name: transformers
pipeline_tag: text-generation
tags:
  - khmer
  - qwen3
  - rag
  - on-device
  - small-language-model
---

# Qwen3-0.6B-khm-ft3 — a small Khmer model for on-device RAG

A **0.6B** Khmer-focused language model fine-tuned from **Qwen3-0.6B**, built to
run **on a phone, offline** — it's the answering model behind
[iAny](https://github.com/sengtha/iAny) and runs on a 2019 Galaxy S10.

The key trick: the base Qwen3 vocabulary (~150k tokens) is **trimmed to ~32k
Khmer-focused tokens**, which shrinks the output logits buffer enough to fit
weak devices while keeping Khmer coverage. It is then continued-pretrained and
instruction-tuned on Khmer.

## Lineage

| Stage | What | Data |
| --- | --- | --- |
| Base | Qwen3-0.6B | — |
| Vocab trim | ~32k Khmer-focused vocabulary (smaller logits → fits low-RAM phones) | via [alphaedge-ai](https://huggingface.co/alphaedge-ai) |
| CPT | Continued pre-training for Khmer fluency | FineWeb-2 (Khmer) + ParaCrawl |
| SFT (ft3) | Instruction / Q&A tuning for correct, fuller answers | [`sengtha/khmer-qa`](https://huggingface.co/datasets/sengtha/khmer-qa) |

`ft3` is the current release — retrained on a richer `khmer-qa` for fuller
answers than the earlier `ft`/`ft2` checkpoints.

## Intended use

Grounded **Khmer question-answering / RAG** on-device: the app retrieves
context from the user's own documents and this model writes the answer in
Khmer. Small and Khmer-first — not a general-purpose assistant, and it can be
wrong or hallucinate, especially without retrieved context.

## Usage (transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

m = "sengtha/Qwen3-0.6B-khm-ft3"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m)

msgs = [{"role": "user", "content": "តើភ្នំពេញជាអ្វី?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=256)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
```

Uses the standard **Qwen3 chat template** (ChatML). Append `/no_think` to a
message to skip the reasoning block for faster, direct answers.

## On-device (GGUF)

Quantized **GGUF** builds (Q4_K_M + Q8_0) for `llama.cpp` / `llama.rn` live in
**[`sengtha/Qwen3-0.6B-khm-ft3-Q8_0-GGUF`](https://huggingface.co/sengtha/Qwen3-0.6B-khm-ft3-Q8_0-GGUF)**.
Q4 (~0.4 GB) fits weak phones; Q8 is higher quality.

## License & attribution

**Apache-2.0**, inherited from the base **[Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B)**
(© Alibaba, Apache-2.0). 32k-vocab base via **alphaedge-ai**. Training corpora:
**FineWeb-2**, **ParaCrawl**, and **`sengtha/khmer-qa`**. Please keep this
attribution in derivatives.