license, language, library_name, pipeline_tag, base_model, tags, datasets, model-index, inference, widget
| license | language | library_name | pipeline_tag | base_model | tags | datasets | model-index | inference | widget | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 |
|
transformers | text-generation | Qwen/Qwen2.5-0.5B-Instruct |
|
|
|
|
|
SakThai Context 0.5B Tools
Ultra-light tool-calling agent · Qwen2.5-0.5B fine-tune · runs in ~1 GB RAM
SakThai Context 0.5B Tools is a prompt-masked SFT of Qwen2.5-0.5B-Instruct optimized for browser/tool calling. It achieves 91.2% selection accuracy on SakThai Bench v2, with 0% degenerate outputs in multi-trial evaluation.
Model Description
SakThai Context 0.5B Tools is a prompt-masked supervised fine-tune of Qwen/Qwen2.5-0.5B-Instruct focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM.
Key points:
- Base:
Qwen/Qwen2.5-0.5B-Instruct - Training: prompt-masked SFT on tool-calling traces from
Nanthasit/sakthai-combined-v7 - Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments
- License: Apache-2.0
Quick Start — Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Nanthasit/sakthai-context-0.5b-tools"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {
"type": "object",
"properties": {"location": {"type": "string"}},
"required": ["location"],
},
}
}
]
messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What's the weather in Tokyo?"},
]
text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
Quick Start — llama.cpp / Ollama
# Convert with llama.cpp and run locally
llama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_M
ollama create sakthai-context-0.5b-tools -f Modelfile
ollama run sakthai-context-0.5b-tools
Usage notes
- For tool calling, always use
apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True)so the model receives the proper<tools>block. - If you want stricter outputs, reduce
temperaturefurther, e.g.0.0. - For CPU-only inference, set
device_map="cpu"; GPU/MPS/CPU auto-detection works withdevice_map="auto".
Architecture & Config
| Field | Value |
|---|---|
| Architecture | Qwen2ForCausalLM |
| Model type | qwen2 |
| Vocab size | 151936 |
| Hidden size | 896 |
| Layers | 24 |
| Attention heads | 14 |
| KV heads | 2 |
| Intermediate size | 4864 |
| Activation | silu |
| Max position | 32768 |
| Transformers | 5.14.1 |
Benchmarks
| Metric | Value | Verified |
|---|---|---|
| Selection Accuracy | 91.2% | true |
| Arguments Accuracy | 45.7% | true |
| Strict Accuracy | 45.7% | true |
| Held-Out Tool Accuracy | 87.8% | true |
| Degenerate Outputs | 0% | true |
Evidence: .eval_results/sakthai-bench-v2.yaml in repo.
Limitations
- 0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy.
- Tool schema adherence degrades on nested arguments and long context traces.
- Current weights are merged; if you need the unmerged adapter, use
Nanthasit/sakthai-context-0.5b-tools-sftorNanthasit/sakthai-context-0.5b-tools-sft-v2.
Citation
If you use this model, please cite the SakThai model family and benchmark:
@misc{sakthai2025context05btools,
title = {SakThai Context 0.5B Tools},
author = {Nanthasit},
year = {2026},
url = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools}
}
SakThai Family
Download counts and sizes were verified from the Hub API at upload time.