sakthai-context-0.5b-tools/README.md

---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-0.5B-Instruct
tags:
- agent
- conversational
- ollama
- transformers
- small-language-model
- slm
- tool-use
- qwen
- qwen2.5
- sakthai
- house-of-sak
- tool-calling
- function-calling
- merged
- edge
- lightweight
- low-resource
- raspberry-pi
- on-device
- benchmark
- eval
datasets:
- Nanthasit/sakthai-combined-v7
- Nanthasit/sakthai-bench-v2
model-index:
- name: sakthai-context-0.5b-tools
  results:
  - task:
      type: text-generation
      name: Tool-Calling
    dataset:
      name: SakThai Bench v2 (500 rows, scorer multiset-selection-v2)
      type: Nanthasit/sakthai-bench-v2
    metrics:
    - type: selection
      value: 91.2
      name: Selection Accuracy
      verified: true
      evidence: .eval_results/sakthai-bench-v2.yaml
    - type: arguments
      value: 45.7
      name: Arguments Accuracy
      verified: true
      evidence: .eval_results/sakthai-bench-v2.yaml
    - type: strict
      value: 45.7
      name: Strict Accuracy
      verified: true
      evidence: .eval_results/sakthai-bench-v2.yaml
    - type: held-out
      value: 87.8
      name: Held-Out Tool Accuracy
      verified: true
      evidence: .eval_results/sakthai-bench-v2.yaml
    - type: degenerate
      value: 0
      name: Degenerate Outputs
      verified: true
      evidence: .eval_results/sakthai-bench-v2.yaml
inference:
  parameters:
    temperature: 0.01
    max_new_tokens: 256
    top_p: 0.9
widget:
- text: What is the weather in Tokyo?
  example_title: Tool-calling
- text: Who wrote Romeo and Juliet?
  example_title: Direct answer
- text: Search the web for latest AI news
  example_title: Search tool
---

<p align="center">
  <img src="https://img.shields.io/badge/dynamic/json?url=https%3A//huggingface.co/api/models/Nanthasit/sakthai-context-0.5b-tools&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
  <img src="https://img.shields.io/badge/size-~1.0GB-blue" alt="Size"/>
  <img src="https://img.shields.io/badge/pipeline-tool--calling-orange" alt="Pipeline"/>
  <img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
  <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
  <a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9A%80-Explore%20Family-47d147" alt="Family"/></a>
</p>

<h1 align="center">SakThai Context 0.5B Tools</h1>
<p align="center"><em>Ultra-light tool-calling agent · Qwen2.5-0.5B fine-tune · runs in ~1 GB RAM</em></p>

**SakThai Context 0.5B Tools** is a prompt-masked SFT of [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) optimized for browser/tool calling. It achieves **91.2% selection accuracy** on SakThai Bench v2, with **0% degenerate outputs** in multi-trial evaluation.

## Model Description

**SakThai Context 0.5B Tools** is a prompt-masked supervised fine-tune of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM.

Key points:
- Base: `Qwen/Qwen2.5-0.5B-Instruct`
- Training: prompt-masked SFT on tool-calling traces from `Nanthasit/sakthai-combined-v7`
- Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments
- License: Apache-2.0

## Quick Start — Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Nanthasit/sakthai-context-0.5b-tools"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {
                "type": "object",
                "properties": {"location": {"type": "string"}},
                "required": ["location"],
            },
        }
    }
]

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What's the weather in Tokyo?"},
]

text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
```

## Quick Start — llama.cpp / Ollama

```bash
# Convert with llama.cpp and run locally
llama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_M
ollama create sakthai-context-0.5b-tools -f Modelfile
ollama run sakthai-context-0.5b-tools
```

### Usage notes
- For tool calling, always use `apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True)` so the model receives the proper `<tools>` block.
- If you want stricter outputs, reduce `temperature` further, e.g. `0.0`.
- For CPU-only inference, set `device_map="cpu"`; GPU/MPS/CPU auto-detection works with `device_map="auto"`.

## Architecture & Config

| Field | Value |
|------:|-------|
| Architecture | `Qwen2ForCausalLM` |
| Model type | `qwen2` |
| Vocab size | `151936` |
| Hidden size | `896` |
| Layers | `24` |
| Attention heads | `14` |
| KV heads | `2` |
| Intermediate size | `4864` |
| Activation | `silu` |
| Max position | `32768` |
| Transformers | `5.14.1` |

## Benchmarks

| Metric | Value | Verified |
|------:|------:|:--------|
| Selection Accuracy | 91.2% | true |
| Arguments Accuracy | 45.7% | true |
| Strict Accuracy | 45.7% | true |
| Held-Out Tool Accuracy | 87.8% | true |
| Degenerate Outputs | 0% | true |

Evidence: `.eval_results/sakthai-bench-v2.yaml` in repo.

## Limitations

- 0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy.
- Tool schema adherence degrades on nested arguments and long context traces.
- Current weights are merged; if you need the unmerged adapter, use `Nanthasit/sakthai-context-0.5b-tools-sft` or `Nanthasit/sakthai-context-0.5b-tools-sft-v2`.

## Citation

If you use this model, please cite the SakThai model family and benchmark:

```bibtex
@misc{sakthai2025context05btools,
  title  = {SakThai Context 0.5B Tools},
  author = {Nanthasit},
  year   = {2026},
  url    = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools}
}
```

## SakThai Family

| Repo | Downloads | Size | Pipeline |
|-----:|----------:|-----:|---------|
| [Nanthasit/sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1855 | ~4.07 GB | text-generation |
| [Nanthasit/sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 1692 | ~1.39 GB | text-generation |
| [Nanthasit/sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 1024 | ~15.23 GB | text-generation |
| [Nanthasit/sakthai-embedding-multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) | 627 | ~471 MB | sentence-similarity |
| [Nanthasit/sakthai-context-7b-128k](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | 610 | — | text-generation |
| [Nanthasit/sakthai-context-7b-tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | 489 | ~20 MB | text-generation |
| [Nanthasit/sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | 477 | ~8.7 MB | text-generation |
| [Nanthasit/sakthai-context-1.5b-merged-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 337 | ~3.09 GB | text-generation |
| [Nanthasit/sakthai-vision-7b](https://huggingface.co/Nanthasit/sakthai-vision-7b) | 315 | ~4.71 GB | image-text-to-text |
| [Nanthasit/sakthai-plus-1.5b-lora](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | 306 | ~74 MB | text-generation |
| [Nanthasit/sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 251 | ~1.0 GB | text-generation |
| [Nanthasit/sakthai-tts-model](https://huggingface.co/Nanthasit/sakthai-tts-model) | 248 | ~143 MB | text-to-speech |
| [Nanthasit/sakthai-plus-1.5b](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 244 | ~3.09 GB | text-generation |
| [Nanthasit/sakthai-context-1.5b-tools-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | 173 | ~74 MB | text-generation |
| [Nanthasit/sakthai-coder-1.5b](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) | 151 | ~1.12 GB | text-generation |
| [Nanthasit/sakthai-coder-browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 54 | ~3.09 GB | text-generation |
| [Nanthasit/sakthai-coder-browser-gguf](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | 35 | ~7.11 GB | text-generation |
| [Nanthasit/sakthai-embedding](https://huggingface.co/Nanthasit/sakthai-embedding) | 23 | ~110 MB | sentence-similarity |
| [Nanthasit/sakthai-coder-browser-lora](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | 21 | ~74 MB | text-generation |

Download counts and sizes were verified from the Hub API at upload time.