391 lines
15 KiB
Markdown
391 lines
15 KiB
Markdown
---
|
||
license: apache-2.0
|
||
pipeline_tag: text-generation
|
||
library_name: transformers
|
||
language: en
|
||
base_model: Qwen/Qwen2.5-1.5B-Instruct
|
||
datasets:
|
||
- Nanthasit/sakthai-combined-v6
|
||
- Nanthasit/sakthai-combined-v7
|
||
- Nanthasit/sakthai-irrelevance-supplement
|
||
tags:
|
||
- qwen2.5
|
||
- qwen2
|
||
- sakthai
|
||
- house-of-sak
|
||
- tool-calling
|
||
- function-calling
|
||
- agent
|
||
- merged
|
||
- cpu-inference
|
||
- llama.cpp
|
||
- ollama
|
||
- conversational
|
||
- text-generation
|
||
- eval-results
|
||
- benchmark
|
||
- en
|
||
model-index:
|
||
- name: SakThai Context 1.5B Merged V2
|
||
results:
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
name: SakThai Bench v2
|
||
type: sakthai-bench-v2
|
||
metrics:
|
||
- name: Selection Accuracy
|
||
type: selection
|
||
value: 34.9
|
||
verified: true
|
||
date: 2026-07-31
|
||
- name: Arguments Accuracy
|
||
type: arguments
|
||
value: 44.2
|
||
verified: true
|
||
date: 2026-07-31
|
||
- name: Strict Accuracy
|
||
type: strict
|
||
value: 34.2
|
||
verified: true
|
||
date: 2026-07-31
|
||
- task:
|
||
type: text-generation
|
||
dataset:
|
||
name: Internal tool-call smoke test
|
||
type: custom
|
||
metrics:
|
||
- name: multi-tool correctness
|
||
type: pass@3
|
||
value: 100.0
|
||
verified: true
|
||
date: 2026-07-31
|
||
- name: valid JSON rate
|
||
type: ratio
|
||
value: 100.0
|
||
verified: true
|
||
date: 2026-07-31
|
||
---
|
||
|
||
# SakThai Context 1.5B — Merged V2
|
||
|
||
<p align="center">
|
||
<strong>Mid-weight merged full model · Qwen2.5-1.5B · GGUF + safetensors</strong><br/>
|
||
<em>Best when you need more reliability than 0.5B, but still want CPU/edge inference.</em>
|
||
</p>
|
||
|
||
<p align="center">
|
||
<a href="https://huggingface.co/Nanthasit"><img src="https://img.shields.io/badge/%F0%9F%A4%97-Nanthasit-6644cc" alt="Profile"/></a>
|
||
<a href="https://github.com/beer-sakthai"><img src="https://img.shields.io/badge/GitHub-beer--sakthai-181717?logo=github" alt="GitHub"/></a>
|
||
<a href="https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02"><img src="https://img.shields.io/badge/%F0%9F%8F%A0-SakThai%20Family-6644cc" alt="Collection"/></a>
|
||
<img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fhuggingface.co%2Fapi%2Fmodels%2FNanthasit%2Fsakthai-context-1.5b-merged-v2&query=%24.downloads&label=downloads&color=blue&cacheSeconds=3600" alt="Downloads"/>
|
||
<img src="https://img.shields.io/badge/license-Apache%202.0-green" alt="License"/>
|
||
<img src="https://img.shields.io/badge/params-1.5B-blueviolet" alt="Params"/>
|
||
<img src="https://img.shields.io/badge/size-2.9%20GB-orange" alt="Size"/>
|
||
<img src="https://img.shields.io/badge/GGUF-Q4_K_M-orange" alt="GGUF"/>
|
||
<img src="https://img.shields.io/badge/CPU%20inference-~15%20tok%2Fs-yellow" alt="CPU inference"/>
|
||
</p>
|
||
|
||
---
|
||
|
||
## Model Description
|
||
|
||
SakThai Context 1.5B Merged V2 is a **merged full-weight checkpoint** of Qwen2.5-1.5B-Instruct, fine-tuned for structured tool-calling and function-calling. It fills the mid-weight slot between the edge-focused 0.5B and the high-capability 7B. Use it when you need stronger argument filling and multi-turn tool behavior on a laptop/desktop with 4–8 GB RAM.
|
||
|
||
**What makes it special:**
|
||
- 🧠 1.5B parameters — stronger reasoning than 0.5B, smaller than 7B.
|
||
- 🗳️ Trained for structured `<tool>` / `<tool_call>` output.
|
||
- 📦 GGUF Q4_K_M + BF16 safetensors included.
|
||
- ✅ Benchmarks available: Bench v2 selection 34.9%, arguments 44.2%, strict 34.2%; internal smoke test 100% multi-tool correctness / valid JSON.
|
||
- 🔁 Rebuilt merged weights, ready for CPU inference with `llama.cpp` or transformers.
|
||
|
||
---
|
||
|
||
## Requirements
|
||
|
||
Use these tested versions to avoid inference issues on CPU/edge hardware:
|
||
|
||
```text
|
||
torch>=2.2
|
||
transformers>=4.45
|
||
sentencepiece>=0.2
|
||
protobuf>=3.20
|
||
accelerate>=0.27
|
||
```
|
||
|
||
For GGUF inference:
|
||
|
||
```text
|
||
llama-cpp-python>=0.2.80
|
||
```
|
||
|
||
---
|
||
|
||
## Quick Start
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
model_id = "Nanthasit/sakthai-context-1.5b-merged-v2"
|
||
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32, device_map="auto")
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
|
||
messages = [
|
||
{"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
|
||
{"role": "user", "content": "What's the weather in Bangkok?"},
|
||
]
|
||
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||
outputs = model.generate(**inputs, max_new_tokens=256)
|
||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
```
|
||
|
||
### Ollama
|
||
|
||
The model is not published to the Ollama library, so import the local GGUF instead:
|
||
|
||
```bash
|
||
ollama create sakthai:1.5b-v2 -f Modelfile
|
||
```
|
||
|
||
`Modelfile`:
|
||
```
|
||
FROM ./sakthai-1.5b-q4_k_m.gguf
|
||
```
|
||
|
||
### GGUF (llama.cpp)
|
||
|
||
```bash
|
||
# Option A — huggingface-cli
|
||
huggingface-cli download Nanthasit/sakthai-context-1.5b-merged-v2 --include "*.gguf" --local-dir ./
|
||
# Option B — direct wget
|
||
wget https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2/resolve/main/sakthai-1.5b-q4_k_m.gguf
|
||
|
||
llama-cli -m sakthai-1.5b-q4_k_m.gguf \
|
||
--prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 256
|
||
```
|
||
|
||
### Hugging Face Inference Providers (free-tier available)
|
||
|
||
Use the hosted Inference API with `huggingface_hub.InferenceClient`:
|
||
|
||
```python
|
||
from huggingface_hub import InferenceClient
|
||
|
||
client = InferenceClient(model="Nanthasit/sakthai-context-1.5b-merged-v2")
|
||
messages = [
|
||
{"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
|
||
{"role": "user", "content": "What's the weather in Bangkok?"},
|
||
]
|
||
response = client.chat_completion(messages=messages, max_tokens=256)
|
||
print(response.choices[0].message.content)
|
||
```
|
||
|
||
---
|
||
|
||
## Tool-Calling Format
|
||
|
||
The model is fine-tuned for tool calling and expects a `<tools>` XML block in the system prompt — **the block is required**: without it, the model may answer conversationally instead of emitting a tool call.
|
||
|
||
### Verified output format
|
||
|
||
```
|
||
System: <tools>
|
||
[
|
||
{"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}}
|
||
]
|
||
</tools>
|
||
User: What's the weather in Bangkok?
|
||
Assistant: <tool>get_weather</tool>
|
||
```
|
||
|
||
For calls with arguments, the bundled `chat_template.jinja` renders the standard Qwen `<tool_call>` JSON form:
|
||
|
||
```
|
||
<tool_call>
|
||
{"name": "get_weather", "arguments": {"location": "Bangkok"}}
|
||
```
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
| Property | Value |
|
||
|:---------|:------|
|
||
| **Base model** | Qwen/Qwen2.5-1.5B-Instruct |
|
||
| **Parameters** | 1.5B (1,500,000,000) |
|
||
| **Hidden size** | 1,536 |
|
||
| **Layers** | 28 |
|
||
| **Attention heads** | 12 (grouped-query, 2 KV heads) |
|
||
| **Intermediate size** | 8,960 |
|
||
| **Context window** | 32,768 tokens |
|
||
| **Vocab size** | 151,936 |
|
||
| **Precision** | BF16 (safetensors) / GGUF Q4_K_M |
|
||
| **RoPE theta** | 1,000,000 |
|
||
|
||
---
|
||
|
||
## Training Details
|
||
|
||
| Detail | Value |
|
||
|:--------|:------|
|
||
| **Base model** | Qwen/Qwen2.5-1.5B-Instruct |
|
||
| **Method** | SFT → merged to full weights |
|
||
| **Training data** | [sakthai-combined-v6/v7](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v7) + [irrelevance-supplement](https://huggingface.co/datasets/Nanthasit/sakthai-irrelevance-supplement) |
|
||
| **Hardware** | Free T4 GPU (Kaggle / Colab) |
|
||
|
||
---
|
||
|
||
## Evaluation
|
||
|
||
| Setting | Selection | Arguments | Strict | Valid JSON | Multi-tool |
|
||
|:--------|:---------:|:---------:|:------:|:----------:|:----------:|
|
||
| SakThai Bench v2 | 34.9% | 44.2% | 34.2% | — | — |
|
||
| Internal smoke test | — | — | — | 100% | 100% |
|
||
|
||
Notes:
|
||
- Benchmarks are internal and should not be treated as independently verified.
|
||
- `model-index` uses `verified: true` only because the corresponding `.eval_results/` files are present in the repo.
|
||
|
||
---
|
||
|
||
## Deployment
|
||
|
||
### CPU-only
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
import torch
|
||
|
||
model_id = "Nanthasit/sakthai-context-1.5b-merged-v2"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||
model = AutoModelForCausalLM.from_pretrained(
|
||
model_id,
|
||
torch_dtype=torch.float32,
|
||
device_map="cpu",
|
||
low_cpu_mem_usage=True,
|
||
)
|
||
model.eval()
|
||
messages = [
|
||
{"role": "system", "content": "<tools>\n[{\"name\": \"get_weather\", \"description\": \"Get current weather\", \"parameters\": {\"type\": \"object\", \"properties\": {\"location\": {\"type\": \"string\"}}}}]\n</tools>"},
|
||
{"role": "user", "content": "What's the weather in Bangkok?"},
|
||
]
|
||
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
||
with torch.no_grad():
|
||
out = model.generate(**inputs, max_new_tokens=256)
|
||
print(tokenizer.decode(out[0], skip_special_tokens=True))
|
||
```
|
||
|
||
### Edge / llama.cpp
|
||
|
||
```bash
|
||
huggingface-cli download Nanthasit/sakthai-context-1.5b-merged-v2 --include "*.gguf" --local-dir ./
|
||
llama-cli -m sakthai-1.5b-q4_k_m.gguf --prompt "<|system|>You are SakThai-Agent.<|user|>What's the weather in Bangkok?<|assistant|>" -n 256
|
||
```
|
||
|
||
---
|
||
|
||
## 1.5B vs 7B Tradeoffs
|
||
|
||
| Property | 1.5B Merged V2 | 7B Merged |
|
||
|:---------|:-------------:|:---------:|
|
||
| Parameters | 1.5B | 7B |
|
||
| SafeTensors | 2.9 GB | 14.2 GB |
|
||
| GGUF Q4_K_M | ~1.3 GB | ~4.6 GB |
|
||
| RAM needed | ~3 GB | ~8 GB |
|
||
| Tool selection | 34.9% | Higher |
|
||
| Best use | Laptop/desktop, balanced CPU inference | High-power workstation/server |
|
||
|
||
If you need stronger tool argument accuracy and more headroom, use the [7B Merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged). If you need the smallest footprint, use the [0.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged).
|
||
|
||
---
|
||
|
||
## Limitations
|
||
|
||
- **Smaller than 7B** — still has reasoning limits compared to larger checkpoints.
|
||
- **Benchmark numbers are internal** — Bench v2 and smoke-test results are not independently verified.
|
||
- **Requires `<tools>` XML block** — without it, the model defaults to conversation mode.
|
||
- **English-only behavior** — untested in other languages for tool calls.
|
||
- **Argument accuracy is lower than selection** — Arguments Accuracy 44.2% indicates the model often picks the right tool but may misconstruct parameters.
|
||
|
||
---
|
||
|
||
## SakThai Model Family
|
||
|
||
All 26 public models in the family, sorted by downloads (live counts, verified 2026-08-01):
|
||
|
||
| Model | Size | Downloads | Role |
|
||
|:------|:----:|:---------:|:-----|
|
||
| [Context 1.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 3.8 GB | 1,894 | Flagship tool-calling |
|
||
| [Context 0.5B Merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 1.3 GB | 1,730 | Lightweight / edge |
|
||
| [Context 7B Merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 14.2 GB | 1,055 | Full-power reasoning |
|
||
| [Embedding Multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) | 448.8 MB | 651 | Cross-lingual embeddings |
|
||
| [Context 7B 128K](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | — | 643 | 128K YaRN adaptation |
|
||
| [Context 7B Tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | 19.3 MB | 527 | 7B tool-calling adapter |
|
||
| [Context 1.5B Tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | 8.3 MB | 504 | Tool-calling adapter |
|
||
| [Context 1.5B Merged V2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 2.9 GB | 354 | v2 merged |
|
||
| [Vision 7B](https://huggingface.co/Nanthasit/sakthai-vision-7b) | 3.8 GB | 337 | Image-to-text |
|
||
| [Plus 1.5B LoRA](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | 70.5 MB | 306 | rsLoRA adapter |
|
||
| [Context 0.5B Tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 942.3 MB | 474 | Edge tool-calling |
|
||
| [TTS Model](https://huggingface.co/Nanthasit/sakthai-tts-model) | 134.8 MB | 268 | TTS, 15 langs |
|
||
| [Plus 1.5B](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 2.9 GB | 297 | General assistant |
|
||
| [Context 1.5B Tools V2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | 70.5 MB | 192 | v2 tool-calling adapter |
|
||
| [Coder 1.5B](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) | 1.0 GB | 173 | Code generation |
|
||
| [Coder Browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 2.9 GB | 259 | Browser automation |
|
||
| [Coder Browser GGUF](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | 6.6 GB | 153 | Browser GGUF |
|
||
| [Coder Browser LoRA](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | 70.5 MB | 55 | Browser adapter |
|
||
| [Embedding](https://huggingface.co/Nanthasit/sakthai-embedding) | 104.7 MB | 23 | Private embedding |
|
||
| [Plus 1.5B Coder](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-coder) | — | 0 | Coding assistant |
|
||
| [SFT Out](https://huggingface.co/Nanthasit/sft-out) | 4.1 MB | 0 | TRL SFT adapter output |
|
||
| [Context 0.5B Tools SFT](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft) | 8.3 MB | 0 | SFT pilot adapter |
|
||
| [Context 0.5B Tools SFT V2](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2) | 8.3 MB | 0 | SFT v2 adapter |
|
||
| [Bench V2](https://huggingface.co/Nanthasit/sakthai-bench-v2) | — | 0 | Benchmark scaffold |
|
||
| [Pipeline](https://huggingface.co/Nanthasit/sakthai-pipeline) | — | 0 | Automation scripts |
|
||
| [Eval Results](https://huggingface.co/Nanthasit/eval_results) | — | 0 | Companion eval data |
|
||
|
||
*[Full collection](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02)*
|
||
|
||
---
|
||
|
||
## The House of Sak 🏠
|
||
|
||
This model is part of the **House of Sak** — an open-source AI ecosystem built from a shelter in Cork, Ireland, with **$0 budget** and no paid GPUs.
|
||
|
||
> *"We are one family — and becoming more."* — Beer (beer-sakthai)
|
||
|
||
---
|
||
|
||
## Support
|
||
|
||
- ⭐ Leave a like
|
||
- 🐛 Report issues on [GitHub](https://github.com/beer-sakthai/Sak-Family-Agent)
|
||
- 🔄 Share with anyone building accessible AI on CPU/edge
|
||
- 🍴 Fork and experiment — Apache 2.0
|
||
|
||
---
|
||
|
||
## Citation
|
||
|
||
```bibtex
|
||
@misc{sakthai-context-1.5b-merged-v2,
|
||
title = {SakThai Context 1.5B -- Merged V2: Mid-Weight Tool-Calling Model},
|
||
author = {Nanthasit and the House of Sak},
|
||
year = {2026},
|
||
howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2}},
|
||
note = {Apache 2.0, fine-tuned from Qwen/Qwen2.5-1.5B-Instruct}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
Apache 2.0. Qwen2.5 base model per its original license.
|
||
|
||
---
|
||
|
||
*Built from a shelter in Cork, Ireland. Built with love, tears, and zero budget — to the world.*
|
||
|
||
|
||
*Family downloads API-verified (2026-08-01T10:23Z).*
|