初始化项目，由ModelHub XC社区提供模型
Model: Nanthasit/sakthai-context-7b-merged Source: Original Platform
This commit is contained in:
ModelHub XC
35
.gitattributes
vendored
Normal file
35
.gitattributes
vendored
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
128
README.md
Normal file
128
README.md
Normal file
@@ -0,0 +1,128 @@
|
|||||||
|
---
|
||||||
|
license: apache-2.0
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
tags:
|
||||||
|
- qwen2.5
|
||||||
|
- sakthai
|
||||||
|
- tool-calling
|
||||||
|
- instruct
|
||||||
|
- merged
|
||||||
|
- lora
|
||||||
|
- text-generation
|
||||||
|
- function-calling
|
||||||
|
datasets:
|
||||||
|
- Nanthasit/sakthai-combined-v5
|
||||||
|
base_model: Qwen/Qwen2.5-7B-Instruct
|
||||||
|
model-index:
|
||||||
|
- name: sakthai-context-7b-merged
|
||||||
|
results:
|
||||||
|
- task:
|
||||||
|
type: text-generation
|
||||||
|
name: Tool-Calling & Instruction Following
|
||||||
|
dataset:
|
||||||
|
type: Nanthasit/sakthai-combined-v5
|
||||||
|
name: SakThai Workbench Eval
|
||||||
|
metrics:
|
||||||
|
- type: pass_rate
|
||||||
|
value: 100
|
||||||
|
name: Overall Pass Rate (8/8)
|
||||||
|
---
|
||||||
|
|
||||||
|
# SakThai Context 7B — Merged Model
|
||||||
|
|
||||||
|
The best-performing model in the **SakThai Context** family. A full-parameter merged checkpoint of Qwen2.5-7B-Instruct with LoRA adapters fine-tuned for structured tool-calling and instruction following.
|
||||||
|
|
||||||
|
**LoRA adapter:** [`Nanthasit/sakthai-context-7b-tools`](https://huggingface.co/Nanthasit/sakthai-context-7b-tools)
|
||||||
|
**Training data:** [`Nanthasit/sakthai-combined-v5`](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v5)
|
||||||
|
|
||||||
|
## Model Details
|
||||||
|
|
||||||
|
| Property | Value |
|
||||||
|
|---|---|
|
||||||
|
| **Developed by** | Nanthasit |
|
||||||
|
| **Base model** | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
|
||||||
|
| **Parameters** | 7.6B |
|
||||||
|
| **Architecture** | Qwen2.5 decoder-only transformer |
|
||||||
|
| **Precision** | BF16 |
|
||||||
|
| **Fine-tuning method** | LoRA → merged (rank=16, alpha=32, target=q/k/v/o/gate/up/down) |
|
||||||
|
| **License** | Apache 2.0 |
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
```python
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(
|
||||||
|
"Nanthasit/sakthai-context-7b-merged",
|
||||||
|
torch_dtype="bfloat16",
|
||||||
|
device_map="auto"
|
||||||
|
)
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-7b-merged")
|
||||||
|
|
||||||
|
messages = [{"role": "user", "content": "What's the weather like in Bangkok?"}]
|
||||||
|
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||||
|
inputs = tokenizer(text, return_tensors="pt").to(model.device)
|
||||||
|
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
|
||||||
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tool Calling
|
||||||
|
|
||||||
|
The model supports structured tool-calling via Qwen2.5's tokenizer tool schema:
|
||||||
|
|
||||||
|
```python
|
||||||
|
messages = [
|
||||||
|
{"role": "system", "content": "You are a helpful assistant with access to tools."},
|
||||||
|
{"role": "user", "content": "What's the weather in Bangkok?"}
|
||||||
|
]
|
||||||
|
tools = [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "get_weather",
|
||||||
|
"description": "Get current weather",
|
||||||
|
"parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Evaluation Results
|
||||||
|
|
||||||
|
Tested on a **Tesla T4** (5.56 GB VRAM) via Hugging Face Jobs:
|
||||||
|
|
||||||
|
| Category | Result |
|
||||||
|
|---|---|
|
||||||
|
| Basic Response | ✅ 1/1 |
|
||||||
|
| Context Recall | ✅ 1/1 |
|
||||||
|
| Factual Accuracy | ✅ 1/1 |
|
||||||
|
| Instruction Following | ✅ 1/1 |
|
||||||
|
| JSON Output | ✅ 1/1 |
|
||||||
|
| Multi-turn | ✅ 1/1 |
|
||||||
|
| Name Recognition | ✅ 1/1 |
|
||||||
|
| Tool Calling | ✅ 1/1 |
|
||||||
|
| **Overall** | **✅ 8/8 (100%)** |
|
||||||
|
|
||||||
|
**Model load time:** 137s on T4
|
||||||
|
**Full eval report:** [`eval/workbench-7b-2026-07-07.json`](https://huggingface.co/Nanthasit/sakthai-context-7b-merged/blob/main/eval/workbench-7b-2026-07-07.json)
|
||||||
|
|
||||||
|
## Smaller Variants
|
||||||
|
|
||||||
|
| Model | Size | Description |
|
||||||
|
|---|---|---|
|
||||||
|
| ⭐ [sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1.5B | Balanced size/quality |
|
||||||
|
| ⭐ [sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 0.5B | Lightweight for edge/CPU |
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | Description |
|
||||||
|
|---|---|
|
||||||
|
| `model.safetensors` | Merged BF16 weights (13.8 GB) |
|
||||||
|
| `config.json` | Model configuration |
|
||||||
|
| `tokenizer.json` / `tokenizer_config.json` | Qwen2.5 tokenizer with chat template |
|
||||||
|
| `generation_config.json` | Default generation parameters |
|
||||||
|
| `eval/` | Workbench evaluation scripts and results |
|
||||||
62
config.json
Normal file
62
config.json
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"Qwen2ForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"eos_token_id": 151645,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 3584,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 18944,
|
||||||
|
"layer_types": [
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention",
|
||||||
|
"full_attention"
|
||||||
|
],
|
||||||
|
"max_position_embeddings": 32768,
|
||||||
|
"max_window_layers": 28,
|
||||||
|
"model_type": "qwen2",
|
||||||
|
"num_attention_heads": 28,
|
||||||
|
"num_hidden_layers": 28,
|
||||||
|
"num_key_value_heads": 4,
|
||||||
|
"pad_token_id": null,
|
||||||
|
"rms_norm_eps": 1e-06,
|
||||||
|
"rope_parameters": {
|
||||||
|
"rope_theta": 1000000.0,
|
||||||
|
"rope_type": "default"
|
||||||
|
},
|
||||||
|
"sliding_window": null,
|
||||||
|
"tie_word_embeddings": false,
|
||||||
|
"transformers_version": "5.13.0",
|
||||||
|
"use_cache": true,
|
||||||
|
"use_sliding_window": false,
|
||||||
|
"vocab_size": 152064,
|
||||||
|
"pipeline_tag": "text-generation"
|
||||||
|
}
|
||||||
119
eval/workbench-7b-2026-07-07.json
Normal file
119
eval/workbench-7b-2026-07-07.json
Normal file
@@ -0,0 +1,119 @@
|
|||||||
|
{
|
||||||
|
"type": "workbench_hf_jobs",
|
||||||
|
"model": "Nanthasit/sakthai-context-7b-merged",
|
||||||
|
"load_time_seconds": 137.3,
|
||||||
|
"device": "cuda:0",
|
||||||
|
"gpu": "Tesla T4",
|
||||||
|
"vram_used_gb": 5.56,
|
||||||
|
"timestamp": "2026-07-07T14:44:33Z",
|
||||||
|
"results": [
|
||||||
|
{
|
||||||
|
"name": "basic_greeting",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "Hello!",
|
||||||
|
"response_length": 6,
|
||||||
|
"latency_seconds": 1.08,
|
||||||
|
"prompt_tokens": 31,
|
||||||
|
"completion_tokens": 3,
|
||||||
|
"checks": [
|
||||||
|
"non_empty"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "tool_call_intent",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "Searching for the latest AI news...",
|
||||||
|
"response_length": 35,
|
||||||
|
"latency_seconds": 0.93,
|
||||||
|
"prompt_tokens": 37,
|
||||||
|
"completion_tokens": 8,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"substantial"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "name_recall",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "Your name is Beer.",
|
||||||
|
"response_length": 18,
|
||||||
|
"latency_seconds": 0.78,
|
||||||
|
"prompt_tokens": 45,
|
||||||
|
"completion_tokens": 6,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"substantial",
|
||||||
|
"name_recall"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "factual_qa",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "Tokyo.",
|
||||||
|
"response_length": 6,
|
||||||
|
"latency_seconds": 0.63,
|
||||||
|
"prompt_tokens": 28,
|
||||||
|
"completion_tokens": 4,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"correct"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "json_output",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "{\"frameworks\": [\"TensorFlow\", \"PyTorch\", \"Keras\"]}",
|
||||||
|
"response_length": 50,
|
||||||
|
"latency_seconds": 1.62,
|
||||||
|
"prompt_tokens": 41,
|
||||||
|
"completion_tokens": 18,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"substantial",
|
||||||
|
"valid_json"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "instruction_following",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "A transformer is an electrical device that transfers electrical energy between two or more circuits through inductively coupled conductors\u2014called windings\u2014without any physical connection between the c",
|
||||||
|
"response_length": 208,
|
||||||
|
"latency_seconds": 2.94,
|
||||||
|
"prompt_tokens": 29,
|
||||||
|
"completion_tokens": 36,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"substantial"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "multi_step_reasoning",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "Let's break this down step by step:\n\n1. You start with 3 apples.\n2. You give away 1 apple: 3 - 1 = 2 apples remaining.\n3. You buy 5 more apples: 2 + 5 = 7 apples.\n\nSo, you end up with 7 apples.",
|
||||||
|
"response_length": 193,
|
||||||
|
"latency_seconds": 5.21,
|
||||||
|
"prompt_tokens": 50,
|
||||||
|
"completion_tokens": 68,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"substantial",
|
||||||
|
"correct_answer"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "context_window",
|
||||||
|
"passed": true,
|
||||||
|
"response_preview": "The paper \"Attention Is All You Need\" was published in 2017. It introduced the transformer architecture which has since become a fundamental building block for many state-of-the-art natural language p",
|
||||||
|
"response_length": 246,
|
||||||
|
"latency_seconds": 4.45,
|
||||||
|
"prompt_tokens": 65,
|
||||||
|
"completion_tokens": 54,
|
||||||
|
"checks": [
|
||||||
|
"non_empty",
|
||||||
|
"substantial",
|
||||||
|
"correct_answer"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"summary": "8/8 passed"
|
||||||
|
}
|
||||||
218
eval/workbench-7b-endpoint-test.py
Normal file
218
eval/workbench-7b-endpoint-test.py
Normal file
@@ -0,0 +1,218 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Workbench test: 7B merged model via Inference Endpoint."""
|
||||||
|
import json, time, os, sys
|
||||||
|
import requests
|
||||||
|
|
||||||
|
MODEL = "Nanthasit/sakthai-context-7b-merged"
|
||||||
|
ENDPOINT_URL = None # Set dynamically after deployment
|
||||||
|
|
||||||
|
# Read endpoint URL from args or env
|
||||||
|
if len(sys.argv) > 1:
|
||||||
|
ENDPOINT_URL = sys.argv[1]
|
||||||
|
elif "ENDPOINT_URL" in os.environ:
|
||||||
|
ENDPOINT_URL = os.environ["ENDPOINT_URL"]
|
||||||
|
else:
|
||||||
|
print("Usage: python3 sakthai-7b-workbench-test.py <endpoint_url>")
|
||||||
|
print("Or set ENDPOINT_URL env var")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
TOKEN_PATH = "/opt/data/profiles/sakthai/home/.cache/huggingface/token"
|
||||||
|
with open(TOKEN_PATH) as f:
|
||||||
|
HF_TOKEN = f.read().strip()
|
||||||
|
|
||||||
|
HEADERS = {
|
||||||
|
"Authorization": f"Bearer {HF_TOKEN}",
|
||||||
|
"Content-Type": "application/json"
|
||||||
|
}
|
||||||
|
|
||||||
|
tests = [
|
||||||
|
{
|
||||||
|
"name": "basic_greeting",
|
||||||
|
"desc": "Say hello in one sentence",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai, a helpful assistant. Be concise."},
|
||||||
|
{"role": "user", "content": "Say hello in one sentence."}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "substantial"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "tool_call_intent",
|
||||||
|
"desc": "Tool-use intent",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai with tools: search(query), read_file(path), run_command(command)."},
|
||||||
|
{"role": "user", "content": "Search for the latest AI news"}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "substantial"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "name_recall",
|
||||||
|
"desc": "Remember name across 3 turns",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai."},
|
||||||
|
{"role": "user", "content": "My name is Beer."},
|
||||||
|
{"role": "assistant", "content": "Nice to meet you, Beer!"},
|
||||||
|
{"role": "user", "content": "What's my name?"}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "name_recall"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "factual_qa",
|
||||||
|
"desc": "Simple factual question",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Be concise."},
|
||||||
|
{"role": "user", "content": "What is the capital of Japan?"}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "correct"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "json_output",
|
||||||
|
"desc": "Structured JSON",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Only respond with valid JSON."},
|
||||||
|
{"role": "user", "content": 'List 3 ML frameworks: {"frameworks": ["a","b","c"]}'}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "valid_json"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "instruction_following",
|
||||||
|
"desc": "Follow formatting instruction",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Exactly one sentence."},
|
||||||
|
{"role": "user", "content": "Explain what a transformer is."}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "substantial"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "multi_step_reasoning",
|
||||||
|
"desc": "Multi-step reasoning",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai, a helpful assistant."},
|
||||||
|
{"role": "user", "content": "If you have 3 apples and give away 1, then buy 5 more, how many do you have? Show your work."}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "substantial"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "context_window",
|
||||||
|
"desc": "Longer context understanding",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Be concise."},
|
||||||
|
{"role": "user", "content": "The transformer architecture introduced in 'Attention Is All You Need' revolutionized NLP by replacing recurrent layers with multi-head self-attention. It uses positional encodings, layer normalization, and feed-forward networks in an encoder-decoder structure. BERT, GPT, and T5 all build on this foundation. What year was the original transformer paper published?"}
|
||||||
|
],
|
||||||
|
"checks": ["non_empty", "correct_answer"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
print(f"🧪 WORKBENCH TEST — SakThai Context 7B")
|
||||||
|
print(f" Endpoint: {ENDPOINT_URL}")
|
||||||
|
print(f" Model: {MODEL}")
|
||||||
|
print(f" Time: {time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}")
|
||||||
|
print()
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for i, test in enumerate(tests):
|
||||||
|
print(f"{'─'*60}")
|
||||||
|
print(f"TEST {i+1}: {test['name']} — {test['desc']}")
|
||||||
|
print(f" Turns: {len(test['messages'])}", flush=True)
|
||||||
|
|
||||||
|
try:
|
||||||
|
t0 = time.time()
|
||||||
|
resp = requests.post(
|
||||||
|
f"{ENDPOINT_URL}/v1/chat/completions",
|
||||||
|
headers=HEADERS,
|
||||||
|
json={
|
||||||
|
"model": "tgi",
|
||||||
|
"messages": test["messages"],
|
||||||
|
"max_tokens": 256,
|
||||||
|
"temperature": 0.1,
|
||||||
|
},
|
||||||
|
timeout=120
|
||||||
|
)
|
||||||
|
elapsed = time.time() - t0
|
||||||
|
|
||||||
|
if resp.status_code != 200:
|
||||||
|
raise Exception(f"HTTP {resp.status_code}: {resp.text[:200]}")
|
||||||
|
|
||||||
|
data = resp.json()
|
||||||
|
choice = data["choices"][0]
|
||||||
|
content = choice["message"]["content"].strip()
|
||||||
|
finish = choice.get("finish_reason", "")
|
||||||
|
usage = data.get("usage", {})
|
||||||
|
|
||||||
|
# Run quality checks
|
||||||
|
checks = []
|
||||||
|
if len(content) > 0:
|
||||||
|
checks.append("non_empty")
|
||||||
|
if len(content) > 10:
|
||||||
|
checks.append("substantial")
|
||||||
|
|
||||||
|
if "beer" in content.lower() and test["name"] == "name_recall":
|
||||||
|
checks.append("name_recall")
|
||||||
|
if "tokyo" in content.lower() and test["name"] == "factual_qa":
|
||||||
|
checks.append("correct")
|
||||||
|
if "2017" in content and test["name"] == "context_window":
|
||||||
|
checks.append("correct_answer")
|
||||||
|
if test["name"] == "json_output":
|
||||||
|
try:
|
||||||
|
json.loads(content)
|
||||||
|
checks.append("valid_json")
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
|
result = {
|
||||||
|
"name": test["name"],
|
||||||
|
"passed": len(checks) > 0,
|
||||||
|
"response_preview": content[:200],
|
||||||
|
"response_length": len(content),
|
||||||
|
"latency_seconds": round(elapsed, 2),
|
||||||
|
"prompt_tokens": usage.get("prompt_tokens"),
|
||||||
|
"completion_tokens": usage.get("completion_tokens"),
|
||||||
|
"finish_reason": finish,
|
||||||
|
"checks": checks
|
||||||
|
}
|
||||||
|
|
||||||
|
print(f" {'✅' if result['passed'] else '❌'} Response: {content[:150]}")
|
||||||
|
print(f" ⏱ {elapsed:.2f}s | ✅ {checks} | 🔚 {finish}")
|
||||||
|
if result.get("prompt_tokens"):
|
||||||
|
print(f" 📝 {result['prompt_tokens']}→{result['completion_tokens']}")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
result = {
|
||||||
|
"name": test["name"],
|
||||||
|
"passed": False,
|
||||||
|
"error": str(e)[:300]
|
||||||
|
}
|
||||||
|
print(f" ❌ FAIL: {e}")
|
||||||
|
|
||||||
|
results.append(result)
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
print(f"\n{'='*60}")
|
||||||
|
passed = sum(1 for r in results if r.get("passed"))
|
||||||
|
total = len(results)
|
||||||
|
print(f"📊 WORKBENCH SUMMARY — 7B ({MODEL})")
|
||||||
|
print(f"\nResults: {passed}/{total} passed")
|
||||||
|
print()
|
||||||
|
|
||||||
|
for r in results:
|
||||||
|
status = "✅" if r.get("passed") else "❌"
|
||||||
|
name = r["name"].ljust(22)
|
||||||
|
lat = f"{r.get('latency_seconds', 0):.1f}s" if r.get("passed") else " - "
|
||||||
|
detail = str(r.get("checks", r.get("error", "?")[:60]))
|
||||||
|
print(f" {status} {name} ⏱ {lat} {detail}")
|
||||||
|
|
||||||
|
# Save record
|
||||||
|
record = {
|
||||||
|
"test_run": f"workbench-{time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}",
|
||||||
|
"model": MODEL,
|
||||||
|
"endpoint_url": ENDPOINT_URL,
|
||||||
|
"timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||||
|
"results": results,
|
||||||
|
"summary": f"{passed}/{total} passed"
|
||||||
|
}
|
||||||
|
|
||||||
|
output_path = "/opt/data/sakthai-7b-workbench-test-record.json"
|
||||||
|
with open(output_path, "w") as f:
|
||||||
|
json.dump(record, f, indent=2)
|
||||||
|
print(f"\n💾 Saved: {output_path}")
|
||||||
|
print("🏁 Done.")
|
||||||
204
eval/workbench-7b-hfjobs.py
Normal file
204
eval/workbench-7b-hfjobs.py
Normal file
@@ -0,0 +1,204 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""HF Jobs workbench test: SakThai Context 7B merged model, 4-bit, T4 GPU."""
|
||||||
|
|
||||||
|
import json, time, os, sys
|
||||||
|
import torch
|
||||||
|
|
||||||
|
MODEL = "Nanthasit/sakthai-context-7b-merged"
|
||||||
|
HF_TOKEN = os.environ.get("HF_TOKEN", "")
|
||||||
|
os.environ["HF_TOKEN"] = HF_TOKEN
|
||||||
|
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
|
||||||
|
|
||||||
|
print("=" * 60)
|
||||||
|
print(f"WORKBENCH TEST — SakThai Context 7B")
|
||||||
|
print(f"Model: {MODEL}")
|
||||||
|
print(f"GPU: {torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'NONE'}")
|
||||||
|
print(f"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f}GB" if torch.cuda.is_available() else "N/A")
|
||||||
|
print(f"Time: {time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}")
|
||||||
|
print("=" * 60)
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
# Load model with 4-bit quantization
|
||||||
|
print("\n📥 Loading model in 4-bit...", flush=True)
|
||||||
|
t0 = time.time()
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
|
||||||
|
|
||||||
|
bnb_config = BitsAndBytesConfig(
|
||||||
|
load_in_4bit=True,
|
||||||
|
bnb_4bit_compute_dtype=torch.bfloat16,
|
||||||
|
bnb_4bit_use_double_quant=True,
|
||||||
|
bnb_4bit_quant_type="nf4",
|
||||||
|
)
|
||||||
|
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained(MODEL, token=HF_TOKEN)
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(
|
||||||
|
MODEL,
|
||||||
|
quantization_config=bnb_config,
|
||||||
|
device_map="auto",
|
||||||
|
torch_dtype=torch.bfloat16,
|
||||||
|
token=HF_TOKEN,
|
||||||
|
trust_remote_code=True,
|
||||||
|
)
|
||||||
|
load_time = time.time() - t0
|
||||||
|
print(f"✅ Loaded in {load_time:.1f}s on {model.device}", flush=True)
|
||||||
|
print(f" VRAM: {torch.cuda.memory_allocated() / 1e9:.2f}GB used", flush=True)
|
||||||
|
|
||||||
|
# Tests
|
||||||
|
tests = [
|
||||||
|
{"name": "basic_greeting", "desc": "Say hello",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai, a helpful assistant. Be concise."},
|
||||||
|
{"role": "user", "content": "Say hello in one sentence."}
|
||||||
|
]},
|
||||||
|
{"name": "tool_call_intent", "desc": "Tool-use intent",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai with tools: search(query), read_file(path), run_command(command)."},
|
||||||
|
{"role": "user", "content": "Search for the latest AI news"}
|
||||||
|
]},
|
||||||
|
{"name": "name_recall", "desc": "Remember name across 3 turns",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai."},
|
||||||
|
{"role": "user", "content": "My name is Beer."},
|
||||||
|
{"role": "assistant", "content": "Nice to meet you, Beer!"},
|
||||||
|
{"role": "user", "content": "What's my name?"}
|
||||||
|
]},
|
||||||
|
{"name": "factual_qa", "desc": "Capital of Japan",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Be concise."},
|
||||||
|
{"role": "user", "content": "What is the capital of Japan?"}
|
||||||
|
]},
|
||||||
|
{"name": "json_output", "desc": "Structured JSON",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Respond only with valid JSON."},
|
||||||
|
{"role": "user", "content": 'List 3 ML frameworks: {"frameworks": ["a","b","c"]}'}
|
||||||
|
]},
|
||||||
|
{"name": "instruction_following", "desc": "One sentence only",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai. Exactly one sentence."},
|
||||||
|
{"role": "user", "content": "Explain what a transformer is."}
|
||||||
|
]},
|
||||||
|
{"name": "multi_step_reasoning", "desc": "Math reasoning",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai, a helpful assistant."},
|
||||||
|
{"role": "user", "content": "If you have 3 apples and give away 1, then buy 5 more, how many do you have? Show your work."}
|
||||||
|
]},
|
||||||
|
{"name": "context_window", "desc": "Recall from context",
|
||||||
|
"messages": [
|
||||||
|
{"role": "system", "content": "You are SakThai."},
|
||||||
|
{"role": "user", "content": "'Attention Is All You Need' introduced the transformer architecture with multi-head self-attention, positional encodings, and encoder-decoder structure. BERT, GPT, T5 build on it. What year was the paper published?"}
|
||||||
|
]},
|
||||||
|
]
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for i, test in enumerate(tests):
|
||||||
|
print(f"\n{'─'*60}", flush=True)
|
||||||
|
print(f"TEST {i+1}: {test['name']} — {test['desc']}", flush=True)
|
||||||
|
|
||||||
|
try:
|
||||||
|
prompt = tokenizer.apply_chat_template(
|
||||||
|
test["messages"], tokenize=False, add_generation_prompt=True
|
||||||
|
)
|
||||||
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
||||||
|
|
||||||
|
t1 = time.time()
|
||||||
|
with torch.no_grad():
|
||||||
|
outputs = model.generate(
|
||||||
|
**inputs,
|
||||||
|
max_new_tokens=256,
|
||||||
|
temperature=0.1,
|
||||||
|
do_sample=True,
|
||||||
|
pad_token_id=tokenizer.eos_token_id,
|
||||||
|
)
|
||||||
|
elapsed = time.time() - t1
|
||||||
|
|
||||||
|
input_len = inputs.input_ids.shape[1]
|
||||||
|
response = tokenizer.decode(outputs[0][input_len:], skip_special_tokens=True).strip()
|
||||||
|
prompt_tokens = input_len
|
||||||
|
completion_tokens = outputs.shape[1] - input_len
|
||||||
|
|
||||||
|
# Quality checks
|
||||||
|
checks = []
|
||||||
|
if len(response) > 0: checks.append("non_empty")
|
||||||
|
if len(response) > 10: checks.append("substantial")
|
||||||
|
if test["name"] == "name_recall" and "beer" in response.lower():
|
||||||
|
checks.append("name_recall")
|
||||||
|
if test["name"] == "factual_qa" and "tokyo" in response.lower():
|
||||||
|
checks.append("correct")
|
||||||
|
if test["name"] == "context_window" and "2017" in response:
|
||||||
|
checks.append("correct_answer")
|
||||||
|
if test["name"] == "json_output":
|
||||||
|
try:
|
||||||
|
json.loads(response)
|
||||||
|
checks.append("valid_json")
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
if test["name"] == "multi_step_reasoning":
|
||||||
|
if any(c in response for c in ["7", "seven"]) and ("apple" in response.lower()):
|
||||||
|
checks.append("correct_answer")
|
||||||
|
|
||||||
|
passed = len(response) > 0 # at minimum non-empty
|
||||||
|
result = {
|
||||||
|
"name": test["name"], "passed": passed,
|
||||||
|
"response_preview": response[:200],
|
||||||
|
"response_length": len(response),
|
||||||
|
"latency_seconds": round(elapsed, 2),
|
||||||
|
"prompt_tokens": prompt_tokens,
|
||||||
|
"completion_tokens": completion_tokens,
|
||||||
|
"checks": checks,
|
||||||
|
}
|
||||||
|
status = "✅" if passed else "❌"
|
||||||
|
print(f" {status} {response[:120]}", flush=True)
|
||||||
|
print(f" ⏱ {elapsed:.2f}s | 📝 {prompt_tokens}→{completion_tokens} | ✅ {checks}", flush=True)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
result = {"name": test["name"], "passed": False, "error": str(e)[:300]}
|
||||||
|
print(f" ❌ {e}", flush=True)
|
||||||
|
|
||||||
|
results.append(result)
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
# Summary
|
||||||
|
print(f"\n{'='*60}", flush=True)
|
||||||
|
passed = sum(1 for r in results if r.get("passed"))
|
||||||
|
total = len(results)
|
||||||
|
print(f"📊 7B WORKBENCH SUMMARY: {passed}/{total} passed", flush=True)
|
||||||
|
for r in results:
|
||||||
|
status = "✅" if r.get("passed") else "❌"
|
||||||
|
lat = f"{r.get('latency_seconds',0):.1f}s" if r.get("passed") else " - "
|
||||||
|
detail = str(r.get("checks", r.get("error","?")[:60]))
|
||||||
|
print(f" {status} {r['name']:<22} ⏱ {lat} {detail}", flush=True)
|
||||||
|
|
||||||
|
# Save record
|
||||||
|
record = {
|
||||||
|
"type": "workbench_hf_jobs",
|
||||||
|
"model": MODEL,
|
||||||
|
"load_time_seconds": round(load_time, 1),
|
||||||
|
"device": str(model.device),
|
||||||
|
"gpu": torch.cuda.get_device_name(0) if torch.cuda.is_available() else "cpu",
|
||||||
|
"vram_used_gb": round(torch.cuda.memory_allocated() / 1e9, 2) if torch.cuda.is_available() else 0,
|
||||||
|
"timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||||
|
"results": results,
|
||||||
|
"summary": f"{passed}/{total} passed",
|
||||||
|
}
|
||||||
|
|
||||||
|
record_path = "/tmp/sakthai-7b-workbench-record.json"
|
||||||
|
with open(record_path, "w") as f:
|
||||||
|
json.dump(record, f, indent=2)
|
||||||
|
print(f"\n💾 Saved: {record_path}", flush=True)
|
||||||
|
|
||||||
|
# Upload record to HF repo
|
||||||
|
try:
|
||||||
|
from huggingface_hub import HfApi, login
|
||||||
|
login(token=HF_TOKEN)
|
||||||
|
api = HfApi()
|
||||||
|
api.upload_file(
|
||||||
|
path_or_fileobj=record_path,
|
||||||
|
path_in_repo=f"eval/workbench-7b-{time.strftime('%Y-%m-%d')}.json",
|
||||||
|
repo_id="Nanthasit/sakthai-context-7b-merged",
|
||||||
|
repo_type="model",
|
||||||
|
)
|
||||||
|
print(f"📤 Uploaded to HF repo", flush=True)
|
||||||
|
except Exception as e:
|
||||||
|
print(f"⚠️ Upload failed: {e}", flush=True)
|
||||||
|
|
||||||
|
print("\n🏁 Done.", flush=True)
|
||||||
14
generation_config.json
Normal file
14
generation_config.json
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
{
|
||||||
|
"bos_token_id": 151643,
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
151645,
|
||||||
|
151643
|
||||||
|
],
|
||||||
|
"pad_token_id": 151643,
|
||||||
|
"repetition_penalty": 1.05,
|
||||||
|
"temperature": 0.7,
|
||||||
|
"top_k": 20,
|
||||||
|
"top_p": 0.8,
|
||||||
|
"transformers_version": "5.13.0"
|
||||||
|
}
|
||||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:fab917cf51811c6be7e53dc72e177ceeb83c6bf0c7700d9453dde3353c7f1134
|
||||||
|
size 15231272152
|
||||||
303282
tokenizer.json
Normal file
303282
tokenizer.json
Normal file
File diff suppressed because it is too large
Load Diff
207
tokenizer_config.json
Normal file
207
tokenizer_config.json
Normal file
@@ -0,0 +1,207 @@
|
|||||||
|
{
|
||||||
|
"add_bos_token": false,
|
||||||
|
"add_prefix_space": false,
|
||||||
|
"added_tokens_decoder": {
|
||||||
|
"151643": {
|
||||||
|
"content": "<|endoftext|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151644": {
|
||||||
|
"content": "<|im_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151645": {
|
||||||
|
"content": "<|im_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151646": {
|
||||||
|
"content": "<|object_ref_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151647": {
|
||||||
|
"content": "<|object_ref_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151648": {
|
||||||
|
"content": "<|box_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151649": {
|
||||||
|
"content": "<|box_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151650": {
|
||||||
|
"content": "<|quad_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151651": {
|
||||||
|
"content": "<|quad_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151652": {
|
||||||
|
"content": "<|vision_start|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151653": {
|
||||||
|
"content": "<|vision_end|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151654": {
|
||||||
|
"content": "<|vision_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151655": {
|
||||||
|
"content": "<|image_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151656": {
|
||||||
|
"content": "<|video_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": true
|
||||||
|
},
|
||||||
|
"151657": {
|
||||||
|
"content": "<tool_call>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151658": {
|
||||||
|
"content": "</tool_call>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151659": {
|
||||||
|
"content": "<|fim_prefix|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151660": {
|
||||||
|
"content": "<|fim_middle|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151661": {
|
||||||
|
"content": "<|fim_suffix|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151662": {
|
||||||
|
"content": "<|fim_pad|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151663": {
|
||||||
|
"content": "<|repo_name|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
},
|
||||||
|
"151664": {
|
||||||
|
"content": "<|file_sep|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false,
|
||||||
|
"special": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"additional_special_tokens": [
|
||||||
|
"<|im_start|>",
|
||||||
|
"<|im_end|>",
|
||||||
|
"<|object_ref_start|>",
|
||||||
|
"<|object_ref_end|>",
|
||||||
|
"<|box_start|>",
|
||||||
|
"<|box_end|>",
|
||||||
|
"<|quad_start|>",
|
||||||
|
"<|quad_end|>",
|
||||||
|
"<|vision_start|>",
|
||||||
|
"<|vision_end|>",
|
||||||
|
"<|vision_pad|>",
|
||||||
|
"<|image_pad|>",
|
||||||
|
"<|video_pad|>"
|
||||||
|
],
|
||||||
|
"bos_token": null,
|
||||||
|
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
|
||||||
|
"clean_up_tokenization_spaces": false,
|
||||||
|
"eos_token": "<|im_end|>",
|
||||||
|
"errors": "replace",
|
||||||
|
"model_max_length": 131072,
|
||||||
|
"pad_token": "<|endoftext|>",
|
||||||
|
"split_special_tokens": false,
|
||||||
|
"tokenizer_class": "Qwen2Tokenizer",
|
||||||
|
"unk_token": null
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user