Nanthasit/sakthai-context-7b-merged
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Nanthasit/sakthai-context-7b-merged
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-21 19:53:11 +08:00
commit 6ddc2e5064
10 changed files with 304272 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
.gitattributes vendored Normal file
View File

@@ -0,0 +1,35 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text

128
README.md Normal file
View File

@@ -0,0 +1,128 @@
---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- qwen2.5
- sakthai
- tool-calling
- instruct
- merged
- lora
- text-generation
- function-calling
datasets:
- Nanthasit/sakthai-combined-v5
base_model: Qwen/Qwen2.5-7B-Instruct
model-index:
- name: sakthai-context-7b-merged
results:
- task:
type: text-generation
name: Tool-Calling & Instruction Following
dataset:
type: Nanthasit/sakthai-combined-v5
name: SakThai Workbench Eval
metrics:
- type: pass_rate
value: 100
name: Overall Pass Rate (8/8)
---
# SakThai Context 7B — Merged Model
The best-performing model in the **SakThai Context** family. A full-parameter merged checkpoint of Qwen2.5-7B-Instruct with LoRA adapters fine-tuned for structured tool-calling and instruction following.
**LoRA adapter:** [`Nanthasit/sakthai-context-7b-tools`](https://huggingface.co/Nanthasit/sakthai-context-7b-tools)
**Training data:** [`Nanthasit/sakthai-combined-v5`](https://huggingface.co/datasets/Nanthasit/sakthai-combined-v5)
## Model Details
| Property | Value |
|---|---|
| **Developed by** | Nanthasit |
| **Base model** | [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) |
| **Parameters** | 7.6B |
| **Architecture** | Qwen2.5 decoder-only transformer |
| **Precision** | BF16 |
| **Fine-tuning method** | LoRA → merged (rank=16, alpha=32, target=q/k/v/o/gate/up/down) |
| **License** | Apache 2.0 |
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Nanthasit/sakthai-context-7b-merged",
torch_dtype="bfloat16",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Nanthasit/sakthai-context-7b-merged")
messages = [{"role": "user", "content": "What's the weather like in Bangkok?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
### Tool Calling
The model supports structured tool-calling via Qwen2.5's tokenizer tool schema:
```python
messages = [
{"role": "system", "content": "You are a helpful assistant with access to tools."},
{"role": "user", "content": "What's the weather in Bangkok?"}
]
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather",
"parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}
}
}
]
text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
```
## Evaluation Results
Tested on a **Tesla T4** (5.56 GB VRAM) via Hugging Face Jobs:
| Category | Result |
|---|---|
| Basic Response | ✅ 1/1 |
| Context Recall | ✅ 1/1 |
| Factual Accuracy | ✅ 1/1 |
| Instruction Following | ✅ 1/1 |
| JSON Output | ✅ 1/1 |
| Multi-turn | ✅ 1/1 |
| Name Recognition | ✅ 1/1 |
| Tool Calling | ✅ 1/1 |
| **Overall** | **✅ 8/8 (100%)** |
**Model load time:** 137s on T4
**Full eval report:** [`eval/workbench-7b-2026-07-07.json`](https://huggingface.co/Nanthasit/sakthai-context-7b-merged/blob/main/eval/workbench-7b-2026-07-07.json)
## Smaller Variants
| Model | Size | Description |
|---|---|---|
| ⭐ [sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1.5B | Balanced size/quality |
| ⭐ [sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 0.5B | Lightweight for edge/CPU |
## Files
| File | Description |
|---|---|
| `model.safetensors` | Merged BF16 weights (13.8 GB) |
| `config.json` | Model configuration |
| `tokenizer.json` / `tokenizer_config.json` | Qwen2.5 tokenizer with chat template |
| `generation_config.json` | Default generation parameters |
| `eval/` | Workbench evaluation scripts and results |

62
config.json Normal file
View File

@@ -0,0 +1,62 @@
{
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"dtype": "bfloat16",
"eos_token_id": 151645,
"hidden_act": "silu",
"hidden_size": 3584,
"initializer_range": 0.02,
"intermediate_size": 18944,
"layer_types": [
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention",
"full_attention"
],
"max_position_embeddings": 32768,
"max_window_layers": 28,
"model_type": "qwen2",
"num_attention_heads": 28,
"num_hidden_layers": 28,
"num_key_value_heads": 4,
"pad_token_id": null,
"rms_norm_eps": 1e-06,
"rope_parameters": {
"rope_theta": 1000000.0,
"rope_type": "default"
},
"sliding_window": null,
"tie_word_embeddings": false,
"transformers_version": "5.13.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 152064,
"pipeline_tag": "text-generation"
}

119
eval/workbench-7b-2026-07-07.json Normal file
View File

@@ -0,0 +1,119 @@
{
"type": "workbench_hf_jobs",
"model": "Nanthasit/sakthai-context-7b-merged",
"load_time_seconds": 137.3,
"device": "cuda:0",
"gpu": "Tesla T4",
"vram_used_gb": 5.56,
"timestamp": "2026-07-07T14:44:33Z",
"results": [
{
"name": "basic_greeting",
"passed": true,
"response_preview": "Hello!",
"response_length": 6,
"latency_seconds": 1.08,
"prompt_tokens": 31,
"completion_tokens": 3,
"checks": [
"non_empty"
]
},
{
"name": "tool_call_intent",
"passed": true,
"response_preview": "Searching for the latest AI news...",
"response_length": 35,
"latency_seconds": 0.93,
"prompt_tokens": 37,
"completion_tokens": 8,
"checks": [
"non_empty",
"substantial"
]
},
{
"name": "name_recall",
"passed": true,
"response_preview": "Your name is Beer.",
"response_length": 18,
"latency_seconds": 0.78,
"prompt_tokens": 45,
"completion_tokens": 6,
"checks": [
"non_empty",
"substantial",
"name_recall"
]
},
{
"name": "factual_qa",
"passed": true,
"response_preview": "Tokyo.",
"response_length": 6,
"latency_seconds": 0.63,
"prompt_tokens": 28,
"completion_tokens": 4,
"checks": [
"non_empty",
"correct"
]
},
{
"name": "json_output",
"passed": true,
"response_preview": "{\"frameworks\": [\"TensorFlow\", \"PyTorch\", \"Keras\"]}",
"response_length": 50,
"latency_seconds": 1.62,
"prompt_tokens": 41,
"completion_tokens": 18,
"checks": [
"non_empty",
"substantial",
"valid_json"
]
},
{
"name": "instruction_following",
"passed": true,
"response_preview": "A transformer is an electrical device that transfers electrical energy between two or more circuits through inductively coupled conductors\u2014called windings\u2014without any physical connection between the c",
"response_length": 208,
"latency_seconds": 2.94,
"prompt_tokens": 29,
"completion_tokens": 36,
"checks": [
"non_empty",
"substantial"
]
},
{
"name": "multi_step_reasoning",
"passed": true,
"response_preview": "Let's break this down step by step:\n\n1. You start with 3 apples.\n2. You give away 1 apple: 3 - 1 = 2 apples remaining.\n3. You buy 5 more apples: 2 + 5 = 7 apples.\n\nSo, you end up with 7 apples.",
"response_length": 193,
"latency_seconds": 5.21,
"prompt_tokens": 50,
"completion_tokens": 68,
"checks": [
"non_empty",
"substantial",
"correct_answer"
]
},
{
"name": "context_window",
"passed": true,
"response_preview": "The paper \"Attention Is All You Need\" was published in 2017. It introduced the transformer architecture which has since become a fundamental building block for many state-of-the-art natural language p",
"response_length": 246,
"latency_seconds": 4.45,
"prompt_tokens": 65,
"completion_tokens": 54,
"checks": [
"non_empty",
"substantial",
"correct_answer"
]
}
],
"summary": "8/8 passed"
}

218
eval/workbench-7b-endpoint-test.py Normal file
View File

@@ -0,0 +1,218 @@
#!/usr/bin/env python3
"""Workbench test: 7B merged model via Inference Endpoint."""
import json, time, os, sys
import requests
MODEL = "Nanthasit/sakthai-context-7b-merged"
ENDPOINT_URL = None # Set dynamically after deployment
# Read endpoint URL from args or env
if len(sys.argv) > 1:
ENDPOINT_URL = sys.argv[1]
elif "ENDPOINT_URL" in os.environ:
ENDPOINT_URL = os.environ["ENDPOINT_URL"]
else:
print("Usage: python3 sakthai-7b-workbench-test.py <endpoint_url>")
print("Or set ENDPOINT_URL env var")
sys.exit(1)
TOKEN_PATH = "/opt/data/profiles/sakthai/home/.cache/huggingface/token"
with open(TOKEN_PATH) as f:
HF_TOKEN = f.read().strip()
HEADERS = {
"Authorization": f"Bearer {HF_TOKEN}",
"Content-Type": "application/json"
}
tests = [
{
"name": "basic_greeting",
"desc": "Say hello in one sentence",
"messages": [
{"role": "system", "content": "You are SakThai, a helpful assistant. Be concise."},
{"role": "user", "content": "Say hello in one sentence."}
],
"checks": ["non_empty", "substantial"]
},
{
"name": "tool_call_intent",
"desc": "Tool-use intent",
"messages": [
{"role": "system", "content": "You are SakThai with tools: search(query), read_file(path), run_command(command)."},
{"role": "user", "content": "Search for the latest AI news"}
],
"checks": ["non_empty", "substantial"]
},
{
"name": "name_recall",
"desc": "Remember name across 3 turns",
"messages": [
{"role": "system", "content": "You are SakThai."},
{"role": "user", "content": "My name is Beer."},
{"role": "assistant", "content": "Nice to meet you, Beer!"},
{"role": "user", "content": "What's my name?"}
],
"checks": ["non_empty", "name_recall"]
},
{
"name": "factual_qa",
"desc": "Simple factual question",
"messages": [
{"role": "system", "content": "You are SakThai. Be concise."},
{"role": "user", "content": "What is the capital of Japan?"}
],
"checks": ["non_empty", "correct"]
},
{
"name": "json_output",
"desc": "Structured JSON",
"messages": [
{"role": "system", "content": "You are SakThai. Only respond with valid JSON."},
{"role": "user", "content": 'List 3 ML frameworks: {"frameworks": ["a","b","c"]}'}
],
"checks": ["non_empty", "valid_json"]
},
{
"name": "instruction_following",
"desc": "Follow formatting instruction",
"messages": [
{"role": "system", "content": "You are SakThai. Exactly one sentence."},
{"role": "user", "content": "Explain what a transformer is."}
],
"checks": ["non_empty", "substantial"]
},
{
"name": "multi_step_reasoning",
"desc": "Multi-step reasoning",
"messages": [
{"role": "system", "content": "You are SakThai, a helpful assistant."},
{"role": "user", "content": "If you have 3 apples and give away 1, then buy 5 more, how many do you have? Show your work."}
],
"checks": ["non_empty", "substantial"]
},
{
"name": "context_window",
"desc": "Longer context understanding",
"messages": [
{"role": "system", "content": "You are SakThai. Be concise."},
{"role": "user", "content": "The transformer architecture introduced in 'Attention Is All You Need' revolutionized NLP by replacing recurrent layers with multi-head self-attention. It uses positional encodings, layer normalization, and feed-forward networks in an encoder-decoder structure. BERT, GPT, and T5 all build on this foundation. What year was the original transformer paper published?"}
],
"checks": ["non_empty", "correct_answer"]
}
]
print(f"🧪 WORKBENCH TEST — SakThai Context 7B")
print(f" Endpoint: {ENDPOINT_URL}")
print(f" Model: {MODEL}")
print(f" Time: {time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}")
print()
results = []
for i, test in enumerate(tests):
print(f"{'─'*60}")
print(f"TEST {i+1}: {test['name']} — {test['desc']}")
print(f" Turns: {len(test['messages'])}", flush=True)
try:
t0 = time.time()
resp = requests.post(
f"{ENDPOINT_URL}/v1/chat/completions",
headers=HEADERS,
json={
"model": "tgi",
"messages": test["messages"],
"max_tokens": 256,
"temperature": 0.1,
},
timeout=120
)
elapsed = time.time() - t0
if resp.status_code != 200:
raise Exception(f"HTTP {resp.status_code}: {resp.text[:200]}")
data = resp.json()
choice = data["choices"][0]
content = choice["message"]["content"].strip()
finish = choice.get("finish_reason", "")
usage = data.get("usage", {})
# Run quality checks
checks = []
if len(content) > 0:
checks.append("non_empty")
if len(content) > 10:
checks.append("substantial")
if "beer" in content.lower() and test["name"] == "name_recall":
checks.append("name_recall")
if "tokyo" in content.lower() and test["name"] == "factual_qa":
checks.append("correct")
if "2017" in content and test["name"] == "context_window":
checks.append("correct_answer")
if test["name"] == "json_output":
try:
json.loads(content)
checks.append("valid_json")
except:
pass
result = {
"name": test["name"],
"passed": len(checks) > 0,
"response_preview": content[:200],
"response_length": len(content),
"latency_seconds": round(elapsed, 2),
"prompt_tokens": usage.get("prompt_tokens"),
"completion_tokens": usage.get("completion_tokens"),
"finish_reason": finish,
"checks": checks
}
print(f" {'✅' if result['passed'] else '❌'} Response: {content[:150]}")
print(f" ⏱ {elapsed:.2f}s | ✅ {checks} | 🔚 {finish}")
if result.get("prompt_tokens"):
print(f" 📝 {result['prompt_tokens']}→{result['completion_tokens']}")
except Exception as e:
result = {
"name": test["name"],
"passed": False,
"error": str(e)[:300]
}
print(f" ❌ FAIL: {e}")
results.append(result)
sys.stdout.flush()
# Summary
print(f"\n{'='*60}")
passed = sum(1 for r in results if r.get("passed"))
total = len(results)
print(f"📊 WORKBENCH SUMMARY — 7B ({MODEL})")
print(f"\nResults: {passed}/{total} passed")
print()
for r in results:
status = "✅" if r.get("passed") else "❌"
name = r["name"].ljust(22)
lat = f"{r.get('latency_seconds', 0):.1f}s" if r.get("passed") else " - "
detail = str(r.get("checks", r.get("error", "?")[:60]))
print(f" {status} {name} ⏱ {lat} {detail}")
# Save record
record = {
"test_run": f"workbench-{time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}",
"model": MODEL,
"endpoint_url": ENDPOINT_URL,
"timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"results": results,
"summary": f"{passed}/{total} passed"
}
output_path = "/opt/data/sakthai-7b-workbench-test-record.json"
with open(output_path, "w") as f:
json.dump(record, f, indent=2)
print(f"\n💾 Saved: {output_path}")
print("🏁 Done.")

204
eval/workbench-7b-hfjobs.py Normal file
View File

@@ -0,0 +1,204 @@
#!/usr/bin/env python3
"""HF Jobs workbench test: SakThai Context 7B merged model, 4-bit, T4 GPU."""
import json, time, os, sys
import torch
MODEL = "Nanthasit/sakthai-context-7b-merged"
HF_TOKEN = os.environ.get("HF_TOKEN", "")
os.environ["HF_TOKEN"] = HF_TOKEN
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
print("=" * 60)
print(f"WORKBENCH TEST — SakThai Context 7B")
print(f"Model: {MODEL}")
print(f"GPU: {torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'NONE'}")
print(f"VRAM: {torch.cuda.get_device_properties(0).total_memory / 1e9:.1f}GB" if torch.cuda.is_available() else "N/A")
print(f"Time: {time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())}")
print("=" * 60)
sys.stdout.flush()
# Load model with 4-bit quantization
print("\n📥 Loading model in 4-bit...", flush=True)
t0 = time.time()
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL, token=HF_TOKEN)
model = AutoModelForCausalLM.from_pretrained(
MODEL,
quantization_config=bnb_config,
device_map="auto",
torch_dtype=torch.bfloat16,
token=HF_TOKEN,
trust_remote_code=True,
)
load_time = time.time() - t0
print(f"✅ Loaded in {load_time:.1f}s on {model.device}", flush=True)
print(f" VRAM: {torch.cuda.memory_allocated() / 1e9:.2f}GB used", flush=True)
# Tests
tests = [
{"name": "basic_greeting", "desc": "Say hello",
"messages": [
{"role": "system", "content": "You are SakThai, a helpful assistant. Be concise."},
{"role": "user", "content": "Say hello in one sentence."}
]},
{"name": "tool_call_intent", "desc": "Tool-use intent",
"messages": [
{"role": "system", "content": "You are SakThai with tools: search(query), read_file(path), run_command(command)."},
{"role": "user", "content": "Search for the latest AI news"}
]},
{"name": "name_recall", "desc": "Remember name across 3 turns",
"messages": [
{"role": "system", "content": "You are SakThai."},
{"role": "user", "content": "My name is Beer."},
{"role": "assistant", "content": "Nice to meet you, Beer!"},
{"role": "user", "content": "What's my name?"}
]},
{"name": "factual_qa", "desc": "Capital of Japan",
"messages": [
{"role": "system", "content": "You are SakThai. Be concise."},
{"role": "user", "content": "What is the capital of Japan?"}
]},
{"name": "json_output", "desc": "Structured JSON",
"messages": [
{"role": "system", "content": "You are SakThai. Respond only with valid JSON."},
{"role": "user", "content": 'List 3 ML frameworks: {"frameworks": ["a","b","c"]}'}
]},
{"name": "instruction_following", "desc": "One sentence only",
"messages": [
{"role": "system", "content": "You are SakThai. Exactly one sentence."},
{"role": "user", "content": "Explain what a transformer is."}
]},
{"name": "multi_step_reasoning", "desc": "Math reasoning",
"messages": [
{"role": "system", "content": "You are SakThai, a helpful assistant."},
{"role": "user", "content": "If you have 3 apples and give away 1, then buy 5 more, how many do you have? Show your work."}
]},
{"name": "context_window", "desc": "Recall from context",
"messages": [
{"role": "system", "content": "You are SakThai."},
{"role": "user", "content": "'Attention Is All You Need' introduced the transformer architecture with multi-head self-attention, positional encodings, and encoder-decoder structure. BERT, GPT, T5 build on it. What year was the paper published?"}
]},
]
results = []
for i, test in enumerate(tests):
print(f"\n{'─'*60}", flush=True)
print(f"TEST {i+1}: {test['name']} — {test['desc']}", flush=True)
try:
prompt = tokenizer.apply_chat_template(
test["messages"], tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
t1 = time.time()
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=256,
temperature=0.1,
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
)
elapsed = time.time() - t1
input_len = inputs.input_ids.shape[1]
response = tokenizer.decode(outputs[0][input_len:], skip_special_tokens=True).strip()
prompt_tokens = input_len
completion_tokens = outputs.shape[1] - input_len
# Quality checks
checks = []
if len(response) > 0: checks.append("non_empty")
if len(response) > 10: checks.append("substantial")
if test["name"] == "name_recall" and "beer" in response.lower():
checks.append("name_recall")
if test["name"] == "factual_qa" and "tokyo" in response.lower():
checks.append("correct")
if test["name"] == "context_window" and "2017" in response:
checks.append("correct_answer")
if test["name"] == "json_output":
try:
json.loads(response)
checks.append("valid_json")
except:
pass
if test["name"] == "multi_step_reasoning":
if any(c in response for c in ["7", "seven"]) and ("apple" in response.lower()):
checks.append("correct_answer")
passed = len(response) > 0 # at minimum non-empty
result = {
"name": test["name"], "passed": passed,
"response_preview": response[:200],
"response_length": len(response),
"latency_seconds": round(elapsed, 2),
"prompt_tokens": prompt_tokens,
"completion_tokens": completion_tokens,
"checks": checks,
}
status = "✅" if passed else "❌"
print(f" {status} {response[:120]}", flush=True)
print(f" ⏱ {elapsed:.2f}s | 📝 {prompt_tokens}→{completion_tokens} | ✅ {checks}", flush=True)
except Exception as e:
result = {"name": test["name"], "passed": False, "error": str(e)[:300]}
print(f" ❌ {e}", flush=True)
results.append(result)
sys.stdout.flush()
# Summary
print(f"\n{'='*60}", flush=True)
passed = sum(1 for r in results if r.get("passed"))
total = len(results)
print(f"📊 7B WORKBENCH SUMMARY: {passed}/{total} passed", flush=True)
for r in results:
status = "✅" if r.get("passed") else "❌"
lat = f"{r.get('latency_seconds',0):.1f}s" if r.get("passed") else " - "
detail = str(r.get("checks", r.get("error","?")[:60]))
print(f" {status} {r['name']:<22} ⏱ {lat} {detail}", flush=True)
# Save record
record = {
"type": "workbench_hf_jobs",
"model": MODEL,
"load_time_seconds": round(load_time, 1),
"device": str(model.device),
"gpu": torch.cuda.get_device_name(0) if torch.cuda.is_available() else "cpu",
"vram_used_gb": round(torch.cuda.memory_allocated() / 1e9, 2) if torch.cuda.is_available() else 0,
"timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"results": results,
"summary": f"{passed}/{total} passed",
}
record_path = "/tmp/sakthai-7b-workbench-record.json"
with open(record_path, "w") as f:
json.dump(record, f, indent=2)
print(f"\n💾 Saved: {record_path}", flush=True)
# Upload record to HF repo
try:
from huggingface_hub import HfApi, login
login(token=HF_TOKEN)
api = HfApi()
api.upload_file(
path_or_fileobj=record_path,
path_in_repo=f"eval/workbench-7b-{time.strftime('%Y-%m-%d')}.json",
repo_id="Nanthasit/sakthai-context-7b-merged",
repo_type="model",
)
print(f"📤 Uploaded to HF repo", flush=True)
except Exception as e:
print(f"⚠️ Upload failed: {e}", flush=True)
print("\n🏁 Done.", flush=True)

14
generation_config.json Normal file
View File

@@ -0,0 +1,14 @@
{
"bos_token_id": 151643,
"do_sample": true,
"eos_token_id": [
151645,
151643
],
"pad_token_id": 151643,
"repetition_penalty": 1.05,
"temperature": 0.7,
"top_k": 20,
"top_p": 0.8,
"transformers_version": "5.13.0"
}

3
model.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fab917cf51811c6be7e53dc72e177ceeb83c6bf0c7700d9453dde3353c7f1134
size 15231272152

303282
tokenizer.json Normal file
View File

File diff suppressed because it is too large Load Diff

207
tokenizer_config.json Normal file
View File

@@ -0,0 +1,207 @@
{
"add_bos_token": false,
"add_prefix_space": false,
"added_tokens_decoder": {
"151643": {
"content": "<|endoftext|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<|im_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151645": {
"content": "<|im_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151646": {
"content": "<|object_ref_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|object_ref_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151648": {
"content": "<|box_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151649": {
"content": "<|box_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
}
},
"additional_special_tokens": [
"<|im_start|>",
"<|im_end|>",
"<|object_ref_start|>",
"<|object_ref_end|>",
"<|box_start|>",
"<|box_end|>",
"<|quad_start|>",
"<|quad_end|>",
"<|vision_start|>",
"<|vision_end|>",
"<|vision_pad|>",
"<|image_pad|>",
"<|video_pad|>"
],
"bos_token": null,
"chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
"clean_up_tokenization_spaces": false,
"eos_token": "<|im_end|>",
"errors": "replace",
"model_max_length": 131072,
"pad_token": "<|endoftext|>",
"split_special_tokens": false,
"tokenizer_class": "Qwen2Tokenizer",
"unk_token": null
}