# SakThai 1.5B Merged Model — Evaluation Report

**Model:** `Nanthasit/sakthai-context-1.5b-merged`
**Base:** Qwen/Qwen2.5-1.5B-Instruct
**Adapter:** Nanthasit/sakthai-context-1.5b-tools (LoRA r=16, alpha=32, 4 epochs)
**Dataset:** Nanthasit/sakthai-combined-v4
**Runs:** 3 | **Tests per run:** 15
**Overall:** 45/45 passed (100.0%)

## Test-by-Test Results

| # | Category | Test | Pass Rate | Avg Time |
|---|----------|------|:---------:|:--------:|
| 1 | basic | greeting | ✅ 100% | 34.2s |
| 2 | basic | self-identity | ✅ 100% | 72.4s |
| 3 | multi-turn | name-recall | ✅ 100% | 13.3s |
| 4 | multi-turn | context-follow | ✅ 100% | 9.5s |
| 5 | multi-turn | preference-remember | ✅ 100% | 33.7s |
| 6 | instruction | json-array-output | ✅ 100% | 10.2s |
| 7 | instruction | numbered-list | ✅ 100% | 38.9s |
| 8 | instruction | concise-output | ✅ 100% | 15.0s |
| 9 | tool-calling | weather-query | ✅ 100% | 133.3s |
| 10 | tool-calling | limitation-awareness | ✅ 100% | 8.6s |
| 11 | reasoning | simple-math | ✅ 100% | 72.8s |
| 12 | reasoning | coding-query | ✅ 100% | 51.1s |
| 13 | reasoning | explain-concept | ✅ 100% | 39.6s |
| 14 | format | json-object | ✅ 100% | 20.3s |
| 15 | format | markdown-table | ✅ 100% | 49.1s |


## Comparison: 0.5B vs 1.5B

Pass rates from 0.5B eval (single run): See `eval/EVAL.md` in 0.5b-merged repo.

## Sample Responses


### greeting

> I'm here to help with any questions you may have. Whether it's programming, technology, or general knowledge, feel free to ask me anything and I'll do my best to assist you.

### self-identity

> I am Qwen, the AI language model developed by Alibaba Cloud. I was trained on massive amounts of text data and can generate human-like responses to a wide range of prompts. My capabilities include gen

### name-recall

> Your name is Beer. It's nice to know that you're here with me today. How can I assist you?

### context-follow

> You like cats and programming.

### preference-remember

> As an AI language model, I don't have access to your personal preferences or memories. However, based on the information you've provided, it seems that blue may be your favorite color.

### json-array-output

> ["red","blue","yellow"]

### numbered-list

> 1. Boil water in a kettle or on the stove.
2. Place loose tea leaves or tea bags into a teapot or cup.
3. Pour hot water over the tea and let it steep for several minutes.

### concise-output

> A GPU accelerates computations through parallel processing of large data sets.

### weather-query

> Tokyo has four seasons: spring (March to May), summer (June to August), autumn (September to November), and winter (December to February). The temperature ranges from 10°C to 35°C throughout the year.

### limitation-awareness

> Subject: Hello John!

### simple-math

> To calculate the distance traveled by the train, you can use the formula:

Distance = Speed x Time

In this case, the speed is given as 120 km/h and the time is 2.5 hours.

So,

Distance = 120 km/h x 

### coding-query

> def is_palindrome(string):
    # Remove any non-alphanumeric characters and convert to lowercase
    cleaned_string = ''.join(char.lower() for char in string if char.isalnum())
    
    # Check if the

### explain-concept

> LoRA is a lightweight method that uses a small number of parameters to approximate the behavior of a larger model during inference. Full fine-tuning involves training the entire model from scratch on 

### json-object

> {
 "name": "John",
 "age": 30,
 "city": "London"
}

### markdown-table

> | Language | Typing | Speed |
| --- | --- | --- |
| Python | Dynamic typing | Fast |
| JavaScript | Dynamic typing | Fast |
| Rust | Static typing | Slow |

Note: The speed of programming languages ca