初始化项目，由ModelHub XC社区提供模型
Model: OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k Source: Original Platform
This commit is contained in:
ModelHub XC
37
.gitattributes
vendored
Normal file
37
.gitattributes
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
assets/mmlu_5k.png filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
138
README.md
Normal file
138
README.md
Normal file
@@ -0,0 +1,138 @@
|
||||
---
|
||||
license: other
|
||||
license_name: llama3.2
|
||||
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
|
||||
base_model: unsloth/Llama-3.2-1B
|
||||
tags:
|
||||
- math
|
||||
- fine-tuned
|
||||
- lora
|
||||
- unsloth
|
||||
- llama
|
||||
datasets:
|
||||
- MathLLMs/MathCodeInstruct
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
model-index:
|
||||
- name: Llama-3.2-1B-MathCodeInstruct-{{SIZE}}
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: GSM8K
|
||||
dataset:
|
||||
type: gsm8k
|
||||
name: GSM8K
|
||||
metrics:
|
||||
- type: exact_match
|
||||
value: {{GSM8K_ACC}}
|
||||
name: exact match (flexible-extract, 5-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: ARC-Challenge
|
||||
dataset:
|
||||
type: ai2_arc
|
||||
name: ARC-Challenge
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: {{ARC_ACC}}
|
||||
name: acc_norm (25-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: HellaSwag
|
||||
dataset:
|
||||
type: hellaswag
|
||||
name: HellaSwag
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: {{HELLASWAG_ACC}}
|
||||
name: acc_norm (10-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: WinoGrande
|
||||
dataset:
|
||||
type: winogrande
|
||||
name: WinoGrande
|
||||
metrics:
|
||||
- type: acc
|
||||
value: {{WINOGRANDE_ACC}}
|
||||
name: acc (5-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: MMLU
|
||||
dataset:
|
||||
type: mmlu
|
||||
name: MMLU
|
||||
metrics:
|
||||
- type: acc
|
||||
value: {{MMLU_ACC}}
|
||||
name: acc (5-shot)
|
||||
---
|
||||
|
||||
# Llama-3.2-1B-MathCodeInstruct-5k
|
||||
|
||||
A [Llama-3.2-1B](https://huggingface.co/unsloth/Llama-3.2-1B) fine-tune on **5k examples** from
|
||||
[MathLLMs/MathCodeInstruct](https://huggingface.co/datasets/MathLLMs/MathCodeInstruct), trained to solve math
|
||||
word problems with step-by-step natural-language reasoning interleaved with executable Python.
|
||||
|
||||
This is one of three sibling models trained on {5k, 10k, 20k}-example subsets of the same dataset, to study how
|
||||
fine-tuning data volume trades off against both math performance and general capability. See the
|
||||
[training write-up](https://github.com/OliverSundaram/finetuning-Llama3.2-1B) for the full comparison across all three.
|
||||
|
||||
## Training details
|
||||
|
||||
| | |
|
||||
|---|-----------------------------------------------------------------------------------------|
|
||||
| Base model | `unsloth/Llama-3.2-1B` |
|
||||
| Method | LoRA (r=16, α=16, dropout=0) on all attention + MLP projections, merged to full weights |
|
||||
| Dataset | MathLLMs/MathCodeInstruct, 5k training examples |
|
||||
| Epochs | 1 |
|
||||
| Effective batch size | 16 (batch 1 × grad. accum. 16) |
|
||||
| Learning rate | 2e-4, cosine schedule, warmup ratio 0.03 |
|
||||
| Hardware | 1× RTX 4060 (8GB) |
|
||||
| Framework | Unsloth + TRL `SFTTrainer` |
|
||||
|
||||
## Benchmark results
|
||||
|
||||
All benchmarks run with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness), each at
|
||||
its standard published shot count, compared against the un-tuned base model.
|
||||
|
||||
| Benchmark | Llama-3.2-1B (base) | MathCodeInstruct-5k | Change |
|
||||
|---|---|---|---|
|
||||
| GSM8K | 5.8% | 7.4% | 🟢 +1.5% |
|
||||
| ARC-Challenge | 36.9% | 36.8% | ⚪ -0.1% |
|
||||
| HellaSwag | 64.2% | 63.8% | 🔴 -0.3% |
|
||||
| WinoGrande | 60.8% | 62.4% | 🟢 +1.7% |
|
||||
|
||||
**Speed:** **40.67 tokens/sec**
|
||||
(base model: 40.59 tokens/sec)
|
||||
|
||||
### MMLU by category
|
||||
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "Below is a math problem. Please solve it step by step."},
|
||||
{"role": "user", "content": "If a train travels 60 miles in 45 minutes, what is its speed in miles per hour?"},
|
||||
]
|
||||
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
|
||||
output = model.generate(inputs, max_new_tokens=512, do_sample=False)
|
||||
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Trained on a single epoch of a 5k-example subset — not intended to be a general-purpose assistant.
|
||||
- MMLU/ARC/HellaSwag/WinoGrande scores reflect a small 1B-parameter base model and should be read relative to
|
||||
the base model's own scores, not against much larger models.
|
||||
- No safety alignment or RLHF was applied beyond what the base Llama-3.2-1B already has.
|
||||
3
assets/mmlu_5k.png
Normal file
3
assets/mmlu_5k.png
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:03893630d52428741a52a22d98c5c1e4ce84337b506aff0e3f3760b813b72158
|
||||
size 441935
|
||||
138
assets/model_card.md
Normal file
138
assets/model_card.md
Normal file
@@ -0,0 +1,138 @@
|
||||
---
|
||||
license: other
|
||||
license_name: llama3.2
|
||||
license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE
|
||||
base_model: unsloth/Llama-3.2-1B
|
||||
tags:
|
||||
- math
|
||||
- fine-tuned
|
||||
- lora
|
||||
- unsloth
|
||||
- llama
|
||||
datasets:
|
||||
- MathLLMs/MathCodeInstruct
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
model-index:
|
||||
- name: Llama-3.2-1B-MathCodeInstruct-{{SIZE}}
|
||||
results:
|
||||
- task:
|
||||
type: text-generation
|
||||
name: GSM8K
|
||||
dataset:
|
||||
type: gsm8k
|
||||
name: GSM8K
|
||||
metrics:
|
||||
- type: exact_match
|
||||
value: {{GSM8K_ACC}}
|
||||
name: exact match (flexible-extract, 5-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: ARC-Challenge
|
||||
dataset:
|
||||
type: ai2_arc
|
||||
name: ARC-Challenge
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: {{ARC_ACC}}
|
||||
name: acc_norm (25-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: HellaSwag
|
||||
dataset:
|
||||
type: hellaswag
|
||||
name: HellaSwag
|
||||
metrics:
|
||||
- type: acc_norm
|
||||
value: {{HELLASWAG_ACC}}
|
||||
name: acc_norm (10-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: WinoGrande
|
||||
dataset:
|
||||
type: winogrande
|
||||
name: WinoGrande
|
||||
metrics:
|
||||
- type: acc
|
||||
value: {{WINOGRANDE_ACC}}
|
||||
name: acc (5-shot)
|
||||
- task:
|
||||
type: text-generation
|
||||
name: MMLU
|
||||
dataset:
|
||||
type: mmlu
|
||||
name: MMLU
|
||||
metrics:
|
||||
- type: acc
|
||||
value: {{MMLU_ACC}}
|
||||
name: acc (5-shot)
|
||||
---
|
||||
|
||||
# Llama-3.2-1B-MathCodeInstruct-5k
|
||||
|
||||
A [Llama-3.2-1B](https://huggingface.co/unsloth/Llama-3.2-1B) fine-tune on **5k examples** from
|
||||
[MathLLMs/MathCodeInstruct](https://huggingface.co/datasets/MathLLMs/MathCodeInstruct), trained to solve math
|
||||
word problems with step-by-step natural-language reasoning interleaved with executable Python.
|
||||
|
||||
This is one of three sibling models trained on {5k, 10k, 20k}-example subsets of the same dataset, to study how
|
||||
fine-tuning data volume trades off against both math performance and general capability. See the
|
||||
[training write-up](https://github.com/OliverSundaram/finetuning-Llama3.2-1B) for the full comparison across all three.
|
||||
|
||||
## Training details
|
||||
|
||||
| | |
|
||||
|---|-----------------------------------------------------------------------------------------|
|
||||
| Base model | `unsloth/Llama-3.2-1B` |
|
||||
| Method | LoRA (r=16, α=16, dropout=0) on all attention + MLP projections, merged to full weights |
|
||||
| Dataset | MathLLMs/MathCodeInstruct, 5k training examples |
|
||||
| Epochs | 1 |
|
||||
| Effective batch size | 16 (batch 1 × grad. accum. 16) |
|
||||
| Learning rate | 2e-4, cosine schedule, warmup ratio 0.03 |
|
||||
| Hardware | 1× RTX 4060 (8GB) |
|
||||
| Framework | Unsloth + TRL `SFTTrainer` |
|
||||
|
||||
## Benchmark results
|
||||
|
||||
All benchmarks run with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness), each at
|
||||
its standard published shot count, compared against the un-tuned base model.
|
||||
|
||||
| Benchmark | Llama-3.2-1B (base) | This model | Change |
|
||||
|---|---|---|---|
|
||||
| GSM8K | 5.8% | 7.4% | 🟢 +1.5% |
|
||||
| ARC-Challenge | 36.9% | 36.8% | ⚪ -0.1% |
|
||||
| HellaSwag | 64.2% | 63.8% | 🔴 -0.3% |
|
||||
| WinoGrande | 60.8% | 62.4% | 🟢 +1.7% |
|
||||
|
||||
**Speed** (single-request generation, greedy, RTX 4060): **38.36 tokens/sec**
|
||||
(base model: 12.74 tokens/sec)
|
||||
|
||||
### MMLU by category
|
||||
|
||||
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model_id = "OliverSundaram/Llama-3.2-1B-MathCodeInstruct-5k}"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "Below is a math problem. Please solve it step by step."},
|
||||
{"role": "user", "content": "If a train travels 60 miles in 45 minutes, what is its speed in miles per hour?"},
|
||||
]
|
||||
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
|
||||
output = model.generate(inputs, max_new_tokens=512, do_sample=False)
|
||||
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Limitations
|
||||
|
||||
- Trained on a single epoch of a 5k-example subset — not intended to be a general-purpose assistant.
|
||||
- MMLU/ARC/HellaSwag/WinoGrande scores reflect a small 1B-parameter base model and should be read relative to
|
||||
the base model's own scores, not against much larger models.
|
||||
- No safety alignment or RLHF was applied beyond what the base Llama-3.2-1B already has.
|
||||
92
assets/speed_results.json
Normal file
92
assets/speed_results.json
Normal file
@@ -0,0 +1,92 @@
|
||||
{
|
||||
"config": {
|
||||
"prompt": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Solve step by step.",
|
||||
"tokens_per_run": 256,
|
||||
"rounds": 3,
|
||||
"interleaved": true,
|
||||
"dtype": "bfloat16",
|
||||
"greedy": true,
|
||||
"device": "NVIDIA GeForce RTX 4060",
|
||||
"torch": "2.11.0+cu128"
|
||||
},
|
||||
"models": {
|
||||
"base": {
|
||||
"model_path": "unsloth/Llama-3.2-1B",
|
||||
"decode_tok_s": 40.59,
|
||||
"decode_tok_s_min": 39.28,
|
||||
"decode_tok_s_max": 40.62,
|
||||
"end_to_end_tok_s": 40.56,
|
||||
"prefill_ms": 29.3,
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"rounds": [
|
||||
{
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"prefill_s": 0.0321,
|
||||
"total_s": 6.5236,
|
||||
"decode_tok_s": 39.28,
|
||||
"end_to_end_tok_s": 39.24,
|
||||
"gpu": "51C / 2775MHz"
|
||||
},
|
||||
{
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"prefill_s": 0.0293,
|
||||
"total_s": 6.307,
|
||||
"decode_tok_s": 40.62,
|
||||
"end_to_end_tok_s": 40.59,
|
||||
"gpu": "58C / 2775MHz"
|
||||
},
|
||||
{
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"prefill_s": 0.0293,
|
||||
"total_s": 6.3119,
|
||||
"decode_tok_s": 40.59,
|
||||
"end_to_end_tok_s": 40.56,
|
||||
"gpu": "51C / 2775MHz"
|
||||
}
|
||||
]
|
||||
},
|
||||
"5k": {
|
||||
"model_path": "Llama-3.2-1B-MathCodeInstruct-5k/outputs/llama-3.2-1b-5k",
|
||||
"decode_tok_s": 40.67,
|
||||
"decode_tok_s_min": 40.47,
|
||||
"decode_tok_s_max": 40.85,
|
||||
"end_to_end_tok_s": 40.63,
|
||||
"prefill_ms": 30.0,
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"rounds": [
|
||||
{
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"prefill_s": 0.0302,
|
||||
"total_s": 6.3309,
|
||||
"decode_tok_s": 40.47,
|
||||
"end_to_end_tok_s": 40.44,
|
||||
"gpu": "53C / 2775MHz"
|
||||
},
|
||||
{
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"prefill_s": 0.0293,
|
||||
"total_s": 6.2714,
|
||||
"decode_tok_s": 40.85,
|
||||
"end_to_end_tok_s": 40.82,
|
||||
"gpu": "53C / 2775MHz"
|
||||
},
|
||||
{
|
||||
"prompt_tokens": 44,
|
||||
"generated_tokens": 256,
|
||||
"prefill_s": 0.03,
|
||||
"total_s": 6.3007,
|
||||
"decode_tok_s": 40.67,
|
||||
"end_to_end_tok_s": 40.63,
|
||||
"gpu": "49C / 2775MHz"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
6
assets/table_5k.md
Normal file
6
assets/table_5k.md
Normal file
@@ -0,0 +1,6 @@
|
||||
| Benchmark | Llama-3.2-1B (base) | MathCodeInstruct-5k | Change |
|
||||
|---|---|---|---|
|
||||
| GSM8K | 5.8% | 7.4% | 🟢 +1.5% |
|
||||
| ARC-Challenge | 36.9% | 36.8% | ⚪ -0.1% |
|
||||
| HellaSwag | 64.2% | 63.8% | 🔴 -0.3% |
|
||||
| WinoGrande | 60.8% | 62.4% | 🟢 +1.7% |
|
||||
139
chat_template.jinja
Normal file
139
chat_template.jinja
Normal file
@@ -0,0 +1,139 @@
|
||||
{{- bos_token }}
|
||||
{%- if custom_tools is defined %}
|
||||
{%- set tools = custom_tools %}
|
||||
{%- endif %}
|
||||
{%- if not tools_in_user_message is defined %}
|
||||
{%- set tools_in_user_message = true %}
|
||||
{%- endif %}
|
||||
{%- if not date_string is defined %}
|
||||
{%- set date_string = "26 July 2024" %}
|
||||
{%- endif %}
|
||||
{%- if not tools is defined %}
|
||||
{%- set tools = none %}
|
||||
{%- endif %}
|
||||
|
||||
{#- This block extracts the system message, so we can slot it into the right place. #}
|
||||
{%- if messages[0]['role'] == 'system' %}
|
||||
{%- set system_message = messages[0]['content'] %}
|
||||
{%- set messages = messages[1:] %}
|
||||
{%- else %}
|
||||
{%- set system_message = "" %}
|
||||
{%- endif %}
|
||||
|
||||
{#- System message + builtin tools #}
|
||||
{{- "<|start_header_id|>system<|end_header_id|>
|
||||
|
||||
" }}
|
||||
{%- if builtin_tools is defined or tools is not none %}
|
||||
{{- "Environment: ipython
|
||||
" }}
|
||||
{%- endif %}
|
||||
{%- if builtin_tools is defined %}
|
||||
{{- "Tools: " + builtin_tools | reject('equalto', 'code_interpreter') | join(", ") + "
|
||||
|
||||
"}}
|
||||
{%- endif %}
|
||||
{{- "Cutting Knowledge Date: December 2023
|
||||
" }}
|
||||
{{- "Today Date: " + date_string + "
|
||||
|
||||
" }}
|
||||
{%- if tools is not none and not tools_in_user_message %}
|
||||
{{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
|
||||
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||
{{- "Do not use variables.
|
||||
|
||||
" }}
|
||||
{%- for t in tools %}
|
||||
{{- t | tojson(indent=4) }}
|
||||
{{- "
|
||||
|
||||
" }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- system_message }}
|
||||
{{- "<|eot_id|>" }}
|
||||
|
||||
{#- Custom tools are passed in a user message with some extra guidance #}
|
||||
{%- if tools_in_user_message and not tools is none %}
|
||||
{#- Extract the first user message so we can plug it in here #}
|
||||
{%- if messages | length != 0 %}
|
||||
{%- set first_user_message = messages[0]['content'] %}
|
||||
{%- set messages = messages[1:] %}
|
||||
{%- else %}
|
||||
{{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
|
||||
{%- endif %}
|
||||
{{- '<|start_header_id|>user<|end_header_id|>
|
||||
|
||||
' -}}
|
||||
{{- "Given the following functions, please respond with a JSON for a function call " }}
|
||||
{{- "with its proper arguments that best answers the given prompt.
|
||||
|
||||
" }}
|
||||
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||
{{- "Do not use variables.
|
||||
|
||||
" }}
|
||||
{%- for t in tools %}
|
||||
{{- t | tojson(indent=4) }}
|
||||
{{- "
|
||||
|
||||
" }}
|
||||
{%- endfor %}
|
||||
{{- first_user_message + "<|eot_id|>"}}
|
||||
{%- endif %}
|
||||
|
||||
{%- for message in messages %}
|
||||
{%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
|
||||
{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>
|
||||
|
||||
'+ message['content'] + '<|eot_id|>' }}
|
||||
{%- elif 'tool_calls' in message %}
|
||||
{%- if not message.tool_calls|length == 1 %}
|
||||
{{- raise_exception("This model only supports single tool-calls at once!") }}
|
||||
{%- endif %}
|
||||
{%- set tool_call = message.tool_calls[0].function %}
|
||||
{%- if builtin_tools is defined and tool_call.name in builtin_tools %}
|
||||
{{- '<|start_header_id|>assistant<|end_header_id|>
|
||||
|
||||
' -}}
|
||||
{{- "<|python_tag|>" + tool_call.name + ".call(" }}
|
||||
{%- for arg_name, arg_val in tool_call.arguments | items %}
|
||||
{{- arg_name + '="' + arg_val + '"' }}
|
||||
{%- if not loop.last %}
|
||||
{{- ", " }}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{{- ")" }}
|
||||
{%- else %}
|
||||
{{- '<|start_header_id|>assistant<|end_header_id|>
|
||||
|
||||
' -}}
|
||||
{{- '{"name": "' + tool_call.name + '", ' }}
|
||||
{{- '"parameters": ' }}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{{- "}" }}
|
||||
{%- endif %}
|
||||
{%- if builtin_tools is defined %}
|
||||
{#- This means we're in ipython mode #}
|
||||
{{- "<|eom_id|>" }}
|
||||
{%- else %}
|
||||
{{- "<|eot_id|>" }}
|
||||
{%- endif %}
|
||||
{%- elif message.role == "tool" or message.role == "ipython" %}
|
||||
{{- "<|start_header_id|>ipython<|end_header_id|>
|
||||
|
||||
" }}
|
||||
{%- if message.content is mapping or message.content is iterable %}
|
||||
{{- message.content | tojson }}
|
||||
{%- else %}
|
||||
{{- message.content }}
|
||||
{%- endif %}
|
||||
{{- "<|eot_id|>" }}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|start_header_id|>assistant<|end_header_id|>
|
||||
|
||||
' }}
|
||||
{%- endif %}
|
||||
37
config.json
Normal file
37
config.json
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"architectures": [
|
||||
"LlamaForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 128000,
|
||||
"torch_dtype": "bfloat16",
|
||||
"eos_token_id": 128001,
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2048,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 8192,
|
||||
"max_position_embeddings": 131072,
|
||||
"mlp_bias": false,
|
||||
"model_type": "llama",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 16,
|
||||
"num_key_value_heads": 8,
|
||||
"pad_token_id": 128004,
|
||||
"pretraining_tp": 1,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_parameters": {
|
||||
"factor": 32.0,
|
||||
"high_freq_factor": 4.0,
|
||||
"low_freq_factor": 1.0,
|
||||
"original_max_position_embeddings": 8192,
|
||||
"rope_theta": 500000.0,
|
||||
"rope_type": "llama3"
|
||||
},
|
||||
"tie_word_embeddings": true,
|
||||
"unsloth_fixed": true,
|
||||
"unsloth_version": "2026.8.15",
|
||||
"use_cache": true,
|
||||
"vocab_size": 128256
|
||||
}
|
||||
13
generation_config.json
Normal file
13
generation_config.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 128000,
|
||||
"do_sample": true,
|
||||
"eos_token_id": [
|
||||
128001
|
||||
],
|
||||
"max_length": 131072,
|
||||
"pad_token_id": 128004,
|
||||
"temperature": 0.6,
|
||||
"top_p": 0.9,
|
||||
"transformers_version": "5.5.0"
|
||||
}
|
||||
3
model.safetensors
Normal file
3
model.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:83ea42b90853b21b15250f2d52941a2e859ba20ce43ff230d4af07aa67cc4167
|
||||
size 2471645608
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
2069
tokenizer_config.json
Normal file
2069
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user