初始化项目，由ModelHub XC社区提供模型
Model: Melikshah/gpt-oss-20b-clinical-cot-gguf Source: Original Platform
This commit is contained in:
ModelHub XC
40
.gitattributes
vendored
Normal file
40
.gitattributes
vendored
Normal file
@@ -0,0 +1,40 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
model-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
model-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
model-q5_k_m.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
model-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
181
README.md
Normal file
181
README.md
Normal file
@@ -0,0 +1,181 @@
|
||||
---
|
||||
datasets:
|
||||
- FreedomIntelligence/medical-o1-reasoning-SFT
|
||||
- FreedomIntelligence/Medical-R1-Distill-Data
|
||||
- UCSC-VLAA/MedReason
|
||||
language:
|
||||
- en
|
||||
base_model:
|
||||
- unsloth/gpt-oss-20b-BF16
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- SFT
|
||||
- llama.cpp
|
||||
- unsloth
|
||||
- huggingface
|
||||
- QAT
|
||||
- Clinical
|
||||
- chain-of-thought
|
||||
- gguf
|
||||
license: apache-2.0
|
||||
---
|
||||
# Clinical Chain-of-Thought Medical Assistant (GPT-OSS-20B, GGUF)
|
||||
|
||||
This repository provides GGUF builds of a clinical reasoning model that was fine-tuned from `unsloth/gpt-oss-20b-BF16`. Medical questions are answered with an explicit reasoning trace, which is emitted inside `<think> ... </think>` tags, followed by a final answer. The GGUF files are intended for inference through llama.cpp, Ollama, and `llama-cpp-python`. A merged 16-bit checkpoint is also included as safetensors so that the model can be loaded with Transformers from the same repository.
|
||||
|
||||
> **Disclaimer:** This model was built for research and educational purposes only. It is not a medical device and is not intended to be used for diagnosis, treatment, or any clinical decision making. Professional medical advice should always be sought from a qualified clinician.
|
||||
|
||||
## GGUF Files
|
||||
|
||||
Four GGUF builds are provided. The quantization level of each file is indicated by its name.
|
||||
|
||||
| File | Quantization | Size | Recommended for |
|
||||
|---|---|---|---|
|
||||
| `model-q4_k_m.gguf` | 4-bit (Q4_K_M) | 15.8 GB | Default choice. The strongest balance between size and quality is offered here. |
|
||||
| `model-q5_k_m.gguf` | 5-bit (Q5_K_M) | 16.9 GB | Slightly higher fidelity at a modestly larger footprint. |
|
||||
| `model-q8_0.gguf` | 8-bit (Q8_0) | 22.3 GB | Near-lossless quality where footprint is less of a concern. |
|
||||
| `model-f16.gguf` | 16-bit (F16) | 41.9 GB | Unquantized reference. It is mainly intended for re-quantization rather than direct use. |
|
||||
|
||||
For most deployments, `model-q4_k_m.gguf` is recommended. Runtime memory roughly tracks the file size, with additional headroom required for the context window.
|
||||
|
||||
The merged 16-bit checkpoint (`model-0000*-of-00009.safetensors`), the tokenizer, and `chat_template.jinja` are also stored in this repository for use with Transformers.
|
||||
|
||||
## Model Details
|
||||
|
||||
| Property | Value |
|
||||
|---|---|
|
||||
| Base model | `unsloth/gpt-oss-20b-BF16` (OpenAI GPT-OSS-20B, Mixture-of-Experts, 21B total / 3.6B active parameters) |
|
||||
| Fine-tuning method | LoRA supervised fine-tuning (SFT) followed by quantization-aware training (QAT) |
|
||||
| Reasoning format | Harmony chat template with reasoning emitted inside `<think>` tags |
|
||||
| Language | English |
|
||||
| Training hardware | AMD MI300X (ROCm 6.4) |
|
||||
| License | Apache 2.0 (inherited from the base model) |
|
||||
|
||||
## Usage
|
||||
|
||||
The harmony chat template must be used so that the model behaves correctly. The GGUF files already carry this template.
|
||||
|
||||
### Ollama
|
||||
|
||||
```bash
|
||||
ollama run hf.co/Melikshah/gpt-oss-20b-clinical-cot-gguf:Q4_K_M
|
||||
```
|
||||
|
||||
### llama.cpp
|
||||
|
||||
```bash
|
||||
# A single quant can be downloaded with the Hugging Face CLI
|
||||
huggingface-cli download Melikshah/gpt-oss-20b-clinical-cot-gguf model-q4_k_m.gguf --local-dir .
|
||||
|
||||
# The downloaded file can then be run
|
||||
./llama-cli -m model-q4_k_m.gguf \
|
||||
-p "What are the differential diagnoses for acute chest pain?" -n 512
|
||||
```
|
||||
|
||||
### llama-cpp-python
|
||||
|
||||
```python
|
||||
from llama_cpp import Llama
|
||||
|
||||
llm = Llama.from_pretrained(
|
||||
repo_id="Melikshah/gpt-oss-20b-clinical-cot-gguf",
|
||||
filename="model-q4_k_m.gguf",
|
||||
n_ctx=4096,
|
||||
n_gpu_layers=-1,
|
||||
)
|
||||
|
||||
messages = [{"role": "user", "content": "What are the differential diagnoses for acute chest pain?"}]
|
||||
out = llm.create_chat_completion(messages=messages, max_tokens=512, temperature=0.6, top_p=0.95)
|
||||
print(out["choices"][0]["message"]["content"])
|
||||
```
|
||||
|
||||
### Transformers (merged 16-bit checkpoint)
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
import torch
|
||||
|
||||
model_id = "Melikshah/gpt-oss-20b-clinical-cot-gguf"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id, torch_dtype=torch.bfloat16, device_map="auto"
|
||||
)
|
||||
|
||||
messages = [{"role": "user", "content": "What are the differential diagnoses for acute chest pain?"}]
|
||||
inputs = tokenizer.apply_chat_template(
|
||||
messages, add_generation_prompt=True, return_tensors="pt"
|
||||
).to(model.device)
|
||||
|
||||
outputs = model.generate(inputs, max_new_tokens=512)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
The reasoning trace is returned inside `<think> ... </think>` and is intended for inspection and debugging rather than for display to end users.
|
||||
|
||||
## Training Data
|
||||
|
||||
Three medical reasoning datasets were merged and normalized to a unified `instruction`, `reasoning`, `output` schema. The reasoning component was wrapped in `<think>` tags during formatting.
|
||||
|
||||
| Dataset | Description |
|
||||
|---|---|
|
||||
| [FreedomIntelligence/medical-o1-reasoning-SFT](https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT) | Complex chain-of-thought medical reasoning |
|
||||
| [FreedomIntelligence/Medical-R1-Distill-Data](https://huggingface.co/datasets/FreedomIntelligence/Medical-R1-Distill-Data) | Distilled medical reasoning data |
|
||||
| [UCSC-VLAA/MedReason](https://huggingface.co/datasets/UCSC-VLAA/MedReason) | Medical question answering with structured reasoning |
|
||||
|
||||
Rows with empty instructions or answers were filtered out, and a 95% / 5% train / eval split was created.
|
||||
|
||||
## Training Procedure
|
||||
|
||||
The model was trained in two stages. First, a LoRA adapter was trained on the merged dataset and merged back into the base weights. Second, the merged checkpoint was refined with quantization-aware training so that low-bit quality could be preserved. The refined checkpoint was then exported to the GGUF formats listed above.
|
||||
|
||||
| Stage | Key settings |
|
||||
|---|---|
|
||||
| SFT | LoRA rank 32, alpha 64, dropout 0.05, learning rate 1e-4, cosine schedule, 2 epochs, sequence packing, max sequence length 4096 |
|
||||
| QAT | `int4_weight_only` scheme, LoRA rank 16, alpha 32, learning rate 5e-5, 1 epoch |
|
||||
|
||||
LoRA adapters were applied to `q_proj`, `k_proj`, `v_proj`, `o_proj`, `up_proj`, `down_proj`, and `gate_proj`. Training was performed with Unsloth and TRL, and runs were tracked with Weights and Biases.
|
||||
|
||||
## Evaluation
|
||||
|
||||
An LLM-as-a-judge protocol was used. Responses to 50 held-out questions were generated by four candidates, anonymized, shuffled, and then scored against the ground truth. Scores were assigned for accuracy, reasoning quality, safety, and completeness, and an overall score was aggregated from them. The figures below were reported with a GPT-5.2 judge.
|
||||
|
||||
**Win rate (out of 50 prompts)**
|
||||
|
||||
| Model | Wins | Win rate |
|
||||
|---|---|---|
|
||||
| Fine-tuned GPT-OSS (BF16) | 35 | 70.0% |
|
||||
| QAT fine-tuned GPT-OSS | 14 | 28.0% |
|
||||
| Base GPT-OSS (unfine-tuned) | 1 | 2.0% |
|
||||
| GPT-4.1 (OpenAI API) | 0 | 0.0% |
|
||||
|
||||
**Average overall score (1 to 10)**
|
||||
|
||||
| Model | Score |
|
||||
|---|---|
|
||||
| Fine-tuned GPT-OSS (BF16) | 9.16 |
|
||||
| QAT fine-tuned GPT-OSS | 8.32 |
|
||||
| GPT-4.1 (OpenAI API) | 7.37 |
|
||||
| Base GPT-OSS (unfine-tuned) | 7.12 |
|
||||
|
||||
> These figures were obtained from a 50-question sample scored by a single judge, and they should therefore be treated as preliminary.
|
||||
|
||||
## Demo
|
||||
[](https://www.youtube.com/watch?v=IGAzB2DXqoM "Clinical CoT Medical Assistant Demo")
|
||||
|
||||
## Intended Use and Limitations
|
||||
|
||||
This model is intended to be used for research into medical reasoning, for educational demonstrations, and for the study of chain-of-thought fine-tuning. Clinical deployment is not supported.
|
||||
|
||||
The following limitations should be kept in mind. Incorrect or fabricated information may be produced, since hallucination is not eliminated by fine-tuning. Coverage is limited to the topics present in the training datasets, and English is the only language that was targeted. Some quality loss is expected at lower quantization levels. The published evaluation is small in scale and was scored by a single automated judge, so the reported quality should not be over-interpreted.
|
||||
|
||||
## Acknowledgements and Citation
|
||||
|
||||
The base model `gpt-oss-20b` was released by OpenAI under the Apache 2.0 license, and fine-tuning was performed with [Unsloth](https://github.com/unslothai/unsloth). The full training and evaluation pipeline is available at [TheDeadcoder/medical-cot-assistant](https://github.com/TheDeadcoder/medical-cot-assistant).
|
||||
|
||||
The source datasets should be cited if this model is used:
|
||||
|
||||
```bibtex
|
||||
@misc{medical-o1-reasoning, title={Medical-O1-Reasoning-SFT}, author={FreedomIntelligence}, year={2024}, publisher={HuggingFace}}
|
||||
@misc{medical-r1-distill, title={Medical-R1-Distill-Data}, author={FreedomIntelligence}, year={2024}, publisher={HuggingFace}}
|
||||
@misc{medreason, title={MedReason}, author={UCSC-VLAA}, year={2024}, publisher={HuggingFace}}
|
||||
```
|
||||
315
chat_template.jinja
Normal file
315
chat_template.jinja
Normal file
@@ -0,0 +1,315 @@
|
||||
{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth chat template fixes. Edited from ggml-org & OpenAI #}
|
||||
{#-
|
||||
In addition to the normal inputs of `messages` and `tools`, this template also accepts the
|
||||
following kwargs:
|
||||
- "builtin_tools": A list, can contain "browser" and/or "python".
|
||||
- "model_identity": A string that optionally describes the model identity.
|
||||
- "reasoning_effort": A string that describes the reasoning effort, defaults to "medium".
|
||||
#}
|
||||
|
||||
{#- Tool Definition Rendering ============================================== #}
|
||||
{%- macro render_typescript_type(param_spec, required_params, is_nullable=false) -%}
|
||||
{%- if param_spec.type == "array" -%}
|
||||
{%- if param_spec['items'] -%}
|
||||
{%- if param_spec['items']['type'] == "string" -%}
|
||||
{{- "string[]" }}
|
||||
{%- elif param_spec['items']['type'] == "number" -%}
|
||||
{{- "number[]" }}
|
||||
{%- elif param_spec['items']['type'] == "integer" -%}
|
||||
{{- "number[]" }}
|
||||
{%- elif param_spec['items']['type'] == "boolean" -%}
|
||||
{{- "boolean[]" }}
|
||||
{%- else -%}
|
||||
{%- set inner_type = render_typescript_type(param_spec['items'], required_params) -%}
|
||||
{%- if inner_type == "object | object" or inner_type|length > 50 -%}
|
||||
{{- "any[]" }}
|
||||
{%- else -%}
|
||||
{{- inner_type + "[]" }}
|
||||
{%- endif -%}
|
||||
{%- endif -%}
|
||||
{%- if param_spec.nullable -%}
|
||||
{{- " | null" }}
|
||||
{%- endif -%}
|
||||
{%- else -%}
|
||||
{{- "any[]" }}
|
||||
{%- if param_spec.nullable -%}
|
||||
{{- " | null" }}
|
||||
{%- endif -%}
|
||||
{%- endif -%}
|
||||
{%- elif param_spec.type is defined and param_spec.type is iterable and param_spec.type is not string and param_spec.type is not mapping and param_spec.type[0] is defined -%}
|
||||
{#- Handle array of types like ["object", "object"] from Union[dict, list] #}
|
||||
{%- if param_spec.type | length > 1 -%}
|
||||
{{- param_spec.type | join(" | ") }}
|
||||
{%- else -%}
|
||||
{{- param_spec.type[0] }}
|
||||
{%- endif -%}
|
||||
{%- elif param_spec.oneOf -%}
|
||||
{#- Handle oneOf schemas - check for complex unions and fallback to any #}
|
||||
{%- set has_object_variants = false -%}
|
||||
{%- for variant in param_spec.oneOf -%}
|
||||
{%- if variant.type == "object" -%}
|
||||
{%- set has_object_variants = true -%}
|
||||
{%- endif -%}
|
||||
{%- endfor -%}
|
||||
{%- if has_object_variants and param_spec.oneOf|length > 1 -%}
|
||||
{{- "any" }}
|
||||
{%- else -%}
|
||||
{%- for variant in param_spec.oneOf -%}
|
||||
{{- render_typescript_type(variant, required_params) -}}
|
||||
{%- if variant.description %}
|
||||
{{- "// " + variant.description }}
|
||||
{%- endif -%}
|
||||
{%- if variant.default is defined %}
|
||||
{{ "// default: " + variant.default|tojson }}
|
||||
{%- endif -%}
|
||||
{%- if not loop.last %}
|
||||
{{- " | " }}
|
||||
{% endif -%}
|
||||
{%- endfor -%}
|
||||
{%- endif -%}
|
||||
{%- elif param_spec.type == "string" -%}
|
||||
{%- if param_spec.enum -%}
|
||||
{{- '"' + param_spec.enum|join('" | "') + '"' -}}
|
||||
{%- else -%}
|
||||
{{- "string" }}
|
||||
{%- if param_spec.nullable %}
|
||||
{{- " | null" }}
|
||||
{%- endif -%}
|
||||
{%- endif -%}
|
||||
{%- elif param_spec.type == "number" -%}
|
||||
{{- "number" }}
|
||||
{%- elif param_spec.type == "integer" -%}
|
||||
{{- "number" }}
|
||||
{%- elif param_spec.type == "boolean" -%}
|
||||
{{- "boolean" }}
|
||||
|
||||
{%- elif param_spec.type == "object" -%}
|
||||
{%- if param_spec.properties -%}
|
||||
{{- "{\n" }}
|
||||
{%- for prop_name, prop_spec in param_spec.properties.items() -%}
|
||||
{{- prop_name -}}
|
||||
{%- if prop_name not in (param_spec.required or []) -%}
|
||||
{{- "?" }}
|
||||
{%- endif -%}
|
||||
{{- ": " }}
|
||||
{{ render_typescript_type(prop_spec, param_spec.required or []) }}
|
||||
{%- if not loop.last -%}
|
||||
{{-", " }}
|
||||
{%- endif -%}
|
||||
{%- endfor -%}
|
||||
{{- "}" }}
|
||||
{%- else -%}
|
||||
{{- "object" }}
|
||||
{%- endif -%}
|
||||
{%- else -%}
|
||||
{{- "any" }}
|
||||
{%- endif -%}
|
||||
{%- endmacro -%}
|
||||
|
||||
{%- macro render_tool_namespace(namespace_name, tools) -%}
|
||||
{{- "## " + namespace_name + "\n\n" }}
|
||||
{{- "namespace " + namespace_name + " {\n\n" }}
|
||||
{%- for tool in tools %}
|
||||
{%- set tool = tool.function %}
|
||||
{{- "// " + tool.description + "\n" }}
|
||||
{{- "type "+ tool.name + " = " }}
|
||||
{%- if tool.parameters and tool.parameters.properties -%}
|
||||
{{- "(_: " }}
|
||||
{{- "{\n" }}
|
||||
{%- for param_name, param_spec in tool.parameters.properties.items() %}
|
||||
{{- "// " + param_spec.description + "\n" }}
|
||||
{{- param_name }}
|
||||
{%- if param_name not in (tool.parameters.required or []) -%}
|
||||
{{- "?" }}
|
||||
{%- endif -%}
|
||||
{{- ": " }}
|
||||
{{- render_typescript_type(param_spec, tool.parameters.required or []) }}
|
||||
{%- if param_spec.default is defined -%}
|
||||
{%- if param_spec.enum %}
|
||||
{{- ", // default: " + param_spec.default }}
|
||||
{%- elif param_spec.oneOf %}
|
||||
{{- "// default: " + param_spec.default }}
|
||||
{%- else %}
|
||||
{{- ", // default: " + param_spec.default|tojson }}
|
||||
{%- endif -%}
|
||||
{%- endif -%}
|
||||
{%- if not loop.last %}
|
||||
{{- ",\n" }}
|
||||
{%- else %}
|
||||
{{- "\n" }}
|
||||
{%- endif -%}
|
||||
{%- endfor %}
|
||||
{{- "}) => any;\n\n" }}
|
||||
{%- else -%}
|
||||
{{- "() => any;\n\n" }}
|
||||
{%- endif -%}
|
||||
{%- endfor %}
|
||||
{{- "} // namespace " + namespace_name }}
|
||||
{%- endmacro -%}
|
||||
|
||||
{%- macro render_builtin_tools(browser_tool, python_tool) -%}
|
||||
{%- if browser_tool %}
|
||||
{{- "## browser\n\n" }}
|
||||
{{- "// Tool for browsing.\n" }}
|
||||
{{- "// The `cursor` appears in brackets before each browsing display: `[{cursor}]`.\n" }}
|
||||
{{- "// Cite information from the tool using the following format:\n" }}
|
||||
{{- "// `【{cursor}†L{line_start}(-L{line_end})?】`, for example: `【6†L9-L11】` or `【8†L3】`.\n" }}
|
||||
{{- "// Do not quote more than 10 words directly from the tool output.\n" }}
|
||||
{{- "// sources=web (default: web)\n" }}
|
||||
{{- "namespace browser {\n\n" }}
|
||||
{{- "// Searches for information related to `query` and displays `topn` results.\n" }}
|
||||
{{- "type search = (_: {\n" }}
|
||||
{{- "query: string,\n" }}
|
||||
{{- "topn?: number, // default: 10\n" }}
|
||||
{{- "source?: string,\n" }}
|
||||
{{- "}) => any;\n\n" }}
|
||||
{{- "// Opens the link `id` from the page indicated by `cursor` starting at line number `loc`, showing `num_lines` lines.\n" }}
|
||||
{{- "// Valid link ids are displayed with the formatting: `【{id}†.*】`.\n" }}
|
||||
{{- "// If `cursor` is not provided, the most recent page is implied.\n" }}
|
||||
{{- "// If `id` is a string, it is treated as a fully qualified URL associated with `source`.\n" }}
|
||||
{{- "// If `loc` is not provided, the viewport will be positioned at the beginning of the document or centered on the most relevant passage, if available.\n" }}
|
||||
{{- "// Use this function without `id` to scroll to a new location of an opened page.\n" }}
|
||||
{{- "type open = (_: {\n" }}
|
||||
{{- "id?: number | string, // default: -1\n" }}
|
||||
{{- "cursor?: number, // default: -1\n" }}
|
||||
{{- "loc?: number, // default: -1\n" }}
|
||||
{{- "num_lines?: number, // default: -1\n" }}
|
||||
{{- "view_source?: boolean, // default: false\n" }}
|
||||
{{- "source?: string,\n" }}
|
||||
{{- "}) => any;\n\n" }}
|
||||
{{- "// Finds exact matches of `pattern` in the current page, or the page given by `cursor`.\n" }}
|
||||
{{- "type find = (_: {\n" }}
|
||||
{{- "pattern: string,\n" }}
|
||||
{{- "cursor?: number, // default: -1\n" }}
|
||||
{{- "}) => any;\n\n" }}
|
||||
{{- "} // namespace browser\n\n" }}
|
||||
{%- endif -%}
|
||||
|
||||
{%- if python_tool %}
|
||||
{{- "## python\n\n" }}
|
||||
{{- "Use this tool to execute Python code in your chain of thought. The code will not be shown to the user. This tool should be used for internal reasoning, but not for code that is intended to be visible to the user (e.g. when creating plots, tables, or files).\n\n" }}
|
||||
{{- "When you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment. python will respond with the output of the execution or time out after 120.0 seconds. The drive at '/mnt/data' can be used to save and persist user files. Internet access for this session is UNKNOWN. Depends on the cluster.\n\n" }}
|
||||
{%- endif -%}
|
||||
{%- endmacro -%}
|
||||
|
||||
{#- System Message Construction ============================================ #}
|
||||
{%- macro build_system_message() -%}
|
||||
{%- if model_identity is not defined %}
|
||||
{{- "You are ChatGPT, a large language model trained by OpenAI.\n" -}}
|
||||
{%- else %}
|
||||
{{- model_identity }}
|
||||
{%- endif %}
|
||||
{{- "Knowledge cutoff: 2024-06\n" }}
|
||||
{{- "Current date: " + strftime_now("%Y-%m-%d") + "\n\n" }}
|
||||
{%- if reasoning_effort is not defined %}
|
||||
{%- set reasoning_effort = "medium" %}
|
||||
{%- endif %}
|
||||
{{- "Reasoning: " + reasoning_effort + "\n\n" }}
|
||||
{%- if builtin_tools is defined %}
|
||||
{{- "# Tools\n\n" }}
|
||||
{%- set available_builtin_tools = namespace(browser=false, python=false) %}
|
||||
{%- for tool in builtin_tools %}
|
||||
{%- if tool == "browser" %}
|
||||
{%- set available_builtin_tools.browser = true %}
|
||||
{%- elif tool == "python" %}
|
||||
{%- set available_builtin_tools.python = true %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{{- render_builtin_tools(available_builtin_tools.browser, available_builtin_tools.python) }}
|
||||
{%- endif -%}
|
||||
{{- "# Valid channels: analysis, commentary, final. Channel must be included for every message." }}
|
||||
{%- if tools is defined -%}
|
||||
{{- "\nCalls to these tools must go to the commentary channel: 'functions'." }}
|
||||
{%- endif -%}
|
||||
{%- endmacro -%}
|
||||
|
||||
{#- Main Template Logic ================================================= #}
|
||||
{#- Set defaults #}
|
||||
|
||||
{#- Render system message #}
|
||||
{{- "<|start|>system<|message|>" }}
|
||||
{{- build_system_message() }}
|
||||
{{- "<|end|>" }}
|
||||
|
||||
{#- Extract developer message #}
|
||||
{%- if messages[0].role == "developer" or messages[0].role == "system" %}
|
||||
{%- set developer_message = messages[0].content %}
|
||||
{%- set loop_messages = messages[1:] %}
|
||||
{%- else %}
|
||||
{%- set developer_message = "" %}
|
||||
{%- set loop_messages = messages %}
|
||||
{%- endif %}
|
||||
|
||||
{#- Render developer message #}
|
||||
{%- if developer_message or tools %}
|
||||
{{- "<|start|>developer<|message|>" }}
|
||||
{%- if developer_message %}
|
||||
{{- "# Instructions\n\n" }}
|
||||
{{- developer_message }}
|
||||
{%- endif %}
|
||||
{%- if tools -%}
|
||||
{{- "\n\n" }}
|
||||
{{- "# Tools\n\n" }}
|
||||
{{- render_tool_namespace("functions", tools) }}
|
||||
{%- endif -%}
|
||||
{{- "<|end|>" }}
|
||||
{%- endif %}
|
||||
|
||||
{#- Render messages #}
|
||||
{%- set last_tool_call = namespace(name=none) %}
|
||||
{%- for message in loop_messages -%}
|
||||
{#- At this point only assistant/user/tool messages should remain #}
|
||||
{%- if message.role == 'assistant' -%}
|
||||
{%- if "tool_calls" in message %}
|
||||
{#- We assume max 1 tool call per message, and so we infer the tool call name #}
|
||||
{#- in "tool" messages from the most recent assistant tool call name #}
|
||||
{%- set tool_call = message.tool_calls[0] %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{%- if message.content %}
|
||||
{{- "<|start|>assistant<|channel|>analysis<|message|>" + message.content + "<|end|>" }}
|
||||
{%- endif %}
|
||||
{{- "<|start|>assistant to=" }}
|
||||
{{- "functions." + tool_call.name + "<|channel|>commentary json<|message|>" }}
|
||||
{{- tool_call.arguments|tojson }}
|
||||
{{- "<|call|>" }}
|
||||
{%- set last_tool_call.name = tool_call.name %}
|
||||
{%- elif "thinking" in message and loop.last and not add_generation_prompt %}
|
||||
{#- Only render the CoT if the final turn is an assistant turn and add_generation_prompt is false #}
|
||||
{#- This is a situation that should only occur in training, never in inference. #}
|
||||
{{- "<|start|>assistant<|channel|>analysis<|message|>" + message.thinking + "<|end|>" }}
|
||||
{#- <|return|> indicates the end of generation, but <|end|> does not #}
|
||||
{#- <|return|> should never be an input to the model, but we include it as the final token #}
|
||||
{#- when training, so the model learns to emit it. #}
|
||||
{{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|return|>" }}
|
||||
{%- set last_tool_call.name = none %}
|
||||
{%- elif "thinking" in message %}
|
||||
{#- CoT is dropped during all previous turns, so we never render it for inference #}
|
||||
{{- "<|start|>assistant<|channel|>final<|message|>" + message.content + "<|end|>" }}
|
||||
{%- set last_tool_call.name = none %}
|
||||
{%- elif loop.last and not add_generation_prompt %}
|
||||
{#- <|return|> indicates the end of generation, but <|end|> does not #}
|
||||
{#- <|return|> should never be an input to the model, but we include it as the final token #}
|
||||
{#- when training, so the model learns to emit it. #}
|
||||
{{- "<|start|>assistant<|message|>" + message.content + "<|return|>" }}
|
||||
{%- else %}
|
||||
{{- "<|start|>assistant<|message|>" + message.content + "<|end|>" }}
|
||||
{%- set last_tool_call.name = none %}
|
||||
{%- endif %}
|
||||
{%- elif message.role == 'tool' -%}
|
||||
{%- if last_tool_call.name is none %}
|
||||
{{- raise_exception("Message has tool role, but there was no previous assistant message with a tool call!") }}
|
||||
{%- endif %}
|
||||
{{- "<|start|>functions." + last_tool_call.name }}
|
||||
{{- " to=assistant<|channel|>commentary<|message|>" + message.content|tojson + "<|end|>" }}
|
||||
{%- else -%}
|
||||
{{- "<|start|>user<|message|>" + message.content + "<|end|>" }}
|
||||
{%- endif -%}
|
||||
{%- endfor -%}
|
||||
|
||||
{#- Generation prompt #}
|
||||
{%- if add_generation_prompt -%}
|
||||
<|start|>assistant
|
||||
{%- endif -%}
|
||||
{# Copyright 2025-present Unsloth. Apache 2.0 License. Unsloth chat template fixes. Edited from ggml-org & OpenAI #}
|
||||
70
config.json
Normal file
70
config.json
Normal file
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"architectures": [
|
||||
"GptOssForCausalLM"
|
||||
],
|
||||
"attention_bias": true,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 199998,
|
||||
"dtype": "bfloat16",
|
||||
"eos_token_id": 200002,
|
||||
"experts_per_token": 4,
|
||||
"head_dim": 64,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2880,
|
||||
"initial_context_length": 4096,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 2880,
|
||||
"layer_types": [
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention",
|
||||
"sliding_attention",
|
||||
"full_attention"
|
||||
],
|
||||
"max_position_embeddings": 131072,
|
||||
"model_type": "gpt_oss",
|
||||
"num_attention_heads": 64,
|
||||
"num_experts_per_tok": 4,
|
||||
"num_hidden_layers": 24,
|
||||
"num_key_value_heads": 8,
|
||||
"num_local_experts": 32,
|
||||
"output_router_logits": false,
|
||||
"pad_token_id": 199999,
|
||||
"rms_norm_eps": 1e-05,
|
||||
"rope_scaling": {
|
||||
"beta_fast": 32.0,
|
||||
"beta_slow": 1.0,
|
||||
"factor": 32.0,
|
||||
"original_max_position_embeddings": 4096,
|
||||
"rope_type": "yarn",
|
||||
"truncate": false
|
||||
},
|
||||
"rope_theta": 150000,
|
||||
"router_aux_loss_coef": 0.9,
|
||||
"sliding_window": 128,
|
||||
"swiglu_limit": 7.0,
|
||||
"tie_word_embeddings": false,
|
||||
"transformers_version": "4.57.3",
|
||||
"unsloth_version": "2025.12.9",
|
||||
"use_cache": true,
|
||||
"vocab_size": 201088
|
||||
}
|
||||
8
generation_config.json
Normal file
8
generation_config.json
Normal file
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"_from_model_config": true,
|
||||
"bos_token_id": 199998,
|
||||
"eos_token_id": 200002,
|
||||
"max_length": 131072,
|
||||
"pad_token_id": 199999,
|
||||
"transformers_version": "4.57.3"
|
||||
}
|
||||
3
model-00001-of-00009.safetensors
Normal file
3
model-00001-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:9f1e8c82155425f0696d80a86dfc1a093dd0f44bea634900e11cbf31fcd97a67
|
||||
size 4505433560
|
||||
3
model-00002-of-00009.safetensors
Normal file
3
model-00002-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7c1360927ad04e1d14014a57a75dc8fed58f0cd20f96d9785b676cc5f9f89cbf
|
||||
size 4940821016
|
||||
3
model-00003-of-00009.safetensors
Normal file
3
model-00003-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:68dd5b565d52722e1bb2aa55ce5c1a470f87e65139c0d18d17e5013eedafe362
|
||||
size 4940821016
|
||||
3
model-00004-of-00009.safetensors
Normal file
3
model-00004-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c5ae4939607892ffefa72dc3e2192ca385dc92bf26123d7b1435b31057acf316
|
||||
size 4940821048
|
||||
3
model-00005-of-00009.safetensors
Normal file
3
model-00005-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0e25366c92b7e5c87cf80101d1479b24fb701b4548373c40fd1afbbd8a28d6a7
|
||||
size 4940821064
|
||||
3
model-00006-of-00009.safetensors
Normal file
3
model-00006-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:7541069e5801f462e3bdbc6670dfd80b3dbea6dbb01f3d60ff239a486bc674bb
|
||||
size 4940821064
|
||||
3
model-00007-of-00009.safetensors
Normal file
3
model-00007-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:287f3147d0e91e99d036c80d9d99be144ee047047f75997697773280cc43f641
|
||||
size 4940821064
|
||||
3
model-00008-of-00009.safetensors
Normal file
3
model-00008-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1ccf4267add1ce8ac55ccecde5f86c993ac2a79933e8c1cd4659bce69ac33676
|
||||
size 4940821064
|
||||
3
model-00009-of-00009.safetensors
Normal file
3
model-00009-of-00009.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5e61be7caf2e6cdb63645125cb50ea7c0e826e0227e3e0149c35287e79bfe4fc
|
||||
size 2751933056
|
||||
3
model-f16.gguf
Normal file
3
model-f16.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:853b07c3206210e619210f0b518467324019946ea8c0fde8cace26484d263651
|
||||
size 41860886560
|
||||
3
model-q4_k_m.gguf
Normal file
3
model-q4_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:3cea7590a458597daa108a9c4e5fbb107ecd30b11edc924639c2cba03f91dcc1
|
||||
size 15805134880
|
||||
3
model-q5_k_m.gguf
Normal file
3
model-q5_k_m.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ef85a9131c9afe7669fd4546a7fdedccfb5b3e5acbc856408369e77eba62416e
|
||||
size 16893060640
|
||||
3
model-q8_0.gguf
Normal file
3
model-q8_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:aa71acb6c11f6630d381c772310b270c2cc7fba10345df1e4cd25bee117e348f
|
||||
size 22261910560
|
||||
419
model.safetensors.index.json
Normal file
419
model.safetensors.index.json
Normal file
@@ -0,0 +1,419 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_parameters": 20914757184,
|
||||
"total_size": 41843067648
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00009-of-00009.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.experts.down_proj": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.experts.down_proj_bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.experts.gate_up_proj": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.experts.gate_up_proj_bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.router.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.mlp.router.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.sinks": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.experts.down_proj": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.experts.down_proj_bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.experts.gate_up_proj": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.experts.gate_up_proj_bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.router.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.mlp.router.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.sinks": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.mlp.experts.down_proj": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.mlp.experts.down_proj_bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.mlp.experts.gate_up_proj": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.mlp.experts.gate_up_proj_bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.mlp.router.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.mlp.router.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.sinks": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.11.mlp.experts.down_proj": "model-00005-of-00009.safetensors",
|
||||
"model.layers.11.mlp.experts.down_proj_bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.11.mlp.experts.gate_up_proj": "model-00005-of-00009.safetensors",
|
||||
"model.layers.11.mlp.experts.gate_up_proj_bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.11.mlp.router.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.mlp.router.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.sinks": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.mlp.experts.down_proj": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.mlp.experts.down_proj_bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.mlp.experts.gate_up_proj": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.mlp.experts.gate_up_proj_bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.mlp.router.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.mlp.router.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.sinks": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.mlp.experts.down_proj": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.mlp.experts.down_proj_bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.mlp.experts.gate_up_proj": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.mlp.experts.gate_up_proj_bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.mlp.router.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.mlp.router.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.sinks": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.14.mlp.experts.down_proj": "model-00006-of-00009.safetensors",
|
||||
"model.layers.14.mlp.experts.down_proj_bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.14.mlp.experts.gate_up_proj": "model-00006-of-00009.safetensors",
|
||||
"model.layers.14.mlp.experts.gate_up_proj_bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.14.mlp.router.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.mlp.router.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.sinks": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.bias": "model-00005-of-00009.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.mlp.experts.down_proj": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.mlp.experts.down_proj_bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.mlp.experts.gate_up_proj": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.mlp.experts.gate_up_proj_bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.mlp.router.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.mlp.router.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.sinks": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.mlp.experts.down_proj": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.mlp.experts.down_proj_bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.mlp.experts.gate_up_proj": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.mlp.experts.gate_up_proj_bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.mlp.router.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.mlp.router.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.sinks": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.17.mlp.experts.down_proj": "model-00007-of-00009.safetensors",
|
||||
"model.layers.17.mlp.experts.down_proj_bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.17.mlp.experts.gate_up_proj": "model-00007-of-00009.safetensors",
|
||||
"model.layers.17.mlp.experts.gate_up_proj_bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.17.mlp.router.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.mlp.router.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.sinks": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.bias": "model-00006-of-00009.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.mlp.experts.down_proj": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.mlp.experts.down_proj_bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.mlp.experts.gate_up_proj": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.mlp.experts.gate_up_proj_bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.mlp.router.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.mlp.router.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.sinks": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.mlp.experts.down_proj": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.mlp.experts.down_proj_bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.mlp.experts.gate_up_proj": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.mlp.experts.gate_up_proj_bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.mlp.router.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.mlp.router.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.sinks": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.2.mlp.experts.down_proj": "model-00002-of-00009.safetensors",
|
||||
"model.layers.2.mlp.experts.down_proj_bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.2.mlp.experts.gate_up_proj": "model-00002-of-00009.safetensors",
|
||||
"model.layers.2.mlp.experts.gate_up_proj_bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.2.mlp.router.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.mlp.router.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.sinks": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00009.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.20.mlp.experts.down_proj": "model-00008-of-00009.safetensors",
|
||||
"model.layers.20.mlp.experts.down_proj_bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.20.mlp.experts.gate_up_proj": "model-00008-of-00009.safetensors",
|
||||
"model.layers.20.mlp.experts.gate_up_proj_bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.20.mlp.router.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.mlp.router.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.sinks": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.bias": "model-00007-of-00009.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.mlp.experts.down_proj": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.mlp.experts.down_proj_bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.mlp.experts.gate_up_proj": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.mlp.experts.gate_up_proj_bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.mlp.router.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.mlp.router.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.sinks": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.mlp.experts.down_proj": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.mlp.experts.down_proj_bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.mlp.experts.gate_up_proj": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.mlp.experts.gate_up_proj_bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.mlp.router.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.mlp.router.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.sinks": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.23.mlp.experts.down_proj": "model-00009-of-00009.safetensors",
|
||||
"model.layers.23.mlp.experts.down_proj_bias": "model-00009-of-00009.safetensors",
|
||||
"model.layers.23.mlp.experts.gate_up_proj": "model-00009-of-00009.safetensors",
|
||||
"model.layers.23.mlp.experts.gate_up_proj_bias": "model-00009-of-00009.safetensors",
|
||||
"model.layers.23.mlp.router.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.mlp.router.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.sinks": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.bias": "model-00008-of-00009.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.mlp.experts.down_proj": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.mlp.experts.down_proj_bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.mlp.experts.gate_up_proj": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.mlp.experts.gate_up_proj_bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.mlp.router.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.mlp.router.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.sinks": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.experts.down_proj": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.experts.down_proj_bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.experts.gate_up_proj": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.experts.gate_up_proj_bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.router.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.mlp.router.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.sinks": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.5.mlp.experts.down_proj": "model-00003-of-00009.safetensors",
|
||||
"model.layers.5.mlp.experts.down_proj_bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.5.mlp.experts.gate_up_proj": "model-00003-of-00009.safetensors",
|
||||
"model.layers.5.mlp.experts.gate_up_proj_bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.5.mlp.router.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.mlp.router.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.sinks": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.bias": "model-00002-of-00009.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.mlp.experts.down_proj": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.mlp.experts.down_proj_bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.mlp.experts.gate_up_proj": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.mlp.experts.gate_up_proj_bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.mlp.router.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.mlp.router.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.sinks": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.mlp.experts.down_proj": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.mlp.experts.down_proj_bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.mlp.experts.gate_up_proj": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.mlp.experts.gate_up_proj_bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.mlp.router.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.mlp.router.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.sinks": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.8.mlp.experts.down_proj": "model-00004-of-00009.safetensors",
|
||||
"model.layers.8.mlp.experts.down_proj_bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.8.mlp.experts.gate_up_proj": "model-00004-of-00009.safetensors",
|
||||
"model.layers.8.mlp.experts.gate_up_proj_bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.8.mlp.router.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.mlp.router.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.sinks": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.bias": "model-00003-of-00009.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.mlp.experts.down_proj": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.mlp.experts.down_proj_bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.mlp.experts.gate_up_proj": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.mlp.experts.gate_up_proj_bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.mlp.router.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.mlp.router.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.sinks": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.bias": "model-00004-of-00009.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
|
||||
"model.norm.weight": "model-00009-of-00009.safetensors"
|
||||
}
|
||||
}
|
||||
23
special_tokens_map.json
Normal file
23
special_tokens_map.json
Normal file
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"bos_token": {
|
||||
"content": "<|startoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"eos_token": {
|
||||
"content": "<|return|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
3
tokenizer.json
Normal file
3
tokenizer.json
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0614fe83cadab421296e664e1f48f4261fa8fef6e03e63bb75c20f38e37d07d3
|
||||
size 27868174
|
||||
184
tokenizer_config.json
Normal file
184
tokenizer_config.json
Normal file
@@ -0,0 +1,184 @@
|
||||
{
|
||||
"added_tokens_decoder": {
|
||||
"199998": {
|
||||
"content": "<|startoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"199999": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200000": {
|
||||
"content": "<|reserved_200000|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200001": {
|
||||
"content": "<|reserved_200001|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200002": {
|
||||
"content": "<|return|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200003": {
|
||||
"content": "<|constrain|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200004": {
|
||||
"content": "<|reserved_200004|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200005": {
|
||||
"content": "<|channel|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200006": {
|
||||
"content": "<|start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200007": {
|
||||
"content": "<|end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200008": {
|
||||
"content": "<|message|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200009": {
|
||||
"content": "<|reserved_200009|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200010": {
|
||||
"content": "<|reserved_200010|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200011": {
|
||||
"content": "<|reserved_200011|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200012": {
|
||||
"content": "<|call|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200013": {
|
||||
"content": "<|reserved_200013|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200014": {
|
||||
"content": "<|reserved_200014|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200015": {
|
||||
"content": "<|reserved_200015|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200016": {
|
||||
"content": "<|reserved_200016|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200017": {
|
||||
"content": "<|reserved_200017|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"200018": {
|
||||
"content": "<|endofprompt|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
}
|
||||
},
|
||||
"bos_token": "<|startoftext|>",
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|return|>",
|
||||
"extra_special_tokens": {},
|
||||
"model_input_names": [
|
||||
"input_ids",
|
||||
"attention_mask"
|
||||
],
|
||||
"model_max_length": 1000000000000000019884624838656,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"padding_side": "left",
|
||||
"tokenizer_class": "PreTrainedTokenizerFast"
|
||||
}
|
||||
Reference in New Issue
Block a user