初始化项目，由ModelHub XC社区提供模型
Model: RedHatAI/Llama-3.1-8B-Instruct-NVFP4 Source: Original Platform
This commit is contained in:
ModelHub XC
51
.gitattributes
vendored
Normal file
51
.gitattributes
vendored
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zstandard filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.db* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ark* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/*ckpt*data* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/*ckpt*.meta filter=lfs diff=lfs merge=lfs -text
|
||||||
|
**/*ckpt*.index filter=lfs diff=lfs merge=lfs -text
|
||||||
|
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gguf* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ggml filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.llamafile* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
|
||||||
|
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||||
|
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
395
README.md
Normal file
395
README.md
Normal file
@@ -0,0 +1,395 @@
|
|||||||
|
---
|
||||||
|
tags:
|
||||||
|
- fp4
|
||||||
|
- vllm
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
- de
|
||||||
|
- fr
|
||||||
|
- it
|
||||||
|
- pt
|
||||||
|
- hi
|
||||||
|
- es
|
||||||
|
- th
|
||||||
|
pipeline_tag: text-generation
|
||||||
|
license: llama3.1
|
||||||
|
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
|
||||||
|
---
|
||||||
|
|
||||||
|
# Meta-Llama-3.1-8B-Instruct-NVFP4
|
||||||
|
|
||||||
|
## Model Overview
|
||||||
|
- **Model Architecture:** Meta-Llama-3.1
|
||||||
|
- **Input:** Text
|
||||||
|
- **Output:** Text
|
||||||
|
- **Model Optimizations:**
|
||||||
|
- **Weight quantization:** FP4
|
||||||
|
- **Activation quantization:** FP4
|
||||||
|
- **Intended Use Cases:** Intended for commercial and research use in multiple languages. Similarly to [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct), this models is intended for assistant-like chat.
|
||||||
|
- **Out-of-scope:** Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in languages other than English.
|
||||||
|
- **Release Date:** 10/23/2025
|
||||||
|
- **Version:** 1.0
|
||||||
|
- **License(s):** [llama3.1](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B/blob/main/LICENSE)
|
||||||
|
- **Model Developers:** RedHatAI
|
||||||
|
|
||||||
|
This model is a quantized version of [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct).
|
||||||
|
It was evaluated on a several tasks to assess the its quality in comparison to the unquatized model.
|
||||||
|
|
||||||
|
### Model Optimizations
|
||||||
|
|
||||||
|
This model was obtained by quantizing the weights and activations of [Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) to FP4 data type, ready for inference with vLLM>=0.9.1
|
||||||
|
This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 75%.
|
||||||
|
|
||||||
|
Only the weights and activations of the linear operators within transformers blocks are quantized using [LLM Compressor](https://github.com/vllm-project/llm-compressor).
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
### Use with vLLM
|
||||||
|
|
||||||
|
This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend, as shown in the example below.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from vllm import LLM, SamplingParams
|
||||||
|
from transformers import AutoTokenizer
|
||||||
|
|
||||||
|
model_id = "RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4"
|
||||||
|
number_gpus = 1
|
||||||
|
|
||||||
|
sampling_params = SamplingParams(temperature=0.6, top_p=0.9, max_tokens=256)
|
||||||
|
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||||
|
|
||||||
|
messages = [
|
||||||
|
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
|
||||||
|
{"role": "user", "content": "Who are you?"},
|
||||||
|
]
|
||||||
|
|
||||||
|
prompts = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
|
||||||
|
|
||||||
|
llm = LLM(model=model_id, tensor_parallel_size=number_gpus)
|
||||||
|
|
||||||
|
outputs = llm.generate(prompts, sampling_params)
|
||||||
|
|
||||||
|
generated_text = outputs[0].outputs[0].text
|
||||||
|
print(generated_text)
|
||||||
|
```
|
||||||
|
|
||||||
|
vLLM aslo supports OpenAI-compatible serving. See the [documentation](https://docs.vllm.ai/en/latest/) for more details.
|
||||||
|
|
||||||
|
## Creation
|
||||||
|
|
||||||
|
This model was created by applying [LLM Compressor with calibration samples from UltraChat](https://github.com/vllm-project/llm-compressor/blob/main/examples/quantization_w4a4_fp4/llama3_example.py), as presented in the code snipet below.
|
||||||
|
|
||||||
|
<details>
|
||||||
|
|
||||||
|
```python
|
||||||
|
from datasets import load_dataset
|
||||||
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||||
|
|
||||||
|
from llmcompressor import oneshot
|
||||||
|
from llmcompressor.modifiers.quantization import QuantizationModifier
|
||||||
|
from llmcompressor.modifiers.smoothquant import SmoothQuantModifier
|
||||||
|
from llmcompressor.utils import dispatch_for_generation
|
||||||
|
|
||||||
|
MODEL_ID = "meta-llama/Meta-Llama-3-8B-Instruct"
|
||||||
|
|
||||||
|
# Load model.
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(MODEL_ID, torch_dtype="auto")
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
|
||||||
|
|
||||||
|
DATASET_ID = "HuggingFaceH4/ultrachat_200k"
|
||||||
|
DATASET_SPLIT = "train_sft"
|
||||||
|
|
||||||
|
# Select number of samples. 512 samples is a good place to start.
|
||||||
|
# Increasing the number of samples can improve accuracy.
|
||||||
|
NUM_CALIBRATION_SAMPLES = 512
|
||||||
|
MAX_SEQUENCE_LENGTH = 2048
|
||||||
|
|
||||||
|
# Load dataset and preprocess.
|
||||||
|
ds = load_dataset(DATASET_ID, split=f"{DATASET_SPLIT}[:{NUM_CALIBRATION_SAMPLES}]")
|
||||||
|
ds = ds.shuffle(seed=42)
|
||||||
|
|
||||||
|
def preprocess(example):
|
||||||
|
return {
|
||||||
|
"text": tokenizer.apply_chat_template(
|
||||||
|
example["messages"],
|
||||||
|
tokenize=False,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
ds = ds.map(preprocess)
|
||||||
|
|
||||||
|
# Tokenize inputs.
|
||||||
|
def tokenize(sample):
|
||||||
|
return tokenizer(
|
||||||
|
sample["text"],
|
||||||
|
padding=False,
|
||||||
|
max_length=MAX_SEQUENCE_LENGTH,
|
||||||
|
truncation=True,
|
||||||
|
add_special_tokens=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
ds = ds.map(tokenize, remove_columns=ds.column_names)
|
||||||
|
|
||||||
|
# Configure the quantization algorithm and scheme.
|
||||||
|
# In this case, we:
|
||||||
|
# * quantize the weights to fp4 with per group 16 via ptq
|
||||||
|
# * calibrate a global_scale for activations, which will be used to
|
||||||
|
# quantize activations to fp4 on the fly
|
||||||
|
smoothing_strength = 0.5
|
||||||
|
recipe = [
|
||||||
|
SmoothQuantModifier(smoothing_strength=smoothing_strength),
|
||||||
|
QuantizationModifier(
|
||||||
|
ignore=["re:.*lm_head.*"],
|
||||||
|
config_groups={
|
||||||
|
"group_0": {
|
||||||
|
"targets": ["Linear"],
|
||||||
|
"weights": {
|
||||||
|
"num_bits": 4,
|
||||||
|
"type": "float",
|
||||||
|
"strategy": "tensor_group",
|
||||||
|
"group_size": 16,
|
||||||
|
"symmetric": True,
|
||||||
|
"observer": "mse",
|
||||||
|
},
|
||||||
|
"input_activations": {
|
||||||
|
"num_bits": 4,
|
||||||
|
"type": "float",
|
||||||
|
"strategy": "tensor_group",
|
||||||
|
"group_size": 16,
|
||||||
|
"symmetric": True,
|
||||||
|
"dynamic": "local",
|
||||||
|
"observer": "minmax",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
# Save to disk in compressed-tensors format.
|
||||||
|
SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-NVFP4"
|
||||||
|
|
||||||
|
# Apply quantization.
|
||||||
|
oneshot(
|
||||||
|
model=model,
|
||||||
|
dataset=ds,
|
||||||
|
recipe=recipe,
|
||||||
|
max_seq_length=MAX_SEQUENCE_LENGTH,
|
||||||
|
num_calibration_samples=NUM_CALIBRATION_SAMPLES,
|
||||||
|
output_dir=SAVE_DIR,
|
||||||
|
)
|
||||||
|
|
||||||
|
print("\n\n")
|
||||||
|
print("========== SAMPLE GENERATION ==============")
|
||||||
|
dispatch_for_generation(model)
|
||||||
|
input_ids = tokenizer("Hello my name is", return_tensors="pt").input_ids.to("cuda")
|
||||||
|
output = model.generate(input_ids, max_new_tokens=100)
|
||||||
|
print(tokenizer.decode(output[0]))
|
||||||
|
print("==========================================\n\n")
|
||||||
|
|
||||||
|
model.save_pretrained(SAVE_DIR, save_compressed=True)
|
||||||
|
tokenizer.save_pretrained(SAVE_DIR)
|
||||||
|
|
||||||
|
```
|
||||||
|
</details>
|
||||||
|
|
||||||
|
## Evaluation
|
||||||
|
|
||||||
|
This model was evaluated on the well-known OpenLLM v1, OpenLLM v2 and HumanEval_64 benchmarks. All evaluations were conducted using [lm-evaluation-harness](https://github.com/neuralmagic/lm-evaluation-harness).
|
||||||
|
|
||||||
|
### Accuracy
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Category</th>
|
||||||
|
<th>Metric</th>
|
||||||
|
<th>Meta-Llama-3.1-8B-Instruct</th>
|
||||||
|
<th>Llama-3.1-8B-Instruct-NVFP4 (this model)</th>
|
||||||
|
<th>Recovery</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr>
|
||||||
|
<td rowspan="8"><b>OpenLLM V1</b></td>
|
||||||
|
<td>arc_challenge_llama</td><td>83.35</td><td>82.32</td><td>98.76</td></tr>
|
||||||
|
<tr>
|
||||||
|
<td>gsm8k_llama</td><td>78.17</td><td>79.30</td><td>101.45</td></tr>
|
||||||
|
<tr><td>hellaswag</td><td>78.43</td><td>78.01</td><td>99.46</td></tr>
|
||||||
|
<tr><td>mmlu_llama</td><td>69.37</td><td>65.95</td><td>95.07</td></tr>
|
||||||
|
<tr><td>mmlu_cot_llama</td><td>72.86</td><td>68.60</td><td>94.15</td></tr>
|
||||||
|
<tr><td>truthfulqa_mc2</td><td>55.09</td><td>52.95</td><td>96.12</td></tr>
|
||||||
|
<tr><td>winogrande</td><td>75.77</td><td>74.03</td><td>97.70</td></tr>
|
||||||
|
<tr><td><b>Average</b></td><td><b>73.29</b></td><td><b>71.59</b></td><td><b>97.68</b></td></tr>
|
||||||
|
</tbody></table>
|
||||||
|
|
||||||
|
<table>
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Category</th>
|
||||||
|
<th>Metric</th>
|
||||||
|
<th>Meta-Llama-3.1-8B-Instruct</th>
|
||||||
|
<th>RedHatAI/Llama-3.1-8B-Instruct-NVFP4 (this model)</th>
|
||||||
|
<th>Recovery (%)</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr>
|
||||||
|
<td rowspan="7"><b>OpenLLM V2</b></td>
|
||||||
|
<td>MMLU-Pro (5-shot)</td>
|
||||||
|
<td>37.69</td>
|
||||||
|
<td>34.43</td>
|
||||||
|
<td>91.35</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td>IFEval (0-shot)</td>
|
||||||
|
<td>80.94</td>
|
||||||
|
<td>79.98</td>
|
||||||
|
<td>98.81</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td>BBH (3-shot)</td>
|
||||||
|
<td>50.76</td>
|
||||||
|
<td>48.62</td>
|
||||||
|
<td>95.78</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td>Math-|v|-5 (4-shot)</td>
|
||||||
|
<td>22.05</td>
|
||||||
|
<td>14.65</td>
|
||||||
|
<td>66.44</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td>GPQA (0-shot)</td>
|
||||||
|
<td>28.44</td>
|
||||||
|
<td>27.94</td>
|
||||||
|
<td>98.24</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td>MuSR (0-shot)</td>
|
||||||
|
<td>38.10</td>
|
||||||
|
<td>37.83</td>
|
||||||
|
<td>99.29</td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td><b>Average</b></td>
|
||||||
|
<td><b>43.00</b></td>
|
||||||
|
<td><b>40.58</b></td>
|
||||||
|
<td><b>94.37</b></td>
|
||||||
|
</tr>
|
||||||
|
<tr>
|
||||||
|
<td><b>Coding</b></td>
|
||||||
|
<td>HumanEval_64 pass@2</td>
|
||||||
|
<td>71.90</td>
|
||||||
|
<td>71.44</td>
|
||||||
|
<td>99.36</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Reproduction
|
||||||
|
|
||||||
|
The results were obtained using the following commands:
|
||||||
|
|
||||||
|
<details>
|
||||||
|
|
||||||
|
#### MMLU_LLAMA
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks mmlu_llama \
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### MMLU_COT_LLAMA
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks mmlu_cot_llama \
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### ARC-Challenge
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks arc_challenge_llama \
|
||||||
|
--apply_chat_template \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### GSM-8K
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks gsm8k_llama \
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Hellaswag
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks hellaswag \
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Winogrande
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks winogrande \
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### TruthfulQA
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True \
|
||||||
|
--tasks truthfulqa \
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### OpenLLM v2
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True\
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--tasks leaderboard \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
#### HumanEval and HumanEval_64
|
||||||
|
```
|
||||||
|
lm_eval \
|
||||||
|
--model vllm \
|
||||||
|
--model_args pretrained="RedHatAI/Meta-Llama-3.1-8B-Instruct-NVFP4",dtype=auto,max_model_len=4096,tensor_parallel_size=1,enable_chunked_prefill=True,enforce_eager=True\
|
||||||
|
--apply_chat_template \
|
||||||
|
--fewshot_as_multiturn \
|
||||||
|
--tasks humaneval_64_instruct \
|
||||||
|
--batch_size auto
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
109
chat_template.jinja
Normal file
109
chat_template.jinja
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
{{- bos_token }}
|
||||||
|
{%- if custom_tools is defined %}
|
||||||
|
{%- set tools = custom_tools %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if not tools_in_user_message is defined %}
|
||||||
|
{%- set tools_in_user_message = true %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if not date_string is defined %}
|
||||||
|
{%- set date_string = "26 Jul 2024" %}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if not tools is defined %}
|
||||||
|
{%- set tools = none %}
|
||||||
|
{%- endif %}
|
||||||
|
|
||||||
|
{#- This block extracts the system message, so we can slot it into the right place. #}
|
||||||
|
{%- if messages[0]['role'] == 'system' %}
|
||||||
|
{%- set system_message = messages[0]['content']|trim %}
|
||||||
|
{%- set messages = messages[1:] %}
|
||||||
|
{%- else %}
|
||||||
|
{%- set system_message = "" %}
|
||||||
|
{%- endif %}
|
||||||
|
|
||||||
|
{#- System message + builtin tools #}
|
||||||
|
{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
|
||||||
|
{%- if builtin_tools is defined or tools is not none %}
|
||||||
|
{{- "Environment: ipython\n" }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if builtin_tools is defined %}
|
||||||
|
{{- "Tools: " + builtin_tools | reject('equalto', 'code_interpreter') | join(", ") + "\n\n"}}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "Cutting Knowledge Date: December 2023\n" }}
|
||||||
|
{{- "Today Date: " + date_string + "\n\n" }}
|
||||||
|
{%- if tools is not none and not tools_in_user_message %}
|
||||||
|
{{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
|
||||||
|
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||||
|
{{- "Do not use variables.\n\n" }}
|
||||||
|
{%- for t in tools %}
|
||||||
|
{{- t | tojson(indent=4) }}
|
||||||
|
{{- "\n\n" }}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- endif %}
|
||||||
|
{{- system_message }}
|
||||||
|
{{- "<|eot_id|>" }}
|
||||||
|
|
||||||
|
{#- Custom tools are passed in a user message with some extra guidance #}
|
||||||
|
{%- if tools_in_user_message and not tools is none %}
|
||||||
|
{#- Extract the first user message so we can plug it in here #}
|
||||||
|
{%- if messages | length != 0 %}
|
||||||
|
{%- set first_user_message = messages[0]['content']|trim %}
|
||||||
|
{%- set messages = messages[1:] %}
|
||||||
|
{%- else %}
|
||||||
|
{{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
|
||||||
|
{{- "Given the following functions, please respond with a JSON for a function call " }}
|
||||||
|
{{- "with its proper arguments that best answers the given prompt.\n\n" }}
|
||||||
|
{{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
|
||||||
|
{{- "Do not use variables.\n\n" }}
|
||||||
|
{%- for t in tools %}
|
||||||
|
{{- t | tojson(indent=4) }}
|
||||||
|
{{- "\n\n" }}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- first_user_message + "<|eot_id|>"}}
|
||||||
|
{%- endif %}
|
||||||
|
|
||||||
|
{%- for message in messages %}
|
||||||
|
{%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
|
||||||
|
{{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
|
||||||
|
{%- elif 'tool_calls' in message %}
|
||||||
|
{%- if not message.tool_calls|length == 1 %}
|
||||||
|
{{- raise_exception("This model only supports single tool-calls at once!") }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- set tool_call = message.tool_calls[0].function %}
|
||||||
|
{%- if builtin_tools is defined and tool_call.name in builtin_tools %}
|
||||||
|
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
|
||||||
|
{{- "<|python_tag|>" + tool_call.name + ".call(" }}
|
||||||
|
{%- for arg_name, arg_val in tool_call.arguments | items %}
|
||||||
|
{{- arg_name + '="' + arg_val + '"' }}
|
||||||
|
{%- if not loop.last %}
|
||||||
|
{{- ", " }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{{- ")" }}
|
||||||
|
{%- else %}
|
||||||
|
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
|
||||||
|
{{- '{"name": "' + tool_call.name + '", ' }}
|
||||||
|
{{- '"parameters": ' }}
|
||||||
|
{{- tool_call.arguments | tojson }}
|
||||||
|
{{- "}" }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- if builtin_tools is defined %}
|
||||||
|
{#- This means we're in ipython mode #}
|
||||||
|
{{- "<|eom_id|>" }}
|
||||||
|
{%- else %}
|
||||||
|
{{- "<|eot_id|>" }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- elif message.role == "tool" or message.role == "ipython" %}
|
||||||
|
{{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
|
||||||
|
{%- if message.content is mapping or message.content is iterable %}
|
||||||
|
{{- message.content | tojson }}
|
||||||
|
{%- else %}
|
||||||
|
{{- message.content }}
|
||||||
|
{%- endif %}
|
||||||
|
{{- "<|eot_id|>" }}
|
||||||
|
{%- endif %}
|
||||||
|
{%- endfor %}
|
||||||
|
{%- if add_generation_prompt %}
|
||||||
|
{{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
|
||||||
|
{%- endif %}
|
||||||
85
config.json
Normal file
85
config.json
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
{
|
||||||
|
"architectures": [
|
||||||
|
"LlamaForCausalLM"
|
||||||
|
],
|
||||||
|
"attention_bias": false,
|
||||||
|
"attention_dropout": 0.0,
|
||||||
|
"bos_token_id": 128000,
|
||||||
|
"dtype": "bfloat16",
|
||||||
|
"eos_token_id": [
|
||||||
|
128001,
|
||||||
|
128008,
|
||||||
|
128009
|
||||||
|
],
|
||||||
|
"head_dim": 128,
|
||||||
|
"hidden_act": "silu",
|
||||||
|
"hidden_size": 4096,
|
||||||
|
"initializer_range": 0.02,
|
||||||
|
"intermediate_size": 14336,
|
||||||
|
"max_position_embeddings": 131072,
|
||||||
|
"mlp_bias": false,
|
||||||
|
"model_type": "llama",
|
||||||
|
"num_attention_heads": 32,
|
||||||
|
"num_hidden_layers": 32,
|
||||||
|
"num_key_value_heads": 8,
|
||||||
|
"pretraining_tp": 1,
|
||||||
|
"quantization_config": {
|
||||||
|
"config_groups": {
|
||||||
|
"group_0": {
|
||||||
|
"format": "nvfp4-pack-quantized",
|
||||||
|
"input_activations": {
|
||||||
|
"actorder": null,
|
||||||
|
"block_structure": null,
|
||||||
|
"dynamic": "local",
|
||||||
|
"group_size": 16,
|
||||||
|
"num_bits": 4,
|
||||||
|
"observer": "minmax",
|
||||||
|
"observer_kwargs": {},
|
||||||
|
"strategy": "tensor_group",
|
||||||
|
"symmetric": true,
|
||||||
|
"type": "float"
|
||||||
|
},
|
||||||
|
"output_activations": null,
|
||||||
|
"targets": [
|
||||||
|
"Linear"
|
||||||
|
],
|
||||||
|
"weights": {
|
||||||
|
"actorder": null,
|
||||||
|
"block_structure": null,
|
||||||
|
"dynamic": false,
|
||||||
|
"group_size": 16,
|
||||||
|
"num_bits": 4,
|
||||||
|
"observer": "mse",
|
||||||
|
"observer_kwargs": {},
|
||||||
|
"strategy": "tensor_group",
|
||||||
|
"symmetric": true,
|
||||||
|
"type": "float"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"format": "nvfp4-pack-quantized",
|
||||||
|
"global_compression_ratio": null,
|
||||||
|
"ignore": [
|
||||||
|
"lm_head"
|
||||||
|
],
|
||||||
|
"kv_cache_scheme": null,
|
||||||
|
"quant_method": "compressed-tensors",
|
||||||
|
"quantization_status": "compressed",
|
||||||
|
"sparsity_config": {},
|
||||||
|
"transform_config": {},
|
||||||
|
"version": "0.11.1.a20250917"
|
||||||
|
},
|
||||||
|
"rms_norm_eps": 1e-05,
|
||||||
|
"rope_scaling": {
|
||||||
|
"factor": 8.0,
|
||||||
|
"high_freq_factor": 4.0,
|
||||||
|
"low_freq_factor": 1.0,
|
||||||
|
"original_max_position_embeddings": 8192,
|
||||||
|
"rope_type": "llama3"
|
||||||
|
},
|
||||||
|
"rope_theta": 500000.0,
|
||||||
|
"tie_word_embeddings": false,
|
||||||
|
"transformers_version": "4.56.1",
|
||||||
|
"use_cache": true,
|
||||||
|
"vocab_size": 128256
|
||||||
|
}
|
||||||
1
configuration.json
Normal file
1
configuration.json
Normal file
@@ -0,0 +1 @@
|
|||||||
|
{"framework": "pytorch", "task": "text-generation", "allow_remote": true}
|
||||||
12
generation_config.json
Normal file
12
generation_config.json
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"bos_token_id": 128000,
|
||||||
|
"do_sample": true,
|
||||||
|
"eos_token_id": [
|
||||||
|
128001,
|
||||||
|
128008,
|
||||||
|
128009
|
||||||
|
],
|
||||||
|
"temperature": 0.6,
|
||||||
|
"top_p": 0.9,
|
||||||
|
"transformers_version": "4.56.1"
|
||||||
|
}
|
||||||
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a0755a0af031703818d6ff722f85976dbeed2c94d65c663c1ae30decfa72a03d
|
||||||
|
size 4977186360
|
||||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5321ca97dd0968a3e2b676b3a42728e9f5327d516bcf0addd4e9cd61c068a1ad
|
||||||
|
size 1050673280
|
||||||
971
model.safetensors.index.json
Normal file
971
model.safetensors.index.json
Normal file
@@ -0,0 +1,971 @@
|
|||||||
|
{
|
||||||
|
"metadata": {
|
||||||
|
"total_parameters": 4976808384,
|
||||||
|
"total_size": 6027749120
|
||||||
|
},
|
||||||
|
"weight_map": {
|
||||||
|
"lm_head.weight": "model-00002-of-00002.safetensors",
|
||||||
|
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.0.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.1.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.10.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.11.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.12.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.13.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.14.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.15.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.16.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.17.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.18.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.19.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.2.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.20.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.21.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.22.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.23.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.24.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.25.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.26.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.27.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.28.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.29.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.3.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.30.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.31.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.4.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.5.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.6.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.7.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.8.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.down_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.down_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.down_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.down_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.gate_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.gate_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.gate_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.gate_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.up_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.up_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.up_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.mlp.up_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.k_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.o_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.o_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.o_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.o_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.q_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.input_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.weight_global_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.weight_packed": "model-00001-of-00002.safetensors",
|
||||||
|
"model.layers.9.self_attn.v_proj.weight_scale": "model-00001-of-00002.safetensors",
|
||||||
|
"model.norm.weight": "model-00001-of-00002.safetensors"
|
||||||
|
}
|
||||||
|
}
|
||||||
42
recipe.yaml
Normal file
42
recipe.yaml
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
default_stage:
|
||||||
|
default_modifiers:
|
||||||
|
SmoothQuantModifier:
|
||||||
|
smoothing_strength: 0.5
|
||||||
|
mappings:
|
||||||
|
- !!python/tuple
|
||||||
|
- ['re:.*q_proj', 're:.*k_proj', 're:.*v_proj']
|
||||||
|
- re:.*input_layernorm
|
||||||
|
- !!python/tuple
|
||||||
|
- ['re:.*gate_proj', 're:.*up_proj']
|
||||||
|
- re:.*post_attention_layernorm
|
||||||
|
ignore: []
|
||||||
|
QuantizationModifier:
|
||||||
|
config_groups:
|
||||||
|
group_0:
|
||||||
|
targets: [Linear]
|
||||||
|
weights:
|
||||||
|
num_bits: 4
|
||||||
|
type: float
|
||||||
|
symmetric: true
|
||||||
|
group_size: 16
|
||||||
|
strategy: tensor_group
|
||||||
|
block_structure: null
|
||||||
|
dynamic: false
|
||||||
|
actorder: null
|
||||||
|
observer: mse
|
||||||
|
observer_kwargs: {}
|
||||||
|
input_activations:
|
||||||
|
num_bits: 4
|
||||||
|
type: float
|
||||||
|
symmetric: true
|
||||||
|
group_size: 16
|
||||||
|
strategy: tensor_group
|
||||||
|
block_structure: null
|
||||||
|
dynamic: local
|
||||||
|
actorder: null
|
||||||
|
observer: minmax
|
||||||
|
observer_kwargs: {}
|
||||||
|
output_activations: null
|
||||||
|
format: null
|
||||||
|
targets: [Linear]
|
||||||
|
ignore: ['re:.*lm_head.*']
|
||||||
16
special_tokens_map.json
Normal file
16
special_tokens_map.json
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
{
|
||||||
|
"bos_token": {
|
||||||
|
"content": "<|begin_of_text|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
},
|
||||||
|
"eos_token": {
|
||||||
|
"content": "<|eot_id|>",
|
||||||
|
"lstrip": false,
|
||||||
|
"normalized": false,
|
||||||
|
"rstrip": false,
|
||||||
|
"single_word": false
|
||||||
|
}
|
||||||
|
}
|
||||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
2062
tokenizer_config.json
Normal file
2062
tokenizer_config.json
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user