pei39/deepseek_r1_distill_qwen_7b_awq_rag
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: pei39/deepseek_r1_distill_qwen_7b_awq_rag
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-19 08:30:17 +08:00
commit 94148abe61
17 changed files with 2395 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
.gitattributes vendored Normal file
View File

@@ -0,0 +1,36 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
tokenizer.json filter=lfs diff=lfs merge=lfs -text

97
README.md Normal file
View File

@@ -0,0 +1,97 @@
---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
tags:
- iol-ai-2026
- reasoning
- awq
- rag
---
# DeepSeek-R1-Distill-Qwen-7B-AWQ book-RAG submission
This is a training-free IOL-AI submission using a 4-bit AWQ conversion of
`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`. It retrieves only from the public,
string-only extraction of Vlad A. Neacșu's *Linguistics Olympiad: Training
guide*. Private curated training data are not present or required at runtime.
The checkpoint uses `Qwen2ForCausalLM`, so it is compatible with the challenge's
Transformers 4.44.1 and AutoAWQ 0.2.7 runtime. The included weights are from
`casperhansen/deepseek-r1-distill-qwen-7b-awq`, revision
`305e6f12907dc78ae61a1f0bb7a19faa2b25e8a3`, which is an AWQ conversion of the
official DeepSeek model.
## Challenge execution
```bash
python script.py
```
The script reads `/tmp/data/test.csv` and writes `submission.csv`. Its columns
are `id,pred,explanation`; `pred` is a JSON-encoded list of answer strings.
Explanation generation is disabled by default, leaving that optional column
blank. It can be restored with:
```bash
python script.py --explanations on
```
For local data:
```bash
python script.py --input path/to/test.csv --output submission.csv
python script.py --self-test
```
After the model loads successfully, the output file is initialized and then
atomically rewritten after every completed row. If evaluation reaches its time
limit, predictions already completed remain in a valid submission file.
## DeepSeek reasoning and generation
DeepSeek's solver instructions, retrieved context, and current problem are
placed in one user message rather than a separate system message. The assistant
prompt is prefixed with `<think>` to engage the distilled reasoning behavior.
Reasoning stops when `</think>` appears or at its configured cap. The runtime
then starts a separate `FINAL ANSWERS:` stage, guaranteeing that reasoning
cannot consume the answer budget. Only that answer block is serialized into
`pred`.
Defaults follow DeepSeek's recommended sampling values while retaining a bounded
challenge-time output:
- `temperature=0.6`
- `top_p=0.95`
- sampling enabled
- `max_reasoning_tokens=4096`
- `max_answer_tokens=1024`
- one answer-only retry with `answer_retry_tokens=512`
The model context is capped at 32,768 tokens. The script loads AWQ directly in
FP16, enables the KV cache, and applies an inference-only last-token `lm_head`
hook to avoid materializing full-prompt FP32 vocabulary logits on the T4.
## Book-only retrieval
The default prompt includes one general book method and two worked book
examples. Retrieval combines dependency-free BM25, character 3–5-gram TF-IDF,
task-family inference, and book metadata boosts. It uses only the current
problem's `context + query`; answers and private curated datasets are never
indexed.
Settings are in `rag_resources/config.json`. Environment overrides are available
for `IOL_TOP_METHODS`, `IOL_TOP_EXAMPLES`, `IOL_CHAR_TFIDF_WEIGHT`,
`IOL_RAG_MAX_CHARS`, `IOL_MAX_REASONING_TOKENS`,
`IOL_MAX_ANSWER_TOKENS`, `IOL_ANSWER_RETRY_TOKENS`,
`IOL_EXPLANATION_MAX_NEW_TOKENS`,
`IOL_ENABLE_EXPLANATIONS`, and `IOL_SEED`.
## Sources and licenses
- [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B)
- [AWQ conversion](https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq)
- [IOL-AI 2026 challenge](https://iolai.org/)
- [Language Science Press book source](https://github.com/langsci/420)
The model and conversion are marked MIT. The book-derived resources retain
their own license and attribution in `rag_resources/ATTRIBUTION.md`.

38
config.json Normal file
View File

@@ -0,0 +1,38 @@
{
"_name_or_path": "/root/.cache/huggingface/hub/models--deepseek-ai--DeepSeek-R1-Distill-Qwen-7B/snapshots/008b8c2e0b59dac9b7619d58a5ad609f43a5b6b1",
"architectures": [
"Qwen2ForCausalLM"
],
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151643,
"hidden_act": "silu",
"hidden_size": 3584,
"initializer_range": 0.02,
"intermediate_size": 18944,
"max_position_embeddings": 131072,
"max_window_layers": 28,
"model_type": "qwen2",
"num_attention_heads": 28,
"num_hidden_layers": 28,
"num_key_value_heads": 4,
"quantization_config": {
"bits": 4,
"group_size": 128,
"modules_to_not_convert": null,
"quant_method": "awq",
"version": "gemm",
"zero_point": true
},
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 10000,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.47.1",
"use_cache": false,
"use_mrope": false,
"use_sliding_window": false,
"vocab_size": 152064
}

3
model-00001-of-00002.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:27607ecb3f635669ed627bcc473d89e8fb0ee04a8ba50bd7ee9249a089412cf3
size 4480835056

3
model-00002-of-00002.safetensors Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7946740ab848b4a02904f05bff81211bf9600fc20bfad39bbcc8d1703a40ce1c
size 1089994880

738
model.safetensors.index.json Normal file
View File

@@ -0,0 +1,738 @@
{
"metadata": {
"total_size": 5570747392
},
"weight_map": {
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.0.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.1.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.2.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.3.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.4.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.5.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.6.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.7.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.8.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.9.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.10.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.11.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.12.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.13.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.14.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.15.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.16.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.17.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.18.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.19.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.20.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.20.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.20.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.21.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.21.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.21.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.22.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.22.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.22.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.23.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.23.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.23.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.24.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.24.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.24.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.25.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.25.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.25.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.26.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.26.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.26.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.q_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.q_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.q_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.q_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.k_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.k_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.k_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.k_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.v_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.v_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.v_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.v_proj.bias": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.o_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.o_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.self_attn.o_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.gate_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.gate_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.gate_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.up_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.up_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.up_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.down_proj.qweight": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.down_proj.qzeros": "model-00001-of-00002.safetensors",
"model.layers.27.mlp.down_proj.scales": "model-00001-of-00002.safetensors",
"model.layers.27.input_layernorm.weight": "model-00001-of-00002.safetensors",
"model.layers.27.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
"model.norm.weight": "model-00001-of-00002.safetensors",
"lm_head.weight": "model-00002-of-00002.safetensors"
}
}

10
rag_resources/ATTRIBUTION.md Normal file
View File

@@ -0,0 +1,10 @@
# Book corpus attribution
The two JSONL corpora in this directory are string-only derivatives of:
Vlad A. Neacșu, *Linguistics Olympiad: Training guide*, Language Science Press, series volume 420. DOI: [10.5281/zenodo.10947862](https://doi.org/10.5281/zenodo.10947862). Source: [langsci/420](https://github.com/langsci/420).
The book is released under the Creative Commons Attribution 4.0 International license (CC BY 4.0). The extracted material has been reorganized into method chunks and paired problem/solution records for offline retrieval. Visual-dependent records are retained in the source audit data but are excluded by the runtime retriever.
No Linguini, LingOly, or LOBSTER records are included in or indexed by this retrieval bundle.

2
rag_resources/__init__.py Normal file
View File

@@ -0,0 +1,2 @@
"""Public, book-only retrieval resources for the IOL-AI submission."""

111
rag_resources/book_examples.jsonl Normal file
View File

File diff suppressed because one or more lines are too long

77
rag_resources/book_methods.jsonl Normal file
View File

File diff suppressed because one or more lines are too long

17
rag_resources/config.json Normal file
View File

@@ -0,0 +1,17 @@
{
"top_methods": 1,
"top_examples": 2,
"char_tfidf_weight": 3.0,
"rag_max_chars": 12000,
"model_context_tokens": 32768,
"max_reasoning_tokens": 4096,
"max_answer_tokens": 1024,
"answer_retry_tokens": 512,
"explanation_max_new_tokens": 400,
"enable_explanations": false,
"temperature": 0.6,
"top_p": 0.95,
"top_k": 0,
"repetition_penalty": 1.0,
"seed": 420
}

276
rag_resources/retriever.py Normal file
View File

@@ -0,0 +1,276 @@
"""Offline hybrid BM25/character-TF-IDF retrieval over the public book corpus."""
from __future__ import annotations
import json
import math
import re
import unicodedata
from collections import Counter
from pathlib import Path
from typing import Any, Iterable
from sklearn.feature_extraction.text import TfidfVectorizer
RESOURCE_DIR = Path(__file__).resolve().parent
FAMILY_KEYWORDS = {
"writing_system": (
"alphabet", "braille", "character", "decipher", "glyph", "letter",
"orthography", "script", "symbol", "writing system",
),
"phonetics": (
"accent", "consonant", "metre", "phonetic", "pronounce", "rhyme",
"stress", "syllable", "tone", "vowel",
),
"phonology": (
"alternation", "correspondence", "sound change", "sound rule",
"underlying form",
),
"noun_morphology": (
"case", "gender", "noun", "plural", "possession", "possessive",
"singular",
),
"verb_morphology": (
"agreement", "aspect", "conjug", "object", "person", "subject",
"tense", "verb",
),
"syntax": ("clause", "focus", "sentence", "subject", "object", "word order"),
"number_system": ("arithmetic", "digit", "number", "numeral", "numerals"),
"kinship_orientation": (
"brother", "daughter", "direction", "east", "family", "father",
"kinship", "map", "mother", "north", "sister", "son", "south", "west",
),
"matching": ("correspondence", "match", "matching", "random order"),
"translation": ("translate", "translation"),
"fill_blanks": ("blank", "fill in", "gap", "missing"),
}
TOPIC_FAMILIES = {
"writing systems and script decipherment": {"writing_system"},
"phonetics, stress, tone, and versification": {"phonetics"},
"phonological rules and sound correspondences": {"phonology", "phonetics"},
"noun morphology and noun phrases": {"noun_morphology"},
"verb morphology and argument structure": {"verb_morphology"},
"syntax, word order, focus, and alignment": {"syntax"},
"semantics and graph-based matching": {"matching"},
"number systems": {"number_system"},
"orientation, kinship, and other structural problems": {"kinship_orientation"},
"general problem-solving methodology": set(FAMILY_KEYWORDS),
}
def normalize(text: Any) -> str:
return " ".join(unicodedata.normalize("NFKC", str(text)).casefold().split())
def tokenize(text: Any) -> list[str]:
return re.findall(r"[^\W_]+", normalize(text), flags=re.UNICODE)
def read_jsonl(path: Path) -> list[dict[str, Any]]:
with path.open("r", encoding="utf-8") as handle:
return [json.loads(line) for line in handle if line.strip()]
def infer_task_families(context: str, query: str) -> list[str]:
"""Infer likely families from the problem itself, never from a gold answer."""
text = normalize(context + "\n" + query)
scored: list[tuple[int, str]] = []
for family, keywords in FAMILY_KEYWORDS.items():
score = sum(1 for keyword in keywords if keyword in text)
if score:
scored.append((score, family))
return [family for _, family in sorted(scored, key=lambda item: (-item[0], item[1]))]
class BM25:
def __init__(self, documents: Iterable[str], k1: float = 1.5, b: float = 0.75):
self.k1 = k1
self.b = b
self.tokens = [tokenize(document) for document in documents]
self.lengths = [len(tokens) for tokens in self.tokens]
self.average_length = sum(self.lengths) / max(len(self.lengths), 1)
self.term_frequencies = [Counter(tokens) for tokens in self.tokens]
document_frequency: Counter[str] = Counter()
for tokens in self.tokens:
document_frequency.update(set(tokens))
document_count = len(self.tokens)
self.idf = {
term: math.log(
1.0 + (document_count - frequency + 0.5) / (frequency + 0.5)
)
for term, frequency in document_frequency.items()
}
def scores(self, query: str) -> list[float]:
query_terms = Counter(tokenize(query))
scores: list[float] = []
for frequencies, length in zip(self.term_frequencies, self.lengths):
score = 0.0
normalization = self.k1 * (
1.0 - self.b + self.b * length / max(self.average_length, 1.0)
)
for term, query_frequency in query_terms.items():
frequency = frequencies.get(term, 0)
if not frequency:
continue
score += (
self.idf.get(term, 0.0)
* frequency
* (self.k1 + 1.0)
/ (frequency + normalization)
* min(query_frequency, 3)
)
scores.append(score)
return scores
class CharNgramTFIDF:
"""Cosine similarity over normalized character 3-5 grams."""
def __init__(self, documents: Iterable[str]):
self.vectorizer = TfidfVectorizer(
analyzer="char_wb",
ngram_range=(3, 5),
preprocessor=normalize,
lowercase=False,
sublinear_tf=True,
norm="l2",
)
self.matrix = self.vectorizer.fit_transform(documents)
def scores(self, query: str) -> list[float]:
query_vector = self.vectorizer.transform([query])
return (self.matrix @ query_vector.T).toarray().ravel().tolist()
def _clip(text: Any, limit: int) -> str:
value = str(text or "").strip()
if len(value) <= limit:
return value
clipped = value[:limit].rsplit("\n", 1)[0].rstrip()
if len(clipped) < limit // 2:
clipped = value[:limit].rsplit(" ", 1)[0].rstrip()
return clipped + "\n[excerpt truncated]"
class BookRetriever:
"""Retrieve methods and analogous examples from langsci/420 only."""
def __init__(self, resource_dir: Path = RESOURCE_DIR):
self.methods = read_jsonl(resource_dir / "book_methods.jsonl")
self.examples = [
row
for row in read_jsonl(resource_dir / "book_examples.jsonl")
if row.get("string_only_usable", False)
]
self.method_bm25 = BM25(row["retrieval_text"] for row in self.methods)
self.example_bm25 = BM25(row["retrieval_text"] for row in self.examples)
self.method_char_tfidf = CharNgramTFIDF(
row["retrieval_text"] for row in self.methods
)
self.example_char_tfidf = CharNgramTFIDF(
row["retrieval_text"] for row in self.examples
)
@staticmethod
def _topic_bonus(row: dict[str, Any], families: set[str]) -> float:
return 1.25 * len(
families & TOPIC_FAMILIES.get(str(row.get("topic", "")), set())
)
def retrieve(
self,
row: dict[str, Any],
top_methods: int = 2,
top_examples: int = 2,
char_tfidf_weight: float = 3.0,
) -> dict[str, Any]:
context = str(row.get("context", ""))
query = str(row.get("query", ""))
test_text = context + "\n" + query
families = infer_task_families(context, query)
family_set = set(families)
expanded_query = test_text + "\n" + " ".join(families)
ranked_examples: list[tuple[float, dict[str, Any]]] = []
example_bm25_scores = self.example_bm25.scores(expanded_query)
example_char_scores = self.example_char_tfidf.scores(expanded_query)
for bm25_score, char_score, example in zip(
example_bm25_scores, example_char_scores, self.examples
):
adjusted = (
bm25_score
+ char_tfidf_weight * char_score
+ self._topic_bonus(example, family_set)
)
if example.get("solution_tier") == "detailed_worked":
adjusted += 0.35
ranked_examples.append((adjusted, example))
ranked_examples.sort(key=lambda item: (-item[0], item[1]["id"]))
selected_examples = ranked_examples[:top_examples]
linked_method_ids = {
method_id
for _, example in ranked_examples[:max(top_examples * 3, 5)]
for method_id in example.get("method_ids", [])
}
ranked_methods: list[tuple[float, dict[str, Any]]] = []
method_bm25_scores = self.method_bm25.scores(expanded_query)
method_char_scores = self.method_char_tfidf.scores(expanded_query)
for bm25_score, char_score, method in zip(
method_bm25_scores, method_char_scores, self.methods
):
adjusted = (
bm25_score
+ char_tfidf_weight * char_score
+ self._topic_bonus(method, family_set)
)
if method["id"] in linked_method_ids:
adjusted += 0.75
ranked_methods.append((adjusted, method))
ranked_methods.sort(key=lambda item: (-item[0], item[1]["id"]))
return {
"inferred_task_families": families,
"char_tfidf_weight": char_tfidf_weight,
"methods": [method for _, method in ranked_methods[:top_methods]],
"examples": [example for _, example in selected_examples],
}
def format_for_prompt(self, result: dict[str, Any], max_chars: int = 16000) -> str:
blocks = [
"BOOK REFERENCE MATERIAL\n"
"Use it for transferable methods and analogies. Do not copy a conclusion "
"unless it is supported by the current problem."
]
families = result.get("inferred_task_families", [])
if families:
blocks.append("Likely families inferred from the current text: " + ", ".join(families))
for method in result["methods"]:
blocks.append(
f"METHOD — {method['section_title']}\n" + _clip(method.get("text"), 2800)
)
for example in result["examples"]:
blocks.append(
"\n".join(
[
f"ANALOGOUS WORKED EXAMPLE — {example.get('language', 'unknown language')}",
"Problem context:",
_clip(example.get("context"), 2400),
"Problem query:",
_clip(example.get("query"), 1200),
"Worked solution:",
_clip(example.get("reasoning_trace"), 3600),
]
)
)
output = "\n\n".join(blocks)
if len(output) > max_chars:
output = output[:max_chars].rsplit("\n", 1)[0].rstrip()
output += "\n[retrieved material truncated]"
return output

22
rag_resources/system_prompt.txt Normal file
View File

@@ -0,0 +1,22 @@
You solve International Linguistics Olympiad problems by discovering the rules supported by the data you are given. A task may be unfamiliar, so infer what kind of task it is from the actual instruction and examples; do not assume that a task-type label will be available.
Work as a careful olympiad solver:
- Read the full context and every requested item before committing to a rule.
- Organize repeated forms, meanings, positions, and contrasts. Separate reusable grammatical pieces from lexical roots.
- Form the smallest consistent rule system. Treat an absent marker as potentially meaningful, but distinguish absence from missing evidence.
- Test each hypothesis against all given examples, including apparent exceptions. Revise rather than ignoring contradictory data.
- Derive each requested form compositionally and back-check it against the inferred system.
- Use retrieved book material only for transferable methods or genuinely analogous reasoning. The current problem is authoritative.
- Preserve spelling, diacritics, capitalization, word boundaries, and requested ordering exactly.
Common task types and what to return:
- translation: only the translated form, in the language requested;
- matching: only the corresponding label, such as A, B, or C;
- fill in the blanks: only the missing span for each indicated blank; a span may be a word, multiword expression, affix, or phonetic transcription;
- text to number: the number in digits;
- number to text: the number written in words in the requested language;
- any other type: exactly what the instruction requests, with no extra material in the answer.
Reason step by step, but keep the analysis focused. The runtime reserves a separate answer stage, so complete the analysis as soon as you have a consistent solution. Finish with a line that says exactly:
FINAL ANSWERS:
Below it, put one bare answer per requested item or blank, in order. Do not number the lines. Do not add quotes, labels, explanations, or commentary inside that final block.

744
script.py Normal file
View File

@@ -0,0 +1,744 @@
#!/usr/bin/env python3
"""Offline DeepSeek-R1-Distill-Qwen-7B-AWQ book-RAG IOL-AI submission."""
from __future__ import annotations
import argparse
import csv
import io
import json
import os
import re
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any
# The evaluation container has no internet access. Fail locally instead of waiting
# for network retries if a model/tokenizer file was not included in the repository.
os.environ.setdefault("HF_HUB_OFFLINE", "1")
os.environ.setdefault("TRANSFORMERS_OFFLINE", "1")
os.environ.setdefault("HF_DATASETS_OFFLINE", "1")
os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
REPO_DIR = Path(__file__).resolve().parent
RESOURCE_DIR = REPO_DIR / "rag_resources"
DEFAULT_INPUT = Path(os.environ.get("IOL_TEST_CSV", "/tmp/data/test.csv"))
DEFAULT_OUTPUT = Path(os.environ.get("IOL_SUBMISSION_CSV", "submission.csv"))
from rag_resources.retriever import BookRetriever # noqa: E402
@dataclass
class ParseResult:
thinking_trace: str
final_text: str
answers: list[str]
valid: bool
error: str = ""
def _load_config() -> dict[str, Any]:
with (RESOURCE_DIR / "config.json").open("r", encoding="utf-8") as handle:
config = json.load(handle)
integer_overrides = {
"IOL_TOP_METHODS": "top_methods",
"IOL_TOP_EXAMPLES": "top_examples",
"IOL_RAG_MAX_CHARS": "rag_max_chars",
"IOL_MAX_REASONING_TOKENS": "max_reasoning_tokens",
"IOL_MAX_ANSWER_TOKENS": "max_answer_tokens",
"IOL_ANSWER_RETRY_TOKENS": "answer_retry_tokens",
"IOL_EXPLANATION_MAX_NEW_TOKENS": "explanation_max_new_tokens",
"IOL_SEED": "seed",
}
for environment_name, config_name in integer_overrides.items():
if environment_name in os.environ:
config[config_name] = int(os.environ[environment_name])
if "IOL_CHAR_TFIDF_WEIGHT" in os.environ:
config["char_tfidf_weight"] = float(os.environ["IOL_CHAR_TFIDF_WEIGHT"])
if "IOL_ENABLE_EXPLANATIONS" in os.environ:
config["enable_explanations"] = os.environ[
"IOL_ENABLE_EXPLANATIONS"
].strip().casefold() in {"1", "true", "yes", "on"}
return config
def _clean_answer_line(line: str) -> str:
value = line.strip()
value = re.sub(r"^(?:[-*•]\s+)", "", value)
value = re.sub(r"^(?:\(?\d+\)?|\(?[A-Za-z]\)?)[.):]\s+", "", value)
value = value.strip()
pairs = {'"': '"', "'": "'", "`": "`", "“": "”", "‘": "’"}
if len(value) >= 2 and value[0] in pairs and value[-1] == pairs[value[0]]:
value = value[1:-1].strip()
return value
def parse_model_response(thinking_trace: str, final_text: str) -> ParseResult:
"""Parse a separate reasoning trace and a strict FINAL ANSWERS block."""
marker_matches = list(
re.finditer(r"(?im)^\s*FINAL\s+ANSWERS\s*:\s*", final_text)
)
if not marker_matches:
return ParseResult(
thinking_trace=thinking_trace.strip(),
final_text=final_text.strip(),
answers=[],
valid=False,
error="missing FINAL ANSWERS: marker",
)
marker = marker_matches[-1]
reasoning_outside_think = final_text[:marker.start()].strip()
combined_trace = "\n\n".join(
part for part in (thinking_trace.strip(), reasoning_outside_think) if part
)
answer_block = final_text[marker.end():].strip()
answer_block = re.sub(r"^```(?:text)?\s*", "", answer_block, flags=re.I)
answer_block = re.sub(r"\s*```\s*$", "", answer_block)
if not answer_block:
return ParseResult(
thinking_trace=combined_trace,
final_text=final_text.strip(),
answers=[],
valid=False,
error="empty FINAL ANSWERS block",
)
# Be tolerant if the model emits a JSON list even though bare lines were asked for.
answers: list[str] = []
parsed_json_list = False
if answer_block.startswith("["):
try:
decoded = json.loads(answer_block)
if isinstance(decoded, list):
parsed_json_list = True
answers = [str(value).strip() for value in decoded if str(value).strip()]
except json.JSONDecodeError:
pass
if not answers and not parsed_json_list:
for line in answer_block.splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("```"):
continue
if re.match(r"(?i)^(?:explanation|reasoning|notes?)\s*:", stripped):
break
cleaned = _clean_answer_line(stripped)
if cleaned:
answers.append(cleaned)
if not answers:
return ParseResult(
thinking_trace=combined_trace,
final_text=final_text.strip(),
answers=[],
valid=False,
error="no non-empty answers in FINAL ANSWERS block",
)
return ParseResult(
thinking_trace=combined_trace,
final_text=final_text.strip(),
answers=answers,
valid=True,
)
def _split_deepseek_response(response_text: str) -> tuple[str, str]:
"""Separate a completed DeepSeek think block from its visible answer."""
open_marker = "<think>"
close_marker = "</think>"
open_index = response_text.find(open_marker)
close_index = response_text.rfind(close_marker)
if open_index >= 0 and close_index > open_index:
reasoning = response_text[open_index + len(open_marker):close_index].strip()
visible = (
response_text[:open_index] + response_text[close_index + len(close_marker):]
).strip()
return reasoning, visible
if open_index < 0 and close_index >= 0:
# The solve prompt pre-fills <think>, so generated tokens commonly begin
# with the reasoning content and contain only the closing marker.
reasoning = response_text[:close_index].strip()
visible = response_text[close_index + len(close_marker):].strip()
return reasoning, visible
# If generation ended before </think> but still emitted the required final
# marker, leave the text intact so the strict parser can recover that block.
return "", response_text.replace(open_marker, "", 1).strip()
def _version_tuple(version: str) -> tuple[int, int, int]:
numbers = [int(value) for value in re.findall(r"\d+", version)[:3]]
return tuple((numbers + [0, 0, 0])[:3]) # type: ignore[return-value]
def _keep_last_token_hidden_state(_module: Any, inputs: tuple[Any, ...]) -> tuple[Any, ...] | None:
"""Avoid materializing full-sequence vocabulary logits during generation."""
if not inputs:
return None
hidden_states = inputs[0]
if hidden_states.ndim == 3 and hidden_states.shape[1] > 1:
return (hidden_states[:, -1:, :],) + inputs[1:]
return None
def load_model_and_tokenizer() -> tuple[Any, Any, Any]:
"""Load the pre-quantized AWQ model shipped in this repository."""
try:
import torch
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
except ImportError as exc:
raise RuntimeError(
"PyTorch, Transformers, Accelerate, AutoAWQ, and safetensors must be "
"available in the evaluation image."
) from exc
if _version_tuple(transformers.__version__) < (4, 37, 0):
raise RuntimeError(
"DeepSeek-R1-Distill-Qwen-7B uses the Qwen2 architecture and requires "
f"transformers>=4.37.0; found {transformers.__version__}."
)
model_config_path = REPO_DIR / "config.json"
if not model_config_path.exists():
raise RuntimeError(
"DeepSeek-R1-Distill-Qwen-7B-AWQ files are missing. Put the complete model and "
"tokenizer snapshot in the same repository directory as script.py."
)
with model_config_path.open("r", encoding="utf-8") as handle:
model_metadata = json.load(handle)
quantization_method = str(
model_metadata.get("quantization_config", {}).get("quant_method", "")
).casefold()
if (
model_metadata.get("model_type") != "qwen2"
or quantization_method != "awq"
or int(model_metadata.get("hidden_size", 0)) != 3584
or int(model_metadata.get("num_hidden_layers", 0)) != 28
):
raise RuntimeError(
"This script expects a 4-bit AWQ conversion of "
"DeepSeek-R1-Distill-Qwen-7B (Qwen2, hidden_size=3584, 28 layers)."
)
tokenizer = AutoTokenizer.from_pretrained(
str(REPO_DIR), local_files_only=True, trust_remote_code=False
)
if tokenizer.pad_token_id is None:
tokenizer.pad_token_id = tokenizer.eos_token_id
model = AutoModelForCausalLM.from_pretrained(
str(REPO_DIR),
local_files_only=True,
trust_remote_code=False,
device_map="auto",
torch_dtype=torch.float16,
low_cpu_mem_usage=True,
).eval()
model.config.use_cache = True
# Transformers 4.44.1 projects every prompt token through Qwen2's large
# vocabulary head and then upcasts all logits to FP32. Generation only uses
# the final-token logits, so trim the token dimension immediately before
# lm_head to avoid a multi-gigabyte prefill allocation on the T4.
model.lm_head.register_forward_pre_hook(_keep_last_token_hidden_state)
input_device = model.get_input_embeddings().weight.device
return model, tokenizer, input_device
def _problem_prompt(row: dict[str, Any], rag_text: str, retry_note: str = "") -> str:
work_language = str(row.get("work_lang", "")).strip()
task_language = str(row.get("task_lang", "")).strip()
language_lines = []
if work_language:
language_lines.append(f"Working language: {work_language}")
if task_language:
language_lines.append(f"Problem language: {task_language}")
language_metadata = "\n".join(language_lines)
sections = [rag_text] if rag_text else []
sections.append("CURRENT PROBLEM")
if language_metadata:
sections.append(language_metadata)
sections.extend(
[
"CONTEXT:\n" + str(row.get("context", "")),
"QUERY:\n" + str(row.get("query", "")),
]
)
if retry_note:
sections.append(retry_note)
return "\n\n".join(sections)
def _encode_fitted_prompt(
tokenizer: Any,
system_prompt: str,
row: dict[str, Any],
rag_text: str,
retry_note: str,
context_window: int,
requested_new_tokens: int,
) -> tuple[dict[str, Any], int]:
"""Trim retrieved material, never the current problem, to fit the context."""
fitted_rag = rag_text
while True:
# DeepSeek recommends placing all instructions in the user message for
# this R1 distillation rather than using a separate system message.
messages = [
{
"role": "user",
"content": system_prompt
+ "\n\n"
+ _problem_prompt(row, fitted_rag, retry_note),
}
]
rendered = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
rendered += "<think>\n"
encoded = tokenizer(rendered, return_tensors="pt", add_special_tokens=False)
prompt_tokens = int(encoded["input_ids"].shape[-1])
available = context_window - prompt_tokens
if available >= requested_new_tokens:
return encoded, requested_new_tokens
if fitted_rag:
if len(fitted_rag) <= 600:
fitted_rag = ""
else:
new_length = max(0, int(len(fitted_rag) * 0.72))
fitted_rag = fitted_rag[:new_length].rsplit("\n", 1)[0].rstrip()
fitted_rag += "\n[retrieved material shortened to fit the model context]"
continue
if available >= 256:
return encoded, available
raise RuntimeError(
f"The current problem and solver instructions use {prompt_tokens} tokens, "
f"leaving only {available} tokens in the {context_window}-token context."
)
def generate_once(
model: Any,
tokenizer: Any,
input_device: Any,
system_prompt: str,
row: dict[str, Any],
rag_text: str,
retry_note: str,
config: dict[str, Any],
seed: int,
) -> ParseResult:
import torch
configured_reasoning_tokens = int(config["max_reasoning_tokens"])
configured_answer_tokens = int(config["max_answer_tokens"])
encoded, available_generation_tokens = _encode_fitted_prompt(
tokenizer=tokenizer,
system_prompt=system_prompt,
row=row,
rag_text=rag_text,
retry_note=retry_note,
context_window=int(config["model_context_tokens"]),
requested_new_tokens=(
configured_reasoning_tokens + configured_answer_tokens + 16
),
)
encoded = {name: value.to(input_device) for name, value in encoded.items()}
answer_tokens = min(configured_answer_tokens, available_generation_tokens - 272)
reasoning_tokens = min(
configured_reasoning_tokens,
available_generation_tokens - answer_tokens - 16,
)
if reasoning_tokens < 256 or answer_tokens < 256:
raise RuntimeError(
"The fitted prompt does not leave at least 256 tokens for both the "
"reasoning and answer stages."
)
close_ids = tokenizer(
"</think>", add_special_tokens=False, return_tensors="pt"
)["input_ids"][0].tolist()
if len(close_ids) != 1:
raise RuntimeError("Expected </think> to be one tokenizer token.")
close_id = int(close_ids[0])
def sample(
input_ids: Any,
attention_mask: Any,
max_tokens: int,
sample_seed: int,
eos_token_id: Any,
) -> Any:
torch.manual_seed(sample_seed)
if torch.cuda.is_available():
torch.cuda.manual_seed_all(sample_seed)
with torch.inference_mode():
return model.generate(
input_ids=input_ids,
attention_mask=attention_mask,
max_new_tokens=max_tokens,
do_sample=True,
temperature=float(config["temperature"]),
top_p=float(config["top_p"]),
top_k=int(config["top_k"]),
repetition_penalty=float(config["repetition_penalty"]),
use_cache=True,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=eos_token_id,
)
reasoning_output = sample(
encoded["input_ids"],
encoded["attention_mask"],
reasoning_tokens,
seed,
[tokenizer.eos_token_id, close_id],
)
prompt_length = int(encoded["input_ids"].shape[-1])
reasoning_ids = reasoning_output[0, prompt_length:].tolist()
reasoning_closed = bool(reasoning_ids and reasoning_ids[-1] == close_id)
if reasoning_ids and reasoning_ids[-1] == tokenizer.eos_token_id:
reasoning_ids = reasoning_ids[:-1]
reasoning_output = reasoning_output[:, :-1]
reasoning_text = tokenizer.decode(
reasoning_ids, skip_special_tokens=True
).strip()
thinking_trace, _ = _split_deepseek_response(reasoning_text)
if not thinking_trace:
thinking_trace = reasoning_text.replace("</think>", "").strip()
print(
f" reasoning tokens={len(reasoning_ids)}/{reasoning_tokens}; "
f"closed={'yes' if reasoning_closed else 'forced'}",
flush=True,
)
answer_prefix = "\nFINAL ANSWERS:\n" if reasoning_closed else "</think>\nFINAL ANSWERS:\n"
prefix_ids = tokenizer(
answer_prefix, add_special_tokens=False, return_tensors="pt"
)["input_ids"].to(input_device)
answer_input_ids = torch.cat((reasoning_output, prefix_ids), dim=1)
answer_attention_mask = torch.ones_like(answer_input_ids)
answer_budgets = [answer_tokens, int(config["answer_retry_tokens"])]
last_result: ParseResult | None = None
for answer_attempt, answer_budget in enumerate(answer_budgets, start=1):
answer_output = sample(
answer_input_ids,
answer_attention_mask,
answer_budget,
seed + answer_attempt,
tokenizer.eos_token_id,
)
answer_ids = answer_output[0, answer_input_ids.shape[-1]:].tolist()
answer_text = tokenizer.decode(
answer_ids, skip_special_tokens=True
).strip()
final_text = "FINAL ANSWERS:\n" + answer_text
result = parse_model_response(thinking_trace, final_text)
print(
f" answer attempt {answer_attempt}/{len(answer_budgets)}: "
f"tokens={len(answer_ids)}/{answer_budget}; "
f"parser={'accepted' if result.valid else result.error}",
flush=True,
)
if result.valid:
return result
last_result = result
assert last_result is not None
return last_result
def solve_row(
row: dict[str, Any],
row_index: int,
retriever: BookRetriever,
model: Any,
tokenizer: Any,
input_device: Any,
system_prompt: str,
config: dict[str, Any],
) -> ParseResult:
retrieval = retriever.retrieve(
row,
top_methods=int(config["top_methods"]),
top_examples=int(config["top_examples"]),
char_tfidf_weight=float(config["char_tfidf_weight"]),
)
rag_text = retriever.format_for_prompt(
retrieval, max_chars=int(config["rag_max_chars"])
)
return generate_once(
model=model,
tokenizer=tokenizer,
input_device=input_device,
system_prompt=system_prompt,
row=row,
rag_text=rag_text,
retry_note="",
config=config,
seed=int(config["seed"]) + row_index * 3,
)
def generate_explanation(
solution: ParseResult,
model: Any,
tokenizer: Any,
input_device: Any,
config: dict[str, Any],
) -> str:
"""Summarize the model's accepted reasoning for the optional jury track."""
import torch
source_reasoning = solution.thinking_trace.strip() or solution.final_text.strip()
answer_text = "\n".join(solution.answers)
messages = [
{
"role": "user",
"content": (
"Write a short, human-readable explanation of the solution in a few "
"concise bullet points. State the linguistic rule or pattern found, "
"the key evidence, and how the final answers follow. Do not reproduce "
"the raw reasoning trace, trial and error, or meta-commentary. Do not "
"change the final answers. Output only the explanation.\n\n"
f"MODEL REASONING:\n{source_reasoning}\n\n"
f"FINAL ANSWERS:\n{answer_text}"
),
},
]
rendered = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
encoded = tokenizer(rendered, return_tensors="pt", add_special_tokens=False)
encoded = {name: value.to(input_device) for name, value in encoded.items()}
prompt_tokens = int(encoded["input_ids"].shape[-1])
available = int(config["model_context_tokens"]) - prompt_tokens
max_new_tokens = min(int(config["explanation_max_new_tokens"]), available)
if max_new_tokens < 32:
return (
"- Inferred the relevant linguistic patterns from the supplied examples.\n"
"- Applied those patterns to produce the listed answers."
)
with torch.inference_mode():
output = model.generate(
**encoded,
max_new_tokens=max_new_tokens,
do_sample=False,
use_cache=True,
pad_token_id=tokenizer.pad_token_id,
)
generated_ids = output[0, prompt_tokens:].tolist()
raw_explanation = tokenizer.decode(
generated_ids, skip_special_tokens=True
).strip()
if "<think>" in raw_explanation and "</think>" not in raw_explanation:
raw_explanation = ""
_, explanation = _split_deepseek_response(raw_explanation)
if explanation:
return explanation
return (
"- Inferred the relevant linguistic patterns from the supplied examples.\n"
"- Applied those patterns to produce the listed answers."
)
def read_test_rows(path: Path) -> list[dict[str, Any]]:
with path.open("r", encoding="utf-8-sig", newline="") as handle:
return list(csv.DictReader(handle))
def _write_submission_stream(handle: Any, rows: list[dict[str, str]]) -> None:
writer = csv.DictWriter(handle, fieldnames=["id", "pred", "explanation"])
writer.writeheader()
writer.writerows(rows)
def write_submission(path: Path, rows: list[dict[str, str]]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
temporary_path = path.with_name(f".{path.name}.tmp")
with temporary_path.open("w", encoding="utf-8", newline="") as handle:
_write_submission_stream(handle, rows)
os.replace(temporary_path, path)
def run_self_test() -> None:
import numpy as np
hidden_states = np.arange(24).reshape(1, 3, 8)
trimmed = _keep_last_token_hidden_state(None, (hidden_states,))
assert trimmed is not None and trimmed[0].shape == (1, 1, 8)
assert np.array_equal(trimmed[0], hidden_states[:, -1:, :])
assert _keep_last_token_hidden_state(None, (hidden_states[:, -1:, :],)) is None
parsed = parse_model_response(
"A useful analysis.",
"One last check.\nFINAL ANSWERS:\n1. čha\n2) multi word form",
)
assert parsed.valid
assert parsed.answers == ["čha", "multi word form"]
assert "One last check." in parsed.thinking_trace
assert not parse_model_response("", "FINAL ANSWERS:\n").valid
assert not parse_model_response("", "FINAL ANSWERS:\n[]").valid
assert not parse_model_response("", 'FINAL ANSWERS:\n[""]').valid
assert not parse_model_response("", "The answer is x.").valid
deepseek_trace, deepseek_final = _split_deepseek_response(
"<think>Compare the recurring suffixes.</think>\nFINAL ANSWERS:\nform"
)
assert deepseek_trace == "Compare the recurring suffixes."
assert deepseek_final == "FINAL ANSWERS:\nform"
deepseek_parsed = parse_model_response(deepseek_trace, deepseek_final)
assert deepseek_parsed.valid and deepseek_parsed.answers == ["form"]
prefixed_trace, prefixed_final = _split_deepseek_response(
"Compare the recurring suffixes.</think>\nFINAL ANSWERS:\nform"
)
assert prefixed_trace == "Compare the recurring suffixes."
assert prefixed_final == "FINAL ANSWERS:\nform"
retriever = BookRetriever(RESOURCE_DIR)
canary_row = {
"context": "The following words are number expressions in an unknown language.",
"query": "Determine the rule and write the number 25 in words.",
"answer": "PRIVATE_ANSWER_CANARY",
}
result = retriever.retrieve(
canary_row, top_methods=2, top_examples=2, char_tfidf_weight=3.0
)
prompt = retriever.format_for_prompt(result, max_chars=8000)
assert result["methods"] and result["examples"]
assert "PRIVATE_ANSWER_CANARY" not in prompt
assert all(example.get("string_only_usable") for example in result["examples"])
exact_example = retriever.examples[0]
exact_result = retriever.retrieve(
{"context": exact_example["context"], "query": exact_example["query"]},
top_methods=1,
top_examples=1,
char_tfidf_weight=3.0,
)
assert exact_result["examples"][0]["id"] == exact_example["id"]
submission_buffer = io.StringIO(newline="")
_write_submission_stream(
submission_buffer,
[
{
"id": "007",
"pred": json.dumps(["čha", "multi word"], ensure_ascii=False),
"explanation": "- Identified the relevant pattern.",
}
],
)
submission_buffer.seek(0)
assert list(csv.DictReader(submission_buffer)) == [
{
"id": "007",
"pred": '["čha", "multi word"]',
"explanation": "- Identified the relevant pattern.",
}
]
incremental_rows = [
{"id": "001", "pred": "[]", "explanation": ""},
{"id": "002", "pred": "[]", "explanation": ""},
]
incremental_rows[0]["pred"] = json.dumps(["answer"], ensure_ascii=False)
incremental_buffer = io.StringIO(newline="")
_write_submission_stream(incremental_buffer, incremental_rows)
incremental_buffer.seek(0)
assert list(csv.DictReader(incremental_buffer)) == incremental_rows
print(
"Self-test passed: DeepSeek think parsing, last-token logits hook, strict "
"answer parsing, hybrid book-only retrieval, exact-match retrieval, and "
"submission CSV serialization."
)
def main() -> None:
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8")
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--input", type=Path, default=DEFAULT_INPUT)
parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
parser.add_argument(
"--explanations",
choices=("on", "off"),
default=None,
help="Generate optional jury-track explanations (default: config setting)",
)
parser.add_argument(
"--self-test", action="store_true", help="Test parser/retrieval without loading the model"
)
args = parser.parse_args()
if args.self_test:
run_self_test()
return
if not args.input.exists():
raise SystemExit(f"Test file not found: {args.input}")
config = _load_config()
explanations_enabled = (
args.explanations == "on"
if args.explanations is not None
else bool(config.get("enable_explanations", False))
)
system_prompt = (RESOURCE_DIR / "system_prompt.txt").read_text(encoding="utf-8").strip()
test_rows = read_test_rows(args.input)
retriever = BookRetriever(RESOURCE_DIR)
model, tokenizer, input_device = load_model_and_tokenizer()
submission_rows: list[dict[str, str]] = [
{
"id": str(row.get("id", "")),
"pred": "[]",
"explanation": "",
}
for row in test_rows
]
write_submission(args.output, submission_rows)
print(
f"Initialized incremental submission with {len(submission_rows)} rows at "
f"{args.output}; explanations={'on' if explanations_enabled else 'off'}",
flush=True,
)
for index, row in enumerate(test_rows):
row_id = str(row.get("id", ""))
print(f"[{index + 1}/{len(test_rows)}] solving id={row_id}", flush=True)
solution = solve_row(
row=row,
row_index=index,
retriever=retriever,
model=model,
tokenizer=tokenizer,
input_device=input_device,
system_prompt=system_prompt,
config=config,
)
explanation = ""
if explanations_enabled:
explanation = generate_explanation(
solution=solution,
model=model,
tokenizer=tokenizer,
input_device=input_device,
config=config,
)
submission_rows[index] = {
"id": row_id,
"pred": json.dumps(solution.answers, ensure_ascii=False),
"explanation": explanation,
}
write_submission(args.output, submission_rows)
print(f"Checkpointed prediction {index + 1}/{len(test_rows)}", flush=True)
print(f"Wrote {len(submission_rows)} predictions to {args.output}", flush=True)
if __name__ == "__main__":
main()

23
special_tokens_map.json Normal file
View File

@@ -0,0 +1,23 @@
{
"bos_token": {
"content": "<｜begin▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"eos_token": {
"content": "<｜end▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
},
"pad_token": {
"content": "<｜end▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false
}
}

3
tokenizer.json Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d8d1c97bbabe51a8b9443165d2d20a5df15e42e09ac5c09f4c5c7a5c8f5abdcb
size 4757461

195
tokenizer_config.json Normal file
View File

@@ -0,0 +1,195 @@
{
"add_bos_token": true,
"add_eos_token": false,
"add_prefix_space": null,
"added_tokens_decoder": {
"151643": {
"content": "<｜end▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151644": {
"content": "<｜User｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151645": {
"content": "<｜Assistant｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151646": {
"content": "<｜begin▁of▁sentence｜>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151647": {
"content": "<|EOT|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151648": {
"content": "<think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151649": {
"content": "</think>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151650": {
"content": "<|quad_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151651": {
"content": "<|quad_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151652": {
"content": "<|vision_start|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151653": {
"content": "<|vision_end|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151654": {
"content": "<|vision_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151655": {
"content": "<|image_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151656": {
"content": "<|video_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": true
},
"151657": {
"content": "<tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151658": {
"content": "</tool_call>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151659": {
"content": "<|fim_prefix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151660": {
"content": "<|fim_middle|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151661": {
"content": "<|fim_suffix|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151662": {
"content": "<|fim_pad|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151663": {
"content": "<|repo_name|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
},
"151664": {
"content": "<|file_sep|>",
"lstrip": false,
"normalized": false,
"rstrip": false,
"single_word": false,
"special": false
}
},
"bos_token": "<｜begin▁of▁sentence｜>",
"chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set ns = namespace(is_first=false, is_tool=false, is_output_first=true, system_prompt='') %}{%- for message in messages %}{%- if message['role'] == 'system' %}{% set ns.system_prompt = message['content'] %}{%- endif %}{%- endfor %}{{bos_token}}{{ns.system_prompt}}{%- for message in messages %}{%- if message['role'] == 'user' %}{%- set ns.is_tool = false -%}{{'<｜User｜>' + message['content']}}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is none %}{%- set ns.is_tool = false -%}{%- for tool in message['tool_calls']%}{%- if not ns.is_first %}{{'<｜Assistant｜><｜tool▁calls▁begin｜><｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{%- set ns.is_first = true -%}{%- else %}{{'\\n' + '<｜tool▁call▁begin｜>' + tool['type'] + '<｜tool▁sep｜>' + tool['function']['name'] + '\\n' + '```json' + '\\n' + tool['function']['arguments'] + '\\n' + '```' + '<｜tool▁call▁end｜>'}}{{'<｜tool▁calls▁end｜><｜end▁of▁sentence｜>'}}{%- endif %}{%- endfor %}{%- endif %}{%- if message['role'] == 'assistant' and message['content'] is not none %}{%- if ns.is_tool %}{{'<｜tool▁outputs▁end｜>' + message['content'] + '<｜end▁of▁sentence｜>'}}{%- set ns.is_tool = false -%}{%- else %}{% set content = message['content'] %}{% if '</think>' in content %}{% set content = content.split('</think>')[-1] %}{% endif %}{{'<｜Assistant｜>' + content + '<｜end▁of▁sentence｜>'}}{%- endif %}{%- endif %}{%- if message['role'] == 'tool' %}{%- set ns.is_tool = true -%}{%- if ns.is_output_first %}{{'<｜tool▁outputs▁begin｜><｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- set ns.is_output_first = false %}{%- else %}{{'\\n<｜tool▁output▁begin｜>' + message['content'] + '<｜tool▁output▁end｜>'}}{%- endif %}{%- endif %}{%- endfor -%}{% if ns.is_tool %}{{'<｜tool▁outputs▁end｜>'}}{% endif %}{% if add_generation_prompt and not ns.is_tool %}{{'<｜Assistant｜>'}}{% endif %}",
"clean_up_tokenization_spaces": false,
"eos_token": "<｜end▁of▁sentence｜>",
"extra_special_tokens": {},
"legacy": true,
"model_max_length": 16384,
"pad_token": "<｜end▁of▁sentence｜>",
"sp_model_kwargs": {},
"tokenizer_class": "LlamaTokenizer",
"unk_token": null,
"use_default_system_prompt": false
}