初始化项目，由ModelHub XC社区提供模型
Model: jepetolee/Qwen3-4B-AMQ3-Math-SFT Source: Original Platform
This commit is contained in:
ModelHub XC
36
.gitattributes
vendored
Normal file
36
.gitattributes
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
||||
198
README.md
Normal file
198
README.md
Normal file
@@ -0,0 +1,198 @@
|
||||
---
|
||||
license: other
|
||||
license_name: research-only
|
||||
base_model: Qwen/Qwen3-4B-Base
|
||||
datasets:
|
||||
- jepetolee/AMQ3-Math-ShortCoT-0k7k
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- math
|
||||
- reasoning
|
||||
- chain-of-thought
|
||||
- qwen3
|
||||
- sft
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
# Qwen3-4B AMQ3 Math Short-CoT SFT
|
||||
|
||||
Supervised fine-tune of **Qwen/Qwen3-4B-Base** on ~292K short chain-of-thought math
|
||||
solutions distilled from **Qwen3-235B-A22B** (via `a-m-team/AM-Qwen3-Distilled`).
|
||||
Intended as a clean math-reasoning cold-start checkpoint (e.g. before RLVR).
|
||||
|
||||
## Training
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Base model | `Qwen/Qwen3-4B-Base` |
|
||||
| Data | [`jepetolee/AMQ3-Math-ShortCoT-0k7k`](https://huggingface.co/datasets/jepetolee/AMQ3-Math-ShortCoT-0k7k) — 292,375 examples, problem+CoT ≤ ~7K tokens |
|
||||
| Format | official Qwen3 chat template, `<think>…</think>` reasoning + `\boxed{}` answer |
|
||||
| Epochs | 1 (full dataset) |
|
||||
| Effective batch | 32 · lr 1e-5 · warmup 0.03 · max_len 9216 |
|
||||
| Final loss | 0.48 (token-weighted, full dataset) |
|
||||
| Tokens seen | ~0.87B (96.9% on the target span) |
|
||||
|
||||
## Prompt format
|
||||
|
||||
The model is trained to open reasoning with `<think>\n` right after the assistant
|
||||
header. Use the chat template and let it generate the `<think>` block:
|
||||
|
||||
```
|
||||
<|im_start|>system
|
||||
Please reason step by step, and put your final answer within \boxed{}.<|im_end|>
|
||||
<|im_start|>user
|
||||
{question}<|im_end|>
|
||||
<|im_start|>assistant
|
||||
<think>
|
||||
```
|
||||
|
||||
## Usage (vLLM)
|
||||
|
||||
```python
|
||||
from vllm import LLM, SamplingParams
|
||||
|
||||
llm = LLM(model="jepetolee/Qwen3-4B-AMQ3-Math-SFT", max_model_len=9216)
|
||||
sp = SamplingParams(temperature=0.7, top_p=0.9, max_tokens=8192)
|
||||
|
||||
prompt = (
|
||||
"<|im_start|>system\n"
|
||||
"Please reason step by step, and put your final answer within \\boxed{}.<|im_end|>\n"
|
||||
"<|im_start|>user\nWhat is the sum of the first 10 primes?<|im_end|>\n"
|
||||
"<|im_start|>assistant\n<think>\n"
|
||||
)
|
||||
print(llm.generate([prompt], sp)[0].outputs[0].text)
|
||||
```
|
||||
|
||||
The `generation_config.json` sets `eos_token_id = [151645, 151643]` so generation
|
||||
stops on `<|im_end|>` out of the box — no manual `stop_token_ids` needed.
|
||||
|
||||
## Recommended sampling
|
||||
|
||||
Hygiene degrades sharply above temperature 0.75 (holdout sweep, no logit processors):
|
||||
|
||||
| temperature | fully-clean generations | ends without an answer |
|
||||
|---|---|---|
|
||||
| 0.5 | 80% | 20% |
|
||||
| 0.75 | 75% | 17% |
|
||||
| 1.0 | 33% | 33% |
|
||||
|
||||
**Use temperature ≤ 0.75**, or use the generation recipe below, which keeps
|
||||
higher temperatures usable by construction.
|
||||
|
||||
## Generation recipe: think/answer budget split (logits processors)
|
||||
|
||||
The failure mode behind the table above: on hard prompts the model keeps thinking
|
||||
until the token cap and never closes `</think>`, so the run truncates with no
|
||||
`\boxed{}` answer. Instead of only lowering temperature, we split the generation
|
||||
budget — **total 10240 tokens = up to 6144 think + ~4096 answer** — and enforce it
|
||||
with two vLLM V1 logits processors, shipped in this repo:
|
||||
|
||||
| file | role |
|
||||
|---|---|
|
||||
| [`vllm_think_format.py`](./vllm_think_format.py) | `<think>` tag grammar + think-budget cut + forced seal |
|
||||
| [`vllm_repetition_abort.py`](./vllm_repetition_abort.py) | early EOS for n-gram repetition runaways |
|
||||
|
||||
How it works:
|
||||
|
||||
1. **Prefill `<think>\n`** after the assistant header (see Prompt format) — the
|
||||
think block opens exactly once, by construction.
|
||||
2. **Tag grammar (token-id state machine, no decoding)**: while think is open,
|
||||
`<think>` is banned; after the first `</think>` both tags are banned forever;
|
||||
optionally `<|im_start|>` is banned (blocks fake new-turn hallucinations).
|
||||
3. **Think-budget cut with forced seal**: when the think span reaches
|
||||
`max_think_tokens`, the processor force-prefills `</think>\n\n` and constrains
|
||||
the *first* answer token to a whitelist of answer-opening tokens
|
||||
(`To / We / Let / The / Given / ### / ( / First / In` — ≥96% coverage of answer
|
||||
openers measured on the 292K SFT set). The model then writes a normal answer
|
||||
with the remaining budget, so a `\boxed{}` answer still appears even when
|
||||
thinking was cut.
|
||||
4. **Repetition abort**: if a rollout's 7-gram repetition ratio exceeds 0.9
|
||||
(checked every 512 tokens, after the first 2048), logits are masked to EOS-only
|
||||
for that request. Rollouts that never trigger are **bit-identical** to running
|
||||
without the processor.
|
||||
|
||||
> **vLLM caveat (important)**: pass `async_scheduling=False` to the engine.
|
||||
> vLLM V1's async scheduling fills `output_tok_ids` with `-1` placeholders, which
|
||||
> silently disables any logits processor that reads output tokens.
|
||||
|
||||
```python
|
||||
from vllm import LLM, SamplingParams
|
||||
from transformers import AutoTokenizer
|
||||
|
||||
# Download vllm_think_format.py / vllm_repetition_abort.py from this repo
|
||||
# and put them on your PYTHONPATH.
|
||||
from vllm_think_format import build_think_format_extra_args
|
||||
from vllm_repetition_abort import build_repetition_abort_extra_args
|
||||
|
||||
model_id = "jepetolee/Qwen3-4B-AMQ3-Math-SFT"
|
||||
tok = AutoTokenizer.from_pretrained(model_id)
|
||||
|
||||
llm = LLM(
|
||||
model=model_id,
|
||||
max_model_len=32768,
|
||||
async_scheduling=False, # REQUIRED for the custom processors
|
||||
logits_processors=[
|
||||
"vllm_think_format:ThinkFormatLogitsProcessor",
|
||||
"vllm_repetition_abort:RepetitionEosLogitsProcessor",
|
||||
],
|
||||
)
|
||||
|
||||
extra_args = {}
|
||||
extra_args.update(build_think_format_extra_args(
|
||||
{"think_format": {
|
||||
"enabled": True,
|
||||
"prefilled_open": True, # prompt ends with "<think>\n"
|
||||
"ban_im_start": True,
|
||||
"max_think_tokens": 6144, # think budget
|
||||
"force_close_prefill": True, # seal "</think>\n\n" + whitelist on cut
|
||||
}},
|
||||
tok, prefilled_open=True) or {})
|
||||
extra_args.update(build_repetition_abort_extra_args(
|
||||
{"repetition_abort": {
|
||||
"enabled": True, "ngram": 7, "threshold": 0.9,
|
||||
"min_tokens": 2048, "check_interval": 512,
|
||||
}},
|
||||
eos_token_id=tok.convert_tokens_to_ids("<|im_end|>")) or {})
|
||||
|
||||
sp = SamplingParams(
|
||||
temperature=0.7, top_p=0.95,
|
||||
max_tokens=10240, # total budget: think 6144 + answer ~4096
|
||||
extra_args=extra_args,
|
||||
)
|
||||
|
||||
prompt = (
|
||||
"<|im_start|>system\n"
|
||||
"Please reason step by step, and put your final answer within \\boxed{}.<|im_end|>\n"
|
||||
"<|im_start|>user\n{question}<|im_end|>\n"
|
||||
"<|im_start|>assistant\n<think>\n"
|
||||
)
|
||||
print(llm.generate([prompt], sp)[0].outputs[0].text)
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- The processor code is research code from our RL training stack (docstrings are in
|
||||
Korean); requests whose `extra_args` omit the config blocks are ignored entirely,
|
||||
so the processors are safe to register globally.
|
||||
- Budget scaling: with 10240 total on this model, per-problem worst-case decode cost
|
||||
scales roughly with the square of the total length — 12288 costs ~2× and 16384
|
||||
~3.5× of an 8192 budget. 6144/4096 was chosen as the stability/cost sweet spot.
|
||||
- With the recipe active, temperature 1.0 remains usable: unclosed-think truncations
|
||||
are eliminated by construction (thinking is force-sealed and the answer budget is
|
||||
reserved).
|
||||
|
||||
## Limitations
|
||||
|
||||
- Math only (English). MCQ items were filtered out of the training data.
|
||||
- Answers are `\boxed{}`; grading assumes boxed-answer extraction.
|
||||
- Distilled from a single teacher (Qwen3-235B-A22B); inherits its style and blind spots.
|
||||
|
||||
## License
|
||||
|
||||
Base model `Qwen/Qwen3-4B-Base` is Apache-2.0, but training data derives from
|
||||
`a-m-team/AM-Qwen3-Distilled`, which restricts use to **research purposes only**.
|
||||
This checkpoint therefore carries the same research-only restriction: no commercial
|
||||
use, no potentially harmful application. The bundled logits-processor files are
|
||||
released under the same research-only terms.
|
||||
28
added_tokens.json
Normal file
28
added_tokens.json
Normal file
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"</think>": 151668,
|
||||
"</tool_call>": 151658,
|
||||
"</tool_response>": 151666,
|
||||
"<think>": 151667,
|
||||
"<tool_call>": 151657,
|
||||
"<tool_response>": 151665,
|
||||
"<|box_end|>": 151649,
|
||||
"<|box_start|>": 151648,
|
||||
"<|endoftext|>": 151643,
|
||||
"<|file_sep|>": 151664,
|
||||
"<|fim_middle|>": 151660,
|
||||
"<|fim_pad|>": 151662,
|
||||
"<|fim_prefix|>": 151659,
|
||||
"<|fim_suffix|>": 151661,
|
||||
"<|im_end|>": 151645,
|
||||
"<|im_start|>": 151644,
|
||||
"<|image_pad|>": 151655,
|
||||
"<|object_ref_end|>": 151647,
|
||||
"<|object_ref_start|>": 151646,
|
||||
"<|quad_end|>": 151651,
|
||||
"<|quad_start|>": 151650,
|
||||
"<|repo_name|>": 151663,
|
||||
"<|video_pad|>": 151656,
|
||||
"<|vision_end|>": 151653,
|
||||
"<|vision_pad|>": 151654,
|
||||
"<|vision_start|>": 151652
|
||||
}
|
||||
85
chat_template.jinja
Normal file
85
chat_template.jinja
Normal file
@@ -0,0 +1,85 @@
|
||||
{%- if tools %}
|
||||
{{- '<|im_start|>system\n' }}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- messages[0].content + '\n\n' }}
|
||||
{%- endif %}
|
||||
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
||||
{%- for tool in tools %}
|
||||
{{- "\n" }}
|
||||
{{- tool | tojson }}
|
||||
{%- endfor %}
|
||||
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
||||
{%- else %}
|
||||
{%- if messages[0].role == 'system' %}
|
||||
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
||||
{%- for message in messages[::-1] %}
|
||||
{%- set index = (messages|length - 1) - loop.index0 %}
|
||||
{%- if ns.multi_step_tool and message.role == "user" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
||||
{%- set ns.multi_step_tool = false %}
|
||||
{%- set ns.last_query_index = index %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- for message in messages %}
|
||||
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
|
||||
{%- elif message.role == "assistant" %}
|
||||
{%- set content = message.content %}
|
||||
{%- set reasoning_content = '' %}
|
||||
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
|
||||
{%- set reasoning_content = message.reasoning_content %}
|
||||
{%- else %}
|
||||
{%- if '</think>' in message.content %}
|
||||
{%- set content = message.content.split('</think>')[-1].lstrip('\n') %}
|
||||
{%- set reasoning_content = message.content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- if loop.index0 > ns.last_query_index %}
|
||||
{%- if loop.last or (not loop.last and reasoning_content) %}
|
||||
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- else %}
|
||||
{{- '<|im_start|>' + message.role + '\n' + content }}
|
||||
{%- endif %}
|
||||
{%- if message.tool_calls %}
|
||||
{%- for tool_call in message.tool_calls %}
|
||||
{%- if (loop.first and content) or (not loop.first) %}
|
||||
{{- '\n' }}
|
||||
{%- endif %}
|
||||
{%- if tool_call.function %}
|
||||
{%- set tool_call = tool_call.function %}
|
||||
{%- endif %}
|
||||
{{- '<tool_call>\n{"name": "' }}
|
||||
{{- tool_call.name }}
|
||||
{{- '", "arguments": ' }}
|
||||
{%- if tool_call.arguments is string %}
|
||||
{{- tool_call.arguments }}
|
||||
{%- else %}
|
||||
{{- tool_call.arguments | tojson }}
|
||||
{%- endif %}
|
||||
{{- '}\n</tool_call>' }}
|
||||
{%- endfor %}
|
||||
{%- endif %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- elif message.role == "tool" %}
|
||||
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
||||
{{- '<|im_start|>user' }}
|
||||
{%- endif %}
|
||||
{{- '\n<tool_response>\n' }}
|
||||
{{- message.content }}
|
||||
{{- '\n</tool_response>' }}
|
||||
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
||||
{{- '<|im_end|>\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
{%- endfor %}
|
||||
{%- if add_generation_prompt %}
|
||||
{{- '<|im_start|>assistant\n' }}
|
||||
{%- if enable_thinking is defined and enable_thinking is false %}
|
||||
{{- '<think>\n\n</think>\n\n' }}
|
||||
{%- endif %}
|
||||
{%- endif %}
|
||||
68
config.json
Normal file
68
config.json
Normal file
@@ -0,0 +1,68 @@
|
||||
{
|
||||
"architectures": [
|
||||
"Qwen3ForCausalLM"
|
||||
],
|
||||
"attention_bias": false,
|
||||
"attention_dropout": 0.0,
|
||||
"bos_token_id": 151643,
|
||||
"dtype": "bfloat16",
|
||||
"eos_token_id": 151643,
|
||||
"head_dim": 128,
|
||||
"hidden_act": "silu",
|
||||
"hidden_size": 2560,
|
||||
"initializer_range": 0.02,
|
||||
"intermediate_size": 9728,
|
||||
"layer_types": [
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention",
|
||||
"full_attention"
|
||||
],
|
||||
"max_position_embeddings": 32768,
|
||||
"max_window_layers": 36,
|
||||
"model_type": "qwen3",
|
||||
"num_attention_heads": 32,
|
||||
"num_hidden_layers": 36,
|
||||
"num_key_value_heads": 8,
|
||||
"rms_norm_eps": 1e-06,
|
||||
"rope_scaling": null,
|
||||
"rope_theta": 1000000,
|
||||
"sliding_window": null,
|
||||
"tie_word_embeddings": true,
|
||||
"transformers_version": "4.57.6",
|
||||
"use_cache": false,
|
||||
"use_sliding_window": false,
|
||||
"vocab_size": 151936
|
||||
}
|
||||
9
generation_config.json
Normal file
9
generation_config.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"bos_token_id": 151643,
|
||||
"eos_token_id": [
|
||||
151645,
|
||||
151643
|
||||
],
|
||||
"pad_token_id": 151643,
|
||||
"transformers_version": "4.57.6"
|
||||
}
|
||||
151388
merges.txt
Normal file
151388
merges.txt
Normal file
File diff suppressed because it is too large
Load Diff
3
model-00001-of-00002.safetensors
Normal file
3
model-00001-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5e6a84e707103ec9f49f2a4e340ef74e45b1831e3686a2372126a3717611b5d6
|
||||
size 4967215360
|
||||
3
model-00002-of-00002.safetensors
Normal file
3
model-00002-of-00002.safetensors
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5774c372ea481686926b8254daeada254f91377582df7efeb52c337fd856e641
|
||||
size 3855679144
|
||||
407
model.safetensors.index.json
Normal file
407
model.safetensors.index.json
Normal file
@@ -0,0 +1,407 @@
|
||||
{
|
||||
"metadata": {
|
||||
"total_parameters": 1005617024,
|
||||
"total_size": 8822848512
|
||||
},
|
||||
"weight_map": {
|
||||
"lm_head.weight": "model-00002-of-00002.safetensors",
|
||||
"model.embed_tokens.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.17.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.18.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.19.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.20.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.24.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.25.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.26.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.27.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.28.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.29.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.30.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.30.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.31.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.32.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.33.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.34.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.input_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_norm.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.35.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
|
||||
"model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_norm.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
|
||||
"model.norm.weight": "model-00002-of-00002.safetensors"
|
||||
}
|
||||
}
|
||||
31
special_tokens_map.json
Normal file
31
special_tokens_map.json
Normal file
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"eos_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
},
|
||||
"pad_token": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false
|
||||
}
|
||||
}
|
||||
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
BIN
tokenizer.json
(Stored with Git LFS)
Normal file
Binary file not shown.
239
tokenizer_config.json
Normal file
239
tokenizer_config.json
Normal file
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"add_bos_token": false,
|
||||
"add_prefix_space": false,
|
||||
"added_tokens_decoder": {
|
||||
"151643": {
|
||||
"content": "<|endoftext|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151644": {
|
||||
"content": "<|im_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151645": {
|
||||
"content": "<|im_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151646": {
|
||||
"content": "<|object_ref_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151647": {
|
||||
"content": "<|object_ref_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151648": {
|
||||
"content": "<|box_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151649": {
|
||||
"content": "<|box_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151650": {
|
||||
"content": "<|quad_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151651": {
|
||||
"content": "<|quad_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151652": {
|
||||
"content": "<|vision_start|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151653": {
|
||||
"content": "<|vision_end|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151654": {
|
||||
"content": "<|vision_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151655": {
|
||||
"content": "<|image_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151656": {
|
||||
"content": "<|video_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": true
|
||||
},
|
||||
"151657": {
|
||||
"content": "<tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151658": {
|
||||
"content": "</tool_call>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151659": {
|
||||
"content": "<|fim_prefix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151660": {
|
||||
"content": "<|fim_middle|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151661": {
|
||||
"content": "<|fim_suffix|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151662": {
|
||||
"content": "<|fim_pad|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151663": {
|
||||
"content": "<|repo_name|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151664": {
|
||||
"content": "<|file_sep|>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151665": {
|
||||
"content": "<tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151666": {
|
||||
"content": "</tool_response>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151667": {
|
||||
"content": "<think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
},
|
||||
"151668": {
|
||||
"content": "</think>",
|
||||
"lstrip": false,
|
||||
"normalized": false,
|
||||
"rstrip": false,
|
||||
"single_word": false,
|
||||
"special": false
|
||||
}
|
||||
},
|
||||
"additional_special_tokens": [
|
||||
"<|im_start|>",
|
||||
"<|im_end|>",
|
||||
"<|object_ref_start|>",
|
||||
"<|object_ref_end|>",
|
||||
"<|box_start|>",
|
||||
"<|box_end|>",
|
||||
"<|quad_start|>",
|
||||
"<|quad_end|>",
|
||||
"<|vision_start|>",
|
||||
"<|vision_end|>",
|
||||
"<|vision_pad|>",
|
||||
"<|image_pad|>",
|
||||
"<|video_pad|>"
|
||||
],
|
||||
"bos_token": null,
|
||||
"clean_up_tokenization_spaces": false,
|
||||
"eos_token": "<|endoftext|>",
|
||||
"errors": "replace",
|
||||
"extra_special_tokens": {},
|
||||
"model_max_length": 131072,
|
||||
"pad_token": "<|endoftext|>",
|
||||
"split_special_tokens": false,
|
||||
"tokenizer_class": "Qwen2Tokenizer",
|
||||
"unk_token": null
|
||||
}
|
||||
179
vllm_repetition_abort.py
Normal file
179
vllm_repetition_abort.py
Normal file
@@ -0,0 +1,179 @@
|
||||
"""과도 반복(폭주) 롤아웃 조기 종료용 vLLM V1 커스텀 logits processor.
|
||||
|
||||
배경: long-CoT eval 실측에서 롤아웃의 ~35%가 20480 캡까지 도달하며, 상당수가
|
||||
n-gram 반복 루프(어차피 reward에서 오답 처리됨)다. 이들이 가장 비싼(긴 컨텍스트)
|
||||
디코드 토큰을 소모한다.
|
||||
|
||||
동작: 요청별로 생성 토큰 목록(vLLM이 라이브 참조로 제공)을 감시하다가
|
||||
n-gram 반복 비율이 임계 이상이면 해당 요청의 로짓을 EOS만 남기고 -inf로 마스킹
|
||||
→ 그 요청만 즉시 종료. **트리거 전에는 로짓을 전혀 건드리지 않으므로 정상
|
||||
롤아웃의 출력은 비트 단위로 동일하다** (출력 불변 원칙 하에 폭주 커팅만 승인됨,
|
||||
2026-07-09 결정). 트리거된 롤아웃은 짧게 잘린 채 수집되고 기존 repetition
|
||||
필터/reward가 오답 처리한다.
|
||||
|
||||
사용법:
|
||||
1) 엔진: vllm_kwargs.logits_processors: ["ralo.vllm_repetition_abort:RepetitionEosLogitsProcessor"]
|
||||
2) 요청: SamplingParams.extra_args = {"repetition_abort": {
|
||||
"eos_token_id": <int>, # 필수 — 강제할 EOS
|
||||
"ngram": 7, # n-gram 크기 (dapo 필터와 동일 계열)
|
||||
"threshold": 0.9, # 반복 비율 임계 (필터 0.8보다 보수적 기본값 —
|
||||
# 출력이 실제로 잘리는 개입이므로)
|
||||
"min_tokens": 2048, # 이 길이 전에는 검사하지 않음
|
||||
"check_interval": 512, # 검사 주기 (토큰)
|
||||
}}
|
||||
extra_args에 repetition_abort가 없는 요청은 완전히 무시된다.
|
||||
|
||||
주의 (2026-07-16 실측): vLLM V1의 async scheduling(기본 자동 활성화)에서는 워커가
|
||||
output_tok_ids에 실제 토큰 대신 -1 플레이스홀더를 채워 이 프로세서가 무력화된다.
|
||||
반드시 엔진에 `async_scheduling=False`를 함께 전달할 것 (think_format·boxed_eos 공통).
|
||||
"""
|
||||
|
||||
from typing import Optional, Sequence
|
||||
|
||||
import torch
|
||||
|
||||
try:
|
||||
from vllm.sampling_params import SamplingParams
|
||||
from vllm.v1.sample.logits_processor import BatchUpdate, LogitsProcessor
|
||||
from vllm.v1.sample.logits_processor.builtin import process_dict_updates
|
||||
_VLLM_OK = True
|
||||
except ImportError: # vLLM 미설치 환경(트레이너 전용 노드 등)에서도 임포트 가능하게
|
||||
_VLLM_OK = False
|
||||
LogitsProcessor = object # type: ignore
|
||||
BatchUpdate = None # type: ignore
|
||||
|
||||
|
||||
def ngram_repetition_ratio(token_ids: Sequence[int], n: int) -> float:
|
||||
"""반복 n-gram에 속하는 "위치"의 비율 (0..1).
|
||||
|
||||
주의: dapo/sampler.py의 필터 함수와 공식이 다르다. 필터는 "2회 이상 나온 고유
|
||||
n-gram 개수 ÷ 위치 수"인데, 이 정의는 순수 반복 루프에서 오히려 0에 수렴한다
|
||||
(고유 n-gram이 몇 개 안 되므로) — 재보정 주석의 실측치(정상 max 0.58, 폭주
|
||||
0.975)와 부합하는 것은 여기 구현한 "반복 n-gram이 차지하는 위치 비율" 쪽이다.
|
||||
순수 루프 → ~1.0, 무반복 텍스트 → 0.0.
|
||||
"""
|
||||
if n <= 0 or len(token_ids) < n * 2:
|
||||
return 0.0
|
||||
counts: dict = {}
|
||||
total = 0
|
||||
for i in range(len(token_ids) - n + 1):
|
||||
total += 1
|
||||
key = tuple(token_ids[i : i + n])
|
||||
counts[key] = counts.get(key, 0) + 1
|
||||
if total == 0:
|
||||
return 0.0
|
||||
repeated_positions = sum(c for c in counts.values() if c > 1)
|
||||
return repeated_positions / max(total, 1)
|
||||
|
||||
|
||||
class _ReqState:
|
||||
__slots__ = ("out", "eos", "ngram", "threshold", "interval", "next_check", "triggered")
|
||||
|
||||
def __init__(self, out, eos, ngram, threshold, min_tokens, interval):
|
||||
self.out = out # vLLM이 넘겨주는 라이브 출력 토큰 리스트 참조
|
||||
self.eos = int(eos)
|
||||
self.ngram = int(ngram)
|
||||
self.threshold = float(threshold)
|
||||
self.interval = max(1, int(interval))
|
||||
self.next_check = max(1, int(min_tokens))
|
||||
self.triggered = False
|
||||
|
||||
def should_trigger(self) -> bool:
|
||||
"""검사 시점 도달 시 반복 비율 평가. 트리거되면 True (한 번만)."""
|
||||
if self.triggered:
|
||||
return False
|
||||
if len(self.out) < self.next_check:
|
||||
return False
|
||||
ratio = ngram_repetition_ratio(self.out, self.ngram)
|
||||
if ratio >= self.threshold:
|
||||
self.triggered = True
|
||||
return True
|
||||
self.next_check = len(self.out) + self.interval
|
||||
return False
|
||||
|
||||
|
||||
class RepetitionEosLogitsProcessor(LogitsProcessor):
|
||||
"""반복 폭주 감지 시 해당 요청 로짓을 EOS 단일화 — 다른 요청/트리거 전 요청은 무변경."""
|
||||
|
||||
def __init__(self, vllm_config, device: torch.device, is_pin_memory: bool):
|
||||
if not _VLLM_OK:
|
||||
raise RuntimeError("vLLM V1 logits processor API를 찾을 수 없음")
|
||||
self.device = device
|
||||
self.pin_memory = is_pin_memory
|
||||
self.states: dict[int, _ReqState] = {}
|
||||
# 트리거된 (배치 인덱스, eos id) 마스크 텐서
|
||||
self._rows = self._tensor([], torch.int64)
|
||||
self._eos = self._tensor([], torch.int64)
|
||||
self._have_triggered = False
|
||||
|
||||
def _tensor(self, data, dtype):
|
||||
return torch.tensor(data, device="cpu", dtype=dtype, pin_memory=self.pin_memory).to(
|
||||
device=self.device, non_blocking=True
|
||||
)
|
||||
|
||||
def is_argmax_invariant(self) -> bool:
|
||||
return False # 트리거 시 argmax를 EOS로 바꾼다
|
||||
|
||||
@staticmethod
|
||||
def add_request(params: "SamplingParams", _prompt, output_tok_ids) -> Optional[_ReqState]:
|
||||
cfg = (getattr(params, "extra_args", None) or {}).get("repetition_abort")
|
||||
if not cfg or cfg.get("eos_token_id") is None:
|
||||
return None
|
||||
return _ReqState(
|
||||
out=output_tok_ids,
|
||||
eos=cfg["eos_token_id"],
|
||||
ngram=cfg.get("ngram", 7),
|
||||
threshold=cfg.get("threshold", 0.9),
|
||||
min_tokens=cfg.get("min_tokens", 2048),
|
||||
interval=cfg.get("check_interval", 512),
|
||||
)
|
||||
|
||||
def update_state(self, batch_update: "BatchUpdate | None") -> None:
|
||||
changed = process_dict_updates(self.states, batch_update, self.add_request)
|
||||
|
||||
newly = False
|
||||
for state in self.states.values():
|
||||
if state.should_trigger():
|
||||
newly = True
|
||||
|
||||
if changed or newly:
|
||||
rows, eos = [], []
|
||||
for idx, state in self.states.items():
|
||||
if state.triggered:
|
||||
rows.append(idx)
|
||||
eos.append(state.eos)
|
||||
self._rows = self._tensor(rows, torch.int64)
|
||||
self._eos = self._tensor(eos, torch.int64)
|
||||
self._have_triggered = bool(rows)
|
||||
if newly:
|
||||
try:
|
||||
print(f"[RepetitionAbort] {len(rows)} request(s) forced to EOS "
|
||||
f"(runaway n-gram repetition)", flush=True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def apply(self, logits: torch.Tensor) -> torch.Tensor:
|
||||
if self._have_triggered and self._rows.numel() > 0:
|
||||
# 트리거 행 전체 -inf 후 EOS만 0 → softmax에서 EOS 확률 1
|
||||
logits[self._rows] = float("-inf")
|
||||
logits[self._rows, self._eos] = 0.0
|
||||
return logits
|
||||
|
||||
|
||||
def build_repetition_abort_extra_args(algo_cfg: dict, eos_token_id) -> Optional[dict]:
|
||||
"""dapo_kwargs/lrs_kwargs의 repetition_abort 블록 → SamplingParams.extra_args.
|
||||
|
||||
블록이 없거나 enabled가 아니면 None (프로세서가 해당 요청을 완전히 무시).
|
||||
"""
|
||||
cfg = (algo_cfg or {}).get("repetition_abort") or {}
|
||||
if not cfg.get("enabled", False) or eos_token_id is None:
|
||||
return None
|
||||
return {
|
||||
"repetition_abort": {
|
||||
"eos_token_id": int(eos_token_id),
|
||||
"ngram": int(cfg.get("ngram", 7)),
|
||||
"threshold": float(cfg.get("threshold", 0.9)),
|
||||
"min_tokens": int(cfg.get("min_tokens", 2048)),
|
||||
"check_interval": int(cfg.get("check_interval", 512)),
|
||||
}
|
||||
}
|
||||
255
vllm_think_format.py
Normal file
255
vllm_think_format.py
Normal file
@@ -0,0 +1,255 @@
|
||||
"""think 태그 문법 강제 vLLM V1 로짓 프로세서 (2026-07-16).
|
||||
|
||||
근거 (temp0.75 1K×64 전수 실측): 생성의 40.1%가 think를 제대로 못 열고(34.3%는
|
||||
</think>부터 시작), 정상 시작조차 태그를 평균 2.6개 사용(재개방·유사 멀티턴). 태그
|
||||
방향 의미론이 학습되지 않아 성능과 무관하게 형식이 붕괴함 — RL이 이를 그대로 강화
|
||||
하기 전에 문법을 생성 단계에서 강제한다.
|
||||
|
||||
규칙 (토큰 id 상태머신 — 디코드 불필요):
|
||||
1) think가 열려 있으면 <think> 재호출 금지 (중첩/재개방 방지)
|
||||
2) </think>가 1회 등장한 순간부터 <think>·</think> 모두 영구 금지
|
||||
3) (옵션) <|im_start|> 금지 — 새 턴 환각 차단
|
||||
<think>\n 프리필(ralo.custom_prompts.official_chat_think_prefill_prompt_fn)과 결합 시
|
||||
문법이 완전 폐쇄된다: 열림 1회(프리필 보장) + 닫힘 정확히 1회 + 이후 답변부.
|
||||
|
||||
사용법 (boxed_eos/repetition_abort와 병행 등록 가능):
|
||||
1) 엔진: vllm_kwargs.logits_processors: ["ralo.vllm_think_format:ThinkFormatLogitsProcessor"]
|
||||
2) 요청: SamplingParams.extra_args = {"think_format": {
|
||||
"think_open_id": <int>, # <think> 토큰 id
|
||||
"think_close_id": <int>, # </think> 토큰 id
|
||||
"prefilled_open": true, # 프롬프트가 <think>로 끝나는 경우 (프리필)
|
||||
"ban_im_start": true, # <|im_start|> 재호출 금지 (옵션)
|
||||
"im_start_id": <int>,
|
||||
}}
|
||||
extra_args에 think_format이 없는 요청은 완전히 무시된다.
|
||||
|
||||
주의 — async scheduling 비호환 (2026-07-16 실측): vLLM V1은 async scheduling을 기본
|
||||
자동 활성화하는데, 이때 워커가 output_tok_ids에 실제 토큰 대신 -1 플레이스홀더를
|
||||
채운다(gpu_model_runner의 use_async_scheduling 경로). 출력 토큰 값을 읽는 커스텀
|
||||
프로세서(이 파일 + vllm_boxed_eos + vllm_repetition_abort)는 전부 무력화된다.
|
||||
반드시 엔진에 `async_scheduling=False`를 함께 줘야 한다. 플레이스홀더가 감지되면
|
||||
아래 상태머신이 1회 경고를 남긴다.
|
||||
"""
|
||||
|
||||
import logging
|
||||
|
||||
from typing import Optional
|
||||
|
||||
import torch
|
||||
|
||||
try:
|
||||
from vllm.sampling_params import SamplingParams
|
||||
from vllm.v1.sample.logits_processor import BatchUpdate, LogitsProcessor
|
||||
from vllm.v1.sample.logits_processor.builtin import process_dict_updates
|
||||
_VLLM_OK = True
|
||||
except ImportError:
|
||||
_VLLM_OK = False
|
||||
LogitsProcessor = object # type: ignore
|
||||
BatchUpdate = None # type: ignore
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
_warned_placeholder = False
|
||||
|
||||
|
||||
class ThinkFormatState:
|
||||
"""토큰 id만으로 금지 목록을 결정하는 상태머신 (HF/vLLM 공용 코어)."""
|
||||
|
||||
__slots__ = ("open_id", "close_id", "im_start_id", "opened", "closed", "consumed",
|
||||
"out", "max_think_tokens", "think_len", "force_prefill",
|
||||
"force_whitelist", "_force_start")
|
||||
|
||||
def __init__(self, out, open_id, close_id, prefilled_open=False, im_start_id=None,
|
||||
prefilled_closed=False, max_think_tokens=None,
|
||||
force_prefill=None, force_whitelist=None):
|
||||
self.out = out # 생성 토큰 리스트 (vLLM 라이브 참조 / HF에선 수동 feed)
|
||||
self.open_id = int(open_id)
|
||||
self.close_id = int(close_id)
|
||||
self.im_start_id = int(im_start_id) if im_start_id is not None else None
|
||||
# prefilled_closed: 프롬프트(프리픽스)에 이미 <think>…</think>가 완결되어 있는 경우
|
||||
# (예: lrs/MCMC 청크 재개 — 누적 텍스트를 프롬프트로 넘기는 후속 요청). 이걸 안 주면
|
||||
# 요청마다 상태가 리셋되어 닫힌 뒤에도 </think> 재방출이 허용된다 (2026-07-21 실측).
|
||||
self.opened = bool(prefilled_open) or bool(prefilled_closed)
|
||||
self.closed = bool(prefilled_closed)
|
||||
self.consumed = 0
|
||||
# max_think_tokens: <think> 안에서 이 토큰 수를 넘으면 강제 봉합
|
||||
# (non-convergent 추론 루프 탈출 → 답변 단계로 밀어냄). None이면 비활성.
|
||||
self.max_think_tokens = int(max_think_tokens) if max_think_tokens else None
|
||||
self.think_len = 0 # <think> 열린 뒤 생성된 토큰 수
|
||||
# 강제 봉합 프리필 시퀀스(예: [</think>, "\n\n"])를 순서대로 강제한 뒤,
|
||||
# 첫 답변 토큰은 force_whitelist(학습데이터 top-N 답변시작 토큰)로만 허용 →
|
||||
# 모델이 그중 최고를 스스로 고르게. 둘 다 없으면 </think> 하나만 강제(구 동작).
|
||||
self.force_prefill = [int(x) for x in force_prefill] if force_prefill else [self.close_id]
|
||||
self.force_whitelist = [int(x) for x in force_whitelist] if force_whitelist else None
|
||||
self._force_start = None # 강제 봉합 시작 시점의 out 길이
|
||||
|
||||
def advance(self):
|
||||
"""새 토큰 소비 후 현재 시점의 금지 토큰 id 리스트 반환."""
|
||||
global _warned_placeholder
|
||||
while self.consumed < len(self.out):
|
||||
t = self.out[self.consumed]
|
||||
self.consumed += 1
|
||||
if t == -1 and not _warned_placeholder:
|
||||
_warned_placeholder = True
|
||||
logger.warning(
|
||||
"[ThinkFormat] output_tok_ids에 -1 플레이스홀더 감지 — vLLM async "
|
||||
"scheduling이 켜져 있어 닫힘 감지가 불가능합니다. 엔진에 "
|
||||
"async_scheduling=False를 전달하세요.")
|
||||
if self.opened and not self.closed:
|
||||
self.think_len += 1
|
||||
if t == self.open_id:
|
||||
self.opened = True
|
||||
elif t == self.close_id and self.opened:
|
||||
self.closed = True
|
||||
return self.banned_ids()
|
||||
|
||||
def force_allowed_ids(self):
|
||||
"""강제 봉합 진행 중이면 이번 스텝 허용 토큰 id 리스트, 아니면 None.
|
||||
|
||||
프리필 시퀀스를 순서대로 1개씩 강제 → 끝나면 답변 첫 토큰을 화이트리스트로
|
||||
1스텝 제한 → 그 뒤 해제(None). 진입 후 self.closed가 True가 돼도 _force_start
|
||||
기준으로 계속 진행한다."""
|
||||
if self.max_think_tokens is None:
|
||||
return None
|
||||
if self._force_start is None:
|
||||
if (self.opened and not self.closed
|
||||
and self.think_len >= self.max_think_tokens):
|
||||
self._force_start = len(self.out) # 강제 봉합 개시
|
||||
else:
|
||||
return None
|
||||
progress = len(self.out) - self._force_start
|
||||
seq = self.force_prefill
|
||||
if progress < len(seq):
|
||||
return [seq[progress]] # 프리필: 그 자리 토큰만 허용
|
||||
if self.force_whitelist and progress == len(seq):
|
||||
return list(self.force_whitelist) # 답변 첫 토큰: top-N만 허용
|
||||
return None # 프리필+화이트리스트 끝 → 해제
|
||||
|
||||
def banned_ids(self):
|
||||
banned = []
|
||||
if self.closed:
|
||||
banned = [self.open_id, self.close_id] # 닫힌 후엔 둘 다 영구 금지
|
||||
elif self.opened:
|
||||
banned = [self.open_id] # 열려 있는 동안 재개방 금지
|
||||
if self.im_start_id is not None:
|
||||
banned.append(self.im_start_id)
|
||||
return banned
|
||||
|
||||
|
||||
class ThinkFormatLogitsProcessor(LogitsProcessor):
|
||||
"""think 태그 문법 강제 — 금지 토큰 로짓만 -inf, 그 외 무변경."""
|
||||
|
||||
def __init__(self, vllm_config, device: torch.device, is_pin_memory: bool):
|
||||
if not _VLLM_OK:
|
||||
raise RuntimeError("vLLM V1 logits processor API를 찾을 수 없음")
|
||||
self.device = device
|
||||
self.pin_memory = is_pin_memory
|
||||
self.states: dict[int, ThinkFormatState] = {}
|
||||
self._rows: list[int] = []
|
||||
self._cols: list[int] = []
|
||||
self._rows_t = None
|
||||
self._cols_t = None
|
||||
# 강제 봉합: force_rows는 전체 -inf, (allow_rows, allow_cols)만 0으로 살림
|
||||
self._force_rows_t = None
|
||||
self._allow_rows_t = None
|
||||
self._allow_cols_t = None
|
||||
|
||||
def is_argmax_invariant(self) -> bool:
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def add_request(params: "SamplingParams", _prompt, output_tok_ids) -> Optional[ThinkFormatState]:
|
||||
cfg = (getattr(params, "extra_args", None) or {}).get("think_format")
|
||||
if not cfg or cfg.get("think_open_id") is None or cfg.get("think_close_id") is None:
|
||||
return None
|
||||
return ThinkFormatState(
|
||||
out=output_tok_ids,
|
||||
open_id=cfg["think_open_id"],
|
||||
close_id=cfg["think_close_id"],
|
||||
prefilled_open=bool(cfg.get("prefilled_open", False)),
|
||||
prefilled_closed=bool(cfg.get("prefilled_closed", False)),
|
||||
im_start_id=cfg.get("im_start_id") if cfg.get("ban_im_start", False) else None,
|
||||
max_think_tokens=cfg.get("max_think_tokens"),
|
||||
force_prefill=cfg.get("force_prefill"),
|
||||
force_whitelist=cfg.get("force_whitelist"),
|
||||
)
|
||||
|
||||
def update_state(self, batch_update: "BatchUpdate | None") -> None:
|
||||
process_dict_updates(self.states, batch_update, self.add_request)
|
||||
rows, cols = [], [] # 일반 금지(-inf)
|
||||
force_rows, allow_rows, allow_cols = [], [], [] # 강제: 행 전체 -inf 후 allow만 0
|
||||
for idx, st in self.states.items():
|
||||
banned = st.advance()
|
||||
allowed = st.force_allowed_ids()
|
||||
if allowed is not None:
|
||||
force_rows.append(idx)
|
||||
for tid in allowed:
|
||||
allow_rows.append(idx)
|
||||
allow_cols.append(tid)
|
||||
else:
|
||||
for tid in banned:
|
||||
rows.append(idx)
|
||||
cols.append(tid)
|
||||
|
||||
def _t(vals):
|
||||
return torch.tensor(vals, device="cpu", dtype=torch.int64,
|
||||
pin_memory=self.pin_memory).to(self.device, non_blocking=True)
|
||||
self._rows_t, self._cols_t = (_t(rows), _t(cols)) if rows else (None, None)
|
||||
if force_rows:
|
||||
self._force_rows_t = _t(force_rows)
|
||||
self._allow_rows_t = _t(allow_rows)
|
||||
self._allow_cols_t = _t(allow_cols)
|
||||
else:
|
||||
self._force_rows_t = self._allow_rows_t = self._allow_cols_t = None
|
||||
|
||||
def apply(self, logits: torch.Tensor) -> torch.Tensor:
|
||||
if self._rows_t is not None:
|
||||
logits[self._rows_t, self._cols_t] = float("-inf")
|
||||
if self._force_rows_t is not None:
|
||||
# 강제 봉합: 해당 행 전체 -inf 후 허용 토큰만 0 (프리필=1개, 화이트리스트=N개)
|
||||
logits[self._force_rows_t] = float("-inf")
|
||||
logits[self._allow_rows_t, self._allow_cols_t] = 0.0
|
||||
return logits
|
||||
|
||||
|
||||
def build_think_format_extra_args(algo_cfg: dict, tokenizer,
|
||||
prefilled_open: bool = False) -> Optional[dict]:
|
||||
"""dapo_kwargs/lrs_kwargs의 think_format 블록 → SamplingParams.extra_args."""
|
||||
cfg = (algo_cfg or {}).get("think_format") or {}
|
||||
if not cfg.get("enabled", False):
|
||||
return None
|
||||
open_id = tokenizer.convert_tokens_to_ids("<think>")
|
||||
close_id = tokenizer.convert_tokens_to_ids("</think>")
|
||||
im_start_id = tokenizer.convert_tokens_to_ids("<|im_start|>")
|
||||
if open_id is None or close_id is None:
|
||||
return None
|
||||
|
||||
# 강제 봉합 프리필: </think> + "\n\n" 시퀀스 후, 답변 첫 토큰을 학습데이터
|
||||
# top-N 답변시작 토큰(화이트리스트)으로만 허용해 모델이 스스로 고르게 한다.
|
||||
# (2026-07-21 amq3 clean 292K 실측: </think> 뒤 100% "\n\n", 첫 실토큰 top20이 97%)
|
||||
force_prefill = force_whitelist = None
|
||||
if cfg.get("force_close_prefill"):
|
||||
nn = tokenizer("\n\n", add_special_tokens=False)["input_ids"]
|
||||
force_prefill = [int(close_id)] + [int(x) for x in nn]
|
||||
# config에서 직접 준 화이트리스트(id 리스트) 우선, 없으면 학습데이터 기반 기본값.
|
||||
# 사람이름(John/Mary/James/Maria)·단일문자는 제외 — 답변 요약이 아니라 스토리
|
||||
# 문제 재서술로 빠질 수 있어서. 일반 답변시작 토큰만 (커버리지 ~96%):
|
||||
# To/We/Let/The/Given/###/(/First/In
|
||||
force_whitelist = cfg.get("force_close_whitelist") or [
|
||||
1249, 1654, 10061, 785, 22043, 14374, 7, 5338, 641,
|
||||
]
|
||||
force_whitelist = [int(x) for x in force_whitelist]
|
||||
|
||||
return {
|
||||
"think_format": {
|
||||
"think_open_id": int(open_id),
|
||||
"think_close_id": int(close_id),
|
||||
"prefilled_open": bool(cfg.get("prefilled_open", prefilled_open)),
|
||||
"prefilled_closed": bool(cfg.get("prefilled_closed", False)),
|
||||
"ban_im_start": bool(cfg.get("ban_im_start", True)),
|
||||
"im_start_id": int(im_start_id) if im_start_id is not None else None,
|
||||
"max_think_tokens": (int(cfg["max_think_tokens"])
|
||||
if cfg.get("max_think_tokens") else None),
|
||||
"force_prefill": force_prefill,
|
||||
"force_whitelist": force_whitelist,
|
||||
}
|
||||
}
|
||||
1
vocab.json
Normal file
1
vocab.json
Normal file
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user