初始化项目，由ModelHub XC社区提供模型

Model: Lzvick/qwen-1.7b-math-reasoner-grpo
Source: Original Platform

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,162 @@
+---
+license: apache-2.0
+base_model: unsloth/Qwen3-1.7B-Base
+tags:
+  - grpo
+  - reinforcement-learning
+  - reasoning
+  - math
+  - rlvr
+  - qwen3
+datasets:
+  - openai/gsm8k
+language:
+  - en
+pipeline_tag: text-generation
+---
+
+# Qwen3-1.7B-GRPO-GSM8K (v1)
+
+A GRPO (Group Relative Policy Optimization) fine-tune of `unsloth/Qwen3-1.7B-Base`,
+trained with verifiable rewards (RLVR) on GSM8K math word problems for one full epoch.
+
+## Model description
+
+- **Base model:** `unsloth/Qwen3-1.7B-Base` (Apache 2.0)
+- **Method:** GRPO (DeepSeekMath / DeepSeek-R1 style), via Unsloth + TRL's `GRPOTrainer`
+- **Adapter:** LoRA rank 32 / alpha 32, merged into the base weights for this release
+- **Training data:** GSM8K `train`+`test` combined into one 8,792-example pool, shuffled
+  once (seed 3407), split 80/10/10 — trained on the 80% partition (7,033 examples) for
+  1 full epoch. Validation (10%) used only for monitoring during training; the eval
+  partition (10%, 880 examples) was never touched until the final evaluation below.
+- **Output format:** the model is trained to respond in a fixed structure:
+  ```
+  <reasoning>
+  ...
+  </reasoning>
+  <answer>
+  ...
+  </answer>
+  ```
+- **Reward functions:** correctness (regex-extracted `<answer>` content vs. GSM8K ground
+  truth, +2.0), format compliance (+0.5 well-formed / +0.25 partial), brevity
+  (+0.2 max, capped small so it can never outweigh correctness or format)
+
+## Evaluation results
+
+All numbers below are from held-out data the model never saw during training or any
+prior evaluation pass (see the source repo's `week-05-06/scripts/data_split.py` for the
+split methodology, and `week-05-06/docs/04-results.md` for the full writeup including
+two scoring bugs that were caught and fixed mid-evaluation).
+
+| Dataset | In/out of distribution | Base model | This model | Change |
+|---|---|---|---|---|
+| GSM8K (880 held-out examples) | in-distribution | 66.9%\* | **87.3%** | +20.4pp |
+| SVAMP (300 examples) | out-of-distribution | 76.7% | **86.7%** | +10.0pp |
+| MATH / algebra (300 examples) | out-of-distribution, harder | 49.0%\*\* | **60.3%** | +11.3pp\*\* |
+
+\*Base model accuracy is scored leniently (correct number anywhere in the response) since
+the base model has no tendency to use the `<answer>` tag format at all (0.6% format
+compliance) — this model's own format compliance is 99.4%.
+
+\*\*MATH's numbers are a **lower bound**, not a precise score — grading MATH answers
+exactly (fractions, intervals, symbolic expressions) without a computer algebra system is
+a known hard problem; some correct-but-differently-formatted answers score as incorrect
+here.
+
+**Manual transcript review** (not just the aggregate numbers) found no evidence of
+reward hacking: correct answers show genuine multi-step arithmetic, incorrect answers
+are real reasoning mistakes on harder problems (not degenerate or gamed output), and
+response length does not correlate with correctness in a gaming direction.
+
+### Independent cross-check: lm-evaluation-harness (base vs. trained)
+
+The numbers above use this project's own zero-shot, custom-prompt eval scripts. Cross-
+checked independently via EleutherAI's `lm-evaluation-harness` (standard 5-shot GSM8K
+config, plus MMLU and HellaSwag as general-capability anchors), run identically against
+both the base model and this one — see the source repo's `week-07/docs/04-results.md`
+for the full writeup.
+
+| Benchmark | Base | This model | Δ |
+|---|---|---|---|
+| GSM8K (5-shot, strict-match) | 67.55% | **77.03%** | +9.5pp |
+| GSM8K (5-shot, flexible-extract) | 71.42% | **77.79%** | +6.4pp |
+| MMLU (57-subject average) | 60.59% | 60.56% | −0.03pp |
+| HellaSwag (acc) | 49.28% | 49.24% | −0.04pp |
+| HellaSwag (acc_norm) | 66.48% | 66.78% | +0.30pp |
+
+**No catastrophic forgetting.** The MMLU and HellaSwag deltas are smaller than each
+benchmark's own standard error (~0.4-0.5pp) — statistically indistinguishable from no
+change. GSM8K improved well outside that noise band, via a completely different
+methodology (5-shot, standard harness prompt format) than this project's own eval —
+independent confirmation that the math-reasoning gain is real, not an artifact of this
+project's own eval design, and that it wasn't purchased at the cost of general
+capability.
+
+## Limitations
+
+- **Narrow, math-specialized training.** This model was trained exclusively on grade-
+  school arithmetic word problems (GSM8K). General instruction-following and chat
+  quality specifically were not evaluated (this model was never trained on
+  conversational data) — though catastrophic forgetting of general *knowledge/reasoning*
+  capability was checked (see the benchmark cross-check above) and not found.
+- **A real hint-exploitation vulnerability was found, quantified at 37%.** A formal
+  hint/no-hint probe (200 held-out problems, each run with a correct hint and a
+  deliberately wrong decoy — see `week-07/docs/04-results.md` for the full method) found
+  that 37% of the time, a plausible-but-wrong hint changes the model's final answer away
+  from what it would otherwise get right. In one documented example, the model correctly
+  re-derived the right answer three separate times, explicitly noted the conflict with
+  the hint, and still deferred to the hint over its own confirmed work. **Do not present
+  this model with untrusted "hint" or "suggested answer" text in the prompt** without
+  accounting for this - it will sometimes follow a wrong suggestion over independently-
+  verified correct reasoning. (Checked and ruled out as *reward hacking* specifically -
+  training prompts never included hints at all, so the reward function never selected
+  for this behavior one way or the other; it's most likely inherited from base-model
+  pretraining and was simply never tested or corrected during RL training.)
+- **MATH evaluation numbers are approximate** (see above) — treat the MATH result
+  directionally, not as a precise benchmark score.
+- **Single training run.** No hyperparameter ablation or multi-seed variance check was
+  performed; this is a v1 release, not a fully tuned final model.
+- **Format compliance is 99.4%, not 100%** — a small fraction of outputs don't follow the
+  `<reasoning>`/`<answer>` structure.
+- **The model does not always stop generating after `</answer>` — measured at ~0.07%
+  of outputs, rare rather than common.** No reward function ever scored anything past
+  the first `<answer>` tag match, so nothing in training penalized rambling afterward —
+  a real design gap. Measured directly against 1,451 raw eval transcripts (GSM8K +
+  SVAMP + MATH), only 1 case had any content past the tag, and it wasn't even the
+  fabricated-dialogue pattern (it was a repetitive reasoning loop that leaked one stray
+  character). The one case that clearly showed the fabricated-dialogue version happened
+  during manual testing on a far-out-of-distribution question style — possibly the same
+  underlying cause as the function-composition gap below, not a frequent, independent
+  problem. Regardless, the fix is free and worth applying — **set a stop sequence at
+  `</answer>` when generating**:
+  ```python
+  # vLLM
+  SamplingParams(stop=["</answer>"], include_stop_str_in_output=True)
+
+  # transformers .generate()
+  from transformers import StopStringCriteria, StoppingCriteriaList
+  model.generate(..., stopping_criteria=StoppingCriteriaList(
+      [StopStringCriteria(tokenizer, ["</answer>"])]
+  ))
+  ```
+- **Symbolic function-composition problems (`f(x)=...`, `g(x)=...` notation) appear to be
+  a specific weak spot**, distinct from the general MATH miss rate — likely because
+  GSM8K (the only training data) is exclusively concrete word problems with no examples
+  of this abstract notation. Observed anecdotally (4/4 wrong on manual testing), not
+  verified at benchmark scale — flagged as a real, specific gap rather than confirmed
+  precisely.
+
+## Training details
+
+- Framework: Unsloth 2026.5.9 + TRL `GRPOTrainer`, run inside the `unsloth/unsloth`
+  Docker image on an RTX 5070 (12GB, Blackwell sm_120)
+- Precision: bf16 (FP8 was evaluated and found broken on this specific software stack —
+  see the source repo's `week-05-06/docs/00-overview.md` for the investigation)
+- `num_generations=4`, `temperature=0.8`, `beta=0.001` (KL penalty), `loss_type=dapo`
+- Learning rate 5e-6, 1 epoch, ~7,033 steps, wall-clock ~7.4 hours
+
+## Citation / provenance
+
+Trained as part of a self-directed practice roadmap covering post-training/RL, systems,
+and scalability topics for LLMs.

added_tokens.json

@@ -0,0 +1,28 @@
+{
+  "</think>": 151668,
+  "</tool_call>": 151658,
+  "</tool_response>": 151666,
+  "<think>": 151667,
+  "<tool_call>": 151657,
+  "<tool_response>": 151665,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

config.json

@@ -0,0 +1,61 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": 151643,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 28,
+  "model_type": "qwen3",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 151654,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.6",
+  "unsloth_fixed": true,
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

generation_config.json

@@ -0,0 +1,8 @@
+{
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "max_length": 32768,
+  "max_new_tokens": 2048,
+  "pad_token_id": 151654,
+  "transformers_version": "4.57.6"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:98f552dc6e75a7c6e1fa6d61bba719dc87709f949496dc7d4ccd5d6bab7842e0
+size 3441185608

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,240 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151665": {
+      "content": "<tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151666": {
+      "content": "</tool_response>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151667": {
+      "content": "<think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151668": {
+      "content": "</think>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 32768,
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "left",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}