初始化项目，由ModelHub XC社区提供模型

Model: sfanm/d24-sft-v5-nogsm8k
Source: Original Platform

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,201 @@
+---
+license: other
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+base_model: ftajwar/d24-climbmix-dolmino-midtrain-100b
+datasets:
+- sfanm/d24-sft-mixture
+tags:
+- llama
+- causal-lm
+- chat
+- sft
+- rlvr
+- gsm8k-free-sft
+- contamination-control
+- gpt2-tokenizer
+---
+
+# d24-sft-v5-nogsm8k
+
+`d24-sft-v5-nogsm8k` is an experimental **756,819,456-parameter**
+English chat model intended as a clean SFT-exposure control for GSM8K
+reinforcement-learning experiments. Its supervised fine-tuning mixture omits
+the complete 7,473-example GSM8K training component.
+
+All 756,819,456 parameters in the published safetensors file are BF16.
+
+> **Scope of the no-GSM8K claim:** the explicit `openai/gsm8k` component was
+> removed from supervised fine-tuning. This is not a claim that the much larger
+> pretraining and midtraining corpora contain no GSM8K overlap, paraphrases, or
+> similar grade-school mathematics. Other non-GSM8K SFT sources can also contain
+> mathematical material.
+
+The standard-mixture sibling, which did include GSM8K during SFT, is
+[`sfanm/d24-sft-v5-simplechat`](https://huggingface.co/sfanm/d24-sft-v5-simplechat).
+
+## Lineage
+
+1. [`ftajwar/d24-climbmix-100b`](https://huggingface.co/ftajwar/d24-climbmix-100b):
+   pretrained from scratch on about 100B tokens of ClimbMix.
+2. [`ftajwar/d24-climbmix-dolmino-midtrain-100b`](https://huggingface.co/ftajwar/d24-climbmix-dolmino-midtrain-100b):
+   continued-pretrained for another approximately 100B tokens on 80% OLMo-3
+   Dolmino plus 20% ClimbMix replay.
+3. This checkpoint: one epoch of SFT on the d24 conversation mixture after
+   removing the GSM8K component, formatted with the simple ChatML template.
+
+The model has seen approximately 200B pretraining/midtraining tokens before SFT.
+The `v5` label identifies this training lineage; it is not a parameter-count
+label.
+
+## What was removed
+
+The no-GSM8K corpus was derived from the standard
+[`sfanm/d24-sft-mixture`](https://huggingface.co/datasets/sfanm/d24-sft-mixture)
+source mixture. The standard and filtered simple-ChatML preparation summaries
+differ by exactly 7,473 source conversations, the size of the GSM8K training
+split used by the standard mixture:
+
+| Preparation count | Standard mixture | No-GSM8K mixture | Difference |
+|---|---:|---:|---:|
+| Training source conversations | 821,884 | 814,644 | 7,240 |
+| Validation source conversations | 26,514 | 26,281 | 233 |
+| Total source conversations | 848,398 | 840,925 | **7,473** |
+| Packed training sequences | 227,443 | 226,879 | 564 |
+| Packed validation sequences | 7,381 | 7,363 | 18 |
+| Packed tokens | 476,320,096 | 475,114,114 | 1,205,982 |
+
+Both preparations used sequence length 2,048, `first_fit_shuffle`, seed 42,
+the GPT-2 tokenizer, and the same simple ChatML template. Removing source
+conversations changes packing boundaries, so packed-sequence differences are
+not expected to equal the source-conversation difference.
+
+## Architecture
+
+| Field | Value |
+|---|---:|
+| Parameters | 756,819,456 |
+| Layers | 24 |
+| Hidden size | 1,536 |
+| Attention heads | 12 (MHA) |
+| FFN size | 4,096 (SwiGLU/SiLU) |
+| Position encoding | RoPE, theta 10,000 |
+| Normalization | RMSNorm |
+| Embeddings | Tied |
+| Tokenizer | GPT-2 BPE, vocabulary padded to 50,304 |
+| Maximum context | 2,048 tokens |
+| Published weight dtype | BF16 |
+
+The Hugging Face architecture is `LlamaForCausalLM`, but the model uses the
+GPT-2 tokenizer and the d24 architecture above. It is not a Llama-family
+pretrained checkpoint.
+
+## SFT training
+
+| Hyperparameter | Value |
+|---|---:|
+| Epochs / steps | 1 / 1,773 |
+| Global / micro batch | 128 / 1 |
+| Sequence length | 2,048 |
+| Optimizer | AdamW |
+| Peak / minimum LR | 1e-4 / 1e-5 |
+| LR schedule | Cosine, 50-step warmup |
+| Adam betas / epsilon | (0.9, 0.95) / 1e-8 |
+| Weight decay | 0.1 |
+| Gradient clip | 1.0 |
+| Precision | BF16 |
+
+The final SFT validation language-model loss was **0.797625** (perplexity
+**2.220262**). This is an in-distribution SFT validation metric, not a
+downstream capability or safety evaluation.
+
+## Chat format and stopping
+
+The template is:
+
+```text
+<|im_start|>user
+...<|im_end|>
+<|im_start|>assistant
+...
+```
+
+`<|im_end|>` is a literal string represented by ordinary GPT-2 BPE tokens. It
+is **not** `eos_token_id=50256` and is not a registered special token.
+Generation must stop on the string `<|im_end|>`; otherwise the model can
+continue into another turn until `max_new_tokens`.
+
+## Transformers usage
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "sfanm/d24-sft-v5-nogsm8k"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    dtype=torch.bfloat16,
+    device_map="auto",
+).eval()
+
+messages = [
+    {
+        "role": "user",
+        "content": "Natalia sold clips to 48 friends in April and half as many in May. How many total?",
+    }
+]
+prompt = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+output = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    do_sample=False,
+    stop_strings=["<|im_end|>"],
+    tokenizer=tokenizer,
+)
+response = tokenizer.decode(
+    output[0, inputs.input_ids.shape[1]:],
+    skip_special_tokens=False,
+)
+print(response)
+```
+
+For vLLM, use `SamplingParams(stop=["<|im_end|>"])`.
+
+## Intended use
+
+- controlled studies of learning GSM8K through RLVR without explicit GSM8K
+  examples during SFT;
+- comparisons against the GSM8K-exposed v5 simplechat sibling;
+- research on GRPO, RLOO, maxRL, SFT, and small-model behavior.
+
+## Limitations
+
+- The no-GSM8K designation applies only to the explicit SFT mixture component;
+  it does not prove global benchmark decontamination across pretraining and
+  midtraining.
+- This is a small experimental research model and has not undergone production
+  safety alignment or a comprehensive capability/safety evaluation.
+- It can produce incorrect, incoherent, biased, or unsafe text. Verify outputs
+  independently and do not use it for high-stakes decisions.
+- The context window is 2,048 tokens.
+- Chat generation is incorrect unless the caller stops on the literal
+  `<|im_end|>` string.
+- The model and SFT mixture derive from multiple upstream datasets. The Hub
+  metadata therefore uses `license: other`; review source licenses and terms
+  before redistribution or downstream use.
+
+## Reproducibility
+
+- SFT checkpoint: `sft-d24-v5-nogsm8k/iter_0001773`
+- HF export: `d24-sft-v5-nogsm8k-hf`
+- Packed corpus: `sft-pack-d24-nogsm8k`
+- Chat template: `src/nemotron/data_prep/templates/simple_chatml.jinja`
+- Training repository: `yudasong/unified-training`

chat_template.jinja

@@ -0,0 +1 @@
+{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{'<|im_start|>assistant\n'}}{% endif %}

config.json

@@ -0,0 +1,29 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50256,
+  "dtype": "float32",
+  "eos_token_id": 50256,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 1536,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "max_position_embeddings": 2048,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 12,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": true,
+  "transformers_version": "4.57.1",
+  "use_cache": true,
+  "vocab_size": 50304
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 50256,
+  "eos_token_id": 50256,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "transformers_version": "5.3.0",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2572d537272bcdba950e4c81468876cd46ace3fe549c3a64aba5f77f415ae7bd
+size 1513663904

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<|endoftext|>",
+  "cls_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "is_local": false,
+  "model_max_length": 2048,
+  "pad_token": null,
+  "sep_token": "<|endoftext|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}