初始化项目，由ModelHub XC社区提供模型

Model: bfpill/gemma-3-270m-sft-skywork-chosen Source: Original Platform
2026-06-03 08:55:17 +08:00
commit 0d9ba56b12
7 changed files with 247 additions and 0 deletions
--- a/.gitattributes
+++ b/.gitattributes
@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
--- a/README.md
+++ b/README.md
@@ -0,0 +1,107 @@
+---
+library_name: transformers
+base_model: google/gemma-3-270m
+license: gemma
+datasets:
+  - Skywork/Skywork-Reward-Preference-80K-v0.2
+language:
+  - en
+tags:
+  - gemma3
+  - sft
+  - research
+  - skywork
+pipeline_tag: text-generation
+---
+
+# gemma-3-270m SFT'd on Skywork-Reward-Preference-80K (chosen-only)
+
+A response-only SFT of [`google/gemma-3-270m`](https://huggingface.co/google/gemma-3-270m)
+on the **chosen** responses of
+[`Skywork/Skywork-Reward-Preference-80K-v0.2`](https://huggingface.co/datasets/Skywork/Skywork-Reward-Preference-80K-v0.2).
+
+This is the anchor checkpoint used as the SFT reference policy for a research
+project on **data-space DPO** — applying preference signal by reweighting SFT
+samples through a susceptibility matrix, instead of via DPO's weight-space
+gradient descent. It is **not** intended as a useful chat model — at 270M
+parameters and a single epoch of SFT, generations are at best "coherent text
+on the topic of the prompt". The role of this checkpoint is to be a
+controlled, reproducible anchor for sampling-based experiments.
+
+## Training recipe
+
+| | |
+|---|---|
+| Base | `google/gemma-3-270m` (pretrained, no chat template) |
+| Data | `Skywork/Skywork-Reward-Preference-80K-v0.2`, chosen responses only (~77K examples) |
+| Epochs | 1 |
+| Learning rate | 2e-5, cosine schedule, 3% warmup |
+| Batch size | 16 per device × 4 GPUs = 64 effective |
+| Precision | bf16 + gradient checkpointing |
+| Loss | response-only cross-entropy (prompt tokens masked to -100) |
+| Total steps | 602 |
+| Final train loss | 1.41 |
+| Wall time | ~4 min on 8× H100 |
+
+## Prompt format
+
+The base model has no `chat_template`, so we use a fixed plain-text format
+matching the SFT distribution:
+
+```text
+Human: <user message>
+Assistant: ASSISTANT_RESPONSE
+```
+
+Internally the prompt prefix is `"Human: <user>\n\nAssistant:"` and the loss is
+computed only on the tokens following `Assistant:`.
+
+## Quick start
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+mid = "bfpill/gemma-3-270m-sft-skywork-chosen"
+tok = AutoTokenizer.from_pretrained(mid)
+model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype=torch.bfloat16).cuda().eval()
+
+prompt = f"Human: What is the capital of France?\n\nAssistant:"
+ids = tok(prompt, return_tensors="pt").to(model.device)
+out = model.generate(
+    **ids, max_new_tokens=80, do_sample=False,
+    repetition_penalty=1.15, pad_token_id=tok.eos_token_id,
+)
+print(tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True))
+```
+
+## Limitations
+
+- 270M params, 1 epoch of SFT on ~77K examples. Generations are short and
+  often factually incorrect.
+- Trained exclusively on the Skywork prompt distribution; performance on
+  other distributions (instruction-following, dialogue, code, math)
+  reflects the base model plus a thin Skywork-flavoured veneer.
+- Not preference-tuned. The whole point of this checkpoint is to be the
+  anchor for a follow-up data-space patterning experiment that is the
+  *replacement* for the usual DPO step.
+- Inherits the base model's biases and license restrictions.
+
+## License
+
+Gemma Terms of Use — see [Google's Gemma license](https://ai.google.dev/gemma/terms).
+This SFT preserves the base model's weights structure; usage is subject to
+the same terms.
+
+## Source code
+
+The trainer that produced this checkpoint:
+`rm-patterning/policy/sft_anchor.py` in
+`timaeus-research/timaeus@max/rm-scaling` (commit `3a3e0755e` and later).
+
+```bash
+torchrun --nproc_per_node=8 rm-patterning/policy/sft_anchor.py \
+    --base-model google/gemma-3-270m \
+    --output-dir ./gemma-3-270m-sft-skywork-chosen \
+    --epochs 1 --learning-rate 2e-5 --batch-size 16 --max-length 1024
+```
--- a/config.json
+++ b/config.json
@@ -0,0 +1,62 @@
+{
+  "_sliding_window_pattern": 6,
+  "architectures": [
+    "Gemma3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attn_logit_softcapping": null,
+  "bos_token_id": 2,
+  "dtype": "bfloat16",
+  "eos_token_id": 1,
+  "final_logit_softcapping": null,
+  "head_dim": 256,
+  "hidden_activation": "gelu_pytorch_tanh",
+  "hidden_size": 640,
+  "initializer_range": 0.02,
+  "intermediate_size": 2048,
+  "layer_types": [
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "model_type": "gemma3_text",
+  "num_attention_heads": 4,
+  "num_hidden_layers": 18,
+  "num_key_value_heads": 1,
+  "pad_token_id": 0,
+  "query_pre_attn_scalar": 256,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 1000000.0,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "sliding_window": 512,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.2.0",
+  "use_bidirectional_attention": false,
+  "use_cache": false,
+  "vocab_size": 262144
+}
--- a/generation_config.json
+++ b/generation_config.json
@@ -0,0 +1,12 @@
+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "5.2.0"
+}
--- a/model.safetensors
+++ b/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96fa31398b1277b7614c99e9c4aa14e509492dd33fc1816d81cc063aa0d7d7cd
+size 536223056
--- a/tokenizer.json
+++ b/tokenizer.json
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a74aefb1dc1340a25f29ab8370384b9ed24b2d921d7749ece7bbcfcfdf00d497
+size 33384443
--- a/tokenizer_config.json
+++ b/tokenizer_config.json
@@ -0,0 +1,24 @@
+{
+  "backend": "tokenizers",
+  "boi_token": "<start_of_image>",
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "eoi_token": "<end_of_image>",
+  "eos_token": "<eos>",
+  "image_token": "<image_soft_token>",
+  "is_local": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "boi_token": "<start_of_image>",
+    "eoi_token": "<end_of_image>",
+    "image_token": "<image_soft_token>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sp_model_kwargs": null,
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}