初始化项目，由ModelHub XC社区提供模型

Model: SeongryongJung/Qwen3-4B-Chemistry-SDPO
Source: Original Platform

README.md

@@ -0,0 +1,126 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-4B
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- qwen3
+- chemistry
+- reinforcement-learning
+- verl
+- sciknoweval
+- sdpo
+- self-distillation
+---
+
+# Qwen3-4B Chemistry SDPO
+
+This repository contains Chemistry fine-tuned Qwen3-4B checkpoints from the local SciKnowEval-style generalization setup.
+
+- Root checkpoint: final `global_step_100` merged to Hugging Face safetensors.
+- `best_avg16/`: checkpoint with the highest validation `avg@16` during training, merged to Hugging Face safetensors.
+
+## Checkpoints
+
+| Checkpoint | Source step | Validation avg@16 | best@16 | maj@16 |
+|---|---:|---:|---:|---:|
+| Root final | 100 | 0.720536 | 0.731567 | 0.721367 |
+| `best_avg16/` | 20 | 0.766369 | 0.822871 | 0.779486 |
+
+## Training Run
+
+`qwen3gen-chemistry-SDPO-Qwen-Qwen3-4B-mbs32-ema0.05-train256-rollout8-lr1e-5-vllm0.8`
+
+W&B run: `https://wandb.ai/seongryongjung-chung-ang-university/qwen3-generalization-batch256/runs/1qazekk3`
+
+## Base Model
+
+- Base model: `Qwen/Qwen3-4B`
+- Fine-tuning type: full-parameter FSDP RL training
+- Dataset: `datasets/sciknoweval/chemistry`
+- Train split: 1,890 examples
+- Validation split: 210 examples
+
+## Method
+
+- Method: SDPO
+- Config: `sdpo`
+- Policy loss mode: `sdpo`
+- Reward: local SciKnowEval multiple-choice reward checker
+- Rollout correction: token-level importance sampling, threshold 2.0
+
+## Hyperparameters
+
+| Field | Value |
+|---|---:|
+| Base model | `Qwen/Qwen3-4B` |
+| Training steps | 100 |
+| Train batch size | 256 |
+| Rollouts per prompt | 8 |
+| Generations per step | 2048 |
+| PPO mini batch size | 32 |
+| Learning rate | `1e-5` |
+| LR warmup steps | 10 |
+| Weight decay | 0.01 |
+| Grad clip | 1.0 |
+| Max prompt length | 2048 |
+| Max response length | 8192 |
+| Max model length | 10240 |
+| Train temperature | 1.0 |
+| Train top_p | 1.0 |
+| Validation generations | 16 |
+| Validation temperature | 0.6 |
+| Validation top_p | 0.95 |
+| vLLM GPU memory utilization | 0.8 |
+| GPUs | 8 x NVIDIA H200 |
+| Save frequency | every 10 steps |
+| Validation frequency | every 10 steps |
+| Distillation top-k | 100 |
+| SDPO alpha | 0.5 |
+| Teacher update rate | 0.05 |
+| Distillation IS clip | 2.0 |
+| Max reprompt length | 10240 |
+
+## Metrics
+
+![Training score](training_score.png)
+
+CSV files:
+
+- [`training_score.csv`](training_score.csv)
+- [`validation_metrics.csv`](validation_metrics.csv)
+
+| Metric | Value |
+|---|---:|
+| Final training step | 100 |
+| Final `critic/score/mean` | 0.845215 |
+| Final `critic/rewards/mean` | 0.845215 |
+| Final validation `avg@16` | 0.720536 |
+| Peak validation `avg@16` | 0.766369 |
+| Peak validation step | 20 |
+
+## Loading
+
+Root final checkpoint:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("SeongryongJung/Qwen3-4B-Chemistry-SDPO")
+tokenizer = AutoTokenizer.from_pretrained("SeongryongJung/Qwen3-4B-Chemistry-SDPO")
+```
+
+Best avg@16 checkpoint:
+
+```python
+model = AutoModelForCausalLM.from_pretrained("SeongryongJung/Qwen3-4B-Chemistry-SDPO", subfolder="best_avg16")
+tokenizer = AutoTokenizer.from_pretrained("SeongryongJung/Qwen3-4B-Chemistry-SDPO", subfolder="best_avg16")
+```
+
+## Intended Use
+
+This model is intended for research on RL fine-tuning and self-distillation behavior on science/generalization tasks. It has not been broadly safety evaluated for production use.
+
+## Limitations
+
+The reported scores are training-time and validation-time metrics from the local experimental setup. They should not be interpreted as broad benchmark results without independent evaluation.