初始化项目,由ModelHub XC社区提供模型
Model: modrill/qwen3-4b-think-s1-ep23-full-sft Source: Original Platform
This commit is contained in:
72
README.md
Normal file
72
README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
---
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen3-4B-Base
|
||||
tags:
|
||||
- llama-factory
|
||||
- full-finetuning
|
||||
- qwen3
|
||||
- qwen3-4b
|
||||
- think
|
||||
- ocr
|
||||
- sft
|
||||
model-index:
|
||||
- name: think_s1_ep23
|
||||
results: []
|
||||
---
|
||||
|
||||
# Qwen3-4B Think S1 Ep23 (Full SFT)
|
||||
|
||||
Full-parameter supervised fine-tuning (SFT) of [Qwen/Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base) on the **ocr_think_50k** dataset with the Qwen3 chat template (think-style reasoning).
|
||||
|
||||
This checkpoint is **stage 1 episode 23**: training continued from an internal `think_s1` run at **checkpoint-302** (same base architecture), then fine-tuned for two additional epochs on ocr_think_50k.
|
||||
|
||||
## Model description
|
||||
|
||||
- **Method**: full SFT (all weights trainable), DeepSpeed ZeRO-3, 4 GPUs
|
||||
- **Dataset**: ocr_think_50k
|
||||
- **Template**: qwen3
|
||||
- **Not LoRA / not QLoRA**: entire 4B model was updated
|
||||
|
||||
## Training details
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Epochs | 2 |
|
||||
| Seed | 42 |
|
||||
| cutoff_len | 24576 |
|
||||
| packing | true |
|
||||
| neat_packing | false |
|
||||
| per_device_train_batch_size | 1 |
|
||||
| gradient_accumulation_steps | 16 |
|
||||
| effective_batch_size | 64 |
|
||||
| learning_rate | 5e-5 |
|
||||
| train_loss | 0.5416 |
|
||||
| train_steps | 604 |
|
||||
| finished_at | 2026-06-10 05:23 CST |
|
||||
|
||||
Optimizer: AdamW (fused), cosine schedule, warmup ratio 0.1. Framework: Transformers 5.6.0, PyTorch 2.8.0+cu128.
|
||||
|
||||
## Related models
|
||||
|
||||
- **No-think SFT (same project)**: [modrill/qwen3-4b-nothink-s1-full-sft](https://huggingface.co/modrill/qwen3-4b-nothink-s1-full-sft) - OpenCodeInstruct, qwen3_nothink template
|
||||
- **Base**: [Qwen/Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base)
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
repo_id = "modrill/qwen3-4b-think-s1-ep23-full-sft"
|
||||
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
repo_id,
|
||||
torch_dtype="auto",
|
||||
device_map="auto",
|
||||
trust_remote_code=True,
|
||||
)
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Released under Apache 2.0 (see `LICENSE` in the upstream Qwen model card if not bundled here).
|
||||
Reference in New Issue
Block a user