---
library_name: transformers
license: apache-2.0
base_model: Qwen/Qwen3-4B-Base
tags:
- llama-factory
- full-finetuning
- qwen3
- qwen3-4b
- think
- ocr
- sft
model-index:
- name: think_s1_ep23
  results: []
---

# Qwen3-4B Think S1 Ep23 (Full SFT)

Full-parameter supervised fine-tuning (SFT) of [Qwen/Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base) on the **ocr_think_50k** dataset with the Qwen3 chat template (think-style reasoning).

This checkpoint is **stage 1 episode 23**: training continued from an internal `think_s1` run at **checkpoint-302** (same base architecture), then fine-tuned for two additional epochs on ocr_think_50k.

## Model description

- **Method**: full SFT (all weights trainable), DeepSpeed ZeRO-3, 4 GPUs
- **Dataset**: ocr_think_50k
- **Template**: qwen3
- **Not LoRA / not QLoRA**: entire 4B model was updated

## Training details

| Field | Value |
|-------|-------|
| Epochs | 2 |
| Seed | 42 |
| cutoff_len | 24576 |
| packing | true |
| neat_packing | false |
| per_device_train_batch_size | 1 |
| gradient_accumulation_steps | 16 |
| effective_batch_size | 64 |
| learning_rate | 5e-5 |
| train_loss | 0.5416 |
| train_steps | 604 |
| finished_at | 2026-06-10 05:23 CST |

Optimizer: AdamW (fused), cosine schedule, warmup ratio 0.1. Framework: Transformers 5.6.0, PyTorch 2.8.0+cu128.

## Related models

- **No-think SFT (same project)**: [modrill/qwen3-4b-nothink-s1-full-sft](https://huggingface.co/modrill/qwen3-4b-nothink-s1-full-sft) - OpenCodeInstruct, qwen3_nothink template
- **Base**: [Qwen/Qwen3-4B-Base](https://huggingface.co/Qwen/Qwen3-4B-Base)

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "modrill/qwen3-4b-think-s1-ep23-full-sft"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
```

## License

Released under Apache 2.0 (see `LICENSE` in the upstream Qwen model card if not bundled here).