280 lines
6.2 KiB
Markdown
280 lines
6.2 KiB
Markdown
---
|
|
language:
|
|
- de
|
|
license: other
|
|
base_model: HuggingFaceTB/SmolLM3-3B
|
|
tags:
|
|
- sft
|
|
- instruction-tuning
|
|
- reasoning
|
|
- german
|
|
- multilingual
|
|
- long-context
|
|
- fsdp
|
|
- transformers
|
|
datasets:
|
|
- DGurgurov/Nemotron-Multilingual-Reasoning
|
|
metrics:
|
|
- token_accuracy
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# SmolLM3-3B — German Reasoning Instruction SFT (Nemotron Multilingual Reasoning)
|
|
|
|
## Model Description
|
|
|
|
This model is a **Supervised Fine-Tuned (SFT)** version of:
|
|
|
|
`HuggingFaceTB/SmolLM3-3B`
|
|
|
|
It was fine-tuned on the **German (`de`) split** of the dataset:
|
|
|
|
`DGurgurov/Nemotron-Multilingual-Reasoning`
|
|
|
|
The goal of the training was to improve:
|
|
|
|
- German instruction following
|
|
- Step-by-step reasoning
|
|
- Long-context conversation behavior
|
|
|
|
The model was trained using chat-formatted conversations and **completion-only loss**, meaning only assistant responses contributed to optimization.
|
|
|
|
Key properties:
|
|
|
|
- Base model: SmolLM3-3B
|
|
- Language specialization: German
|
|
- Context length during training: **16,384 tokens**
|
|
- Chat formatted dataset
|
|
- Long-context packing enabled
|
|
|
|
---
|
|
|
|
## Intended Uses
|
|
|
|
### Suitable For
|
|
- German conversational assistants
|
|
- Educational tutoring
|
|
- Reasoning and structured explanation tasks
|
|
- Long-document Q&A in German
|
|
- Research experiments with long-context small LLMs
|
|
|
|
### Not Suitable For
|
|
- Medical or legal advice without human review
|
|
- Autonomous decision-making
|
|
- Safety-critical systems
|
|
- High-stakes financial decisions
|
|
|
|
---
|
|
|
|
## Training Data
|
|
|
|
Dataset used:
|
|
|
|
`DGurgurov/Nemotron-Multilingual-Reasoning`
|
|
|
|
Processing configuration:
|
|
|
|
- Language filtering: **German only**
|
|
- Converted into chat messages (`prepare_messages=True`)
|
|
- Assistant-only optimization (`completion_only_loss=True`)
|
|
|
|
Only the assistant responses were used to compute loss; user and system messages were masked.
|
|
|
|
Please review the dataset card for provenance and limitations.
|
|
|
|
---
|
|
|
|
## Training Procedure
|
|
|
|
Training was performed using **HuggingFace Accelerate with FSDP (Fully Sharded Data Parallel)** across 8 processes.
|
|
|
|
### Core Setup
|
|
|
|
- Training method: Supervised fine-tuning (SFT)
|
|
- Epochs: **3**
|
|
- Maximum sequence length: **16,384**
|
|
- Sequence packing: enabled
|
|
- Precision: **bfloat16**
|
|
- Kernel optimization: Liger kernel enabled
|
|
- Gradient checkpointing: enabled
|
|
- Distributed: FSDP (8 processes)
|
|
|
|
---
|
|
|
|
### Optimization
|
|
|
|
- Optimizer: `adamw_torch_fused`
|
|
- Per-device batch size: 4
|
|
- Gradient accumulation: 4
|
|
- Effective batch size (per GPU): 16 sequences per step
|
|
- Weight decay: 0.05
|
|
|
|
Learning rate schedule:
|
|
|
|
- Scheduler: `cosine_with_min_lr`
|
|
- Warmup ratio: 0.05
|
|
- Minimum LR: 5e-6
|
|
|
|
---
|
|
|
|
### Logging & Checkpoints
|
|
|
|
- Logging every 5 steps
|
|
- Checkpoint every 450 steps
|
|
- Weights & Biases tracking enabled
|
|
- Token accuracy logged during training
|
|
|
|
---
|
|
|
|
### Data Processing
|
|
|
|
- Dataset workers: 16
|
|
- Dataset preparation: enabled
|
|
- Chat message preparation: enabled
|
|
- German split: enabled
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
### Transformers
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModelForCausalLM
|
|
import torch
|
|
|
|
model_id = "YOUR_USERNAME/YOUR_MODEL_NAME"
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
|
|
model = AutoModelForCausalLM.from_pretrained(
|
|
model_id,
|
|
device_map="auto",
|
|
torch_dtype=torch.bfloat16,
|
|
)
|
|
|
|
messages = [
|
|
{"role": "system", "content": "Du bist ein hilfreicher Assistent."},
|
|
{"role": "user", "content": "Warum ist der Himmel blau?"}
|
|
]
|
|
|
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
|
|
|
outputs = model.generate(
|
|
**inputs,
|
|
max_new_tokens=512,
|
|
temperature=0.7,
|
|
top_p=0.9,
|
|
do_sample=True
|
|
)
|
|
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
```
|
|
**Important:**
|
|
You should use `apply_chat_template()` when prompting. The model was trained on chat-formatted conversations and performance will degrade without it.
|
|
|
|
---
|
|
|
|
## Evaluation
|
|
|
|
During training, **token accuracy** was logged as a diagnostic metric.
|
|
|
|
Token accuracy:
|
|
- is useful for monitoring training stability
|
|
- is **NOT** a benchmark score
|
|
- does not represent real reasoning performance
|
|
|
|
For proper evaluation, use:
|
|
- German instruction-following benchmarks
|
|
- reasoning datasets
|
|
- long-context evaluation tasks
|
|
|
|
---
|
|
|
|
## Limitations
|
|
|
|
- May hallucinate facts
|
|
- Reasoning chains can still contain logical errors
|
|
- Performance near 16k context depends heavily on prompt structure
|
|
- Improvements mainly apply to German
|
|
- Smaller model size means weaker world knowledge than large LLMs
|
|
- Not aligned for safety-critical deployment
|
|
|
|
---
|
|
|
|
## Bias & Safety
|
|
|
|
This model inherits biases from:
|
|
- the base model
|
|
- the training dataset
|
|
|
|
Recommended mitigations:
|
|
- add moderation filters
|
|
- use system prompts enforcing safe behavior
|
|
- include human review for sensitive deployments
|
|
|
|
---
|
|
|
|
## License
|
|
|
|
This model is a derivative of:
|
|
|
|
`HuggingFaceTB/SmolLM3-3B`
|
|
|
|
Therefore, the original base model license and usage restrictions apply, along with any dataset terms.
|
|
|
|
Verify compatibility before commercial deployment.
|
|
|
|
---
|
|
|
|
## Reproducibility (Training Arguments)
|
|
|
|
```text
|
|
accelerate launch --use_fsdp --num_processes 8 --config_file sft/my_config.yaml sft/sft_trainer.py
|
|
|
|
--model_name HuggingFaceTB/SmolLM3-3B
|
|
--tokenizer_name HuggingFaceTB/SmolLM3-3B
|
|
--dataset_path DGurgurov/Nemotron-Multilingual-Reasoning
|
|
--skip_prepare_dataset False
|
|
--lang_split de
|
|
--prepare_messages True
|
|
--completion_only_loss True
|
|
--max_length 16384
|
|
--dataset_num_proc 16
|
|
--packing True
|
|
--use_liger_kernel True
|
|
--bf16 True
|
|
--log_token_accuracy True
|
|
--optim adamw_torch_fused
|
|
--gradient_checkpointing True
|
|
--per_device_train_batch_size 4
|
|
--gradient_accumulation_steps 4
|
|
--ddp_find_unused_parameters False
|
|
--lr_scheduler_type cosine_with_min_lr
|
|
--lr_scheduler_kwargs {"min_lr": 5.0e-6}
|
|
--warmup_ratio 0.05
|
|
--weight_decay 0.05
|
|
--report_to wandb
|
|
--run_name smol_3b_3epochs_lns_de
|
|
--num_train_epochs 3
|
|
--save_strategy steps
|
|
--logging_steps 5
|
|
--save_steps 450
|
|
```
|
|
---
|
|
|
|
## Citation
|
|
|
|
If you use this model, please cite:
|
|
|
|
- `HuggingFaceTB/SmolLM3-3B`
|
|
- `DGurgurov/Nemotron-Multilingual-Reasoning`
|
|
|
|
---
|
|
|
|
## Acknowledgements
|
|
|
|
- HuggingFaceTB — SmolLM3 base model
|
|
- Nemotron Multilingual Reasoning dataset authors
|
|
- HuggingFace Accelerate and Transformers libraries |