初始化项目,由ModelHub XC社区提供模型
Model: lino-levan/qwen3-1.7b-smoltalk Source: Original Platform
This commit is contained in:
48
README.md
Normal file
48
README.md
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
library_name: transformers
|
||||
base_model:
|
||||
- Qwen/Qwen3-1.7B-Base
|
||||
---
|
||||
|
||||
# Qwen3 1.7B Smoltalk
|
||||
|
||||
Qwen3 1.7B just SFT'd on [smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk).
|
||||
|
||||
Trained using TRL with:
|
||||
|
||||
- 4x H100 from lambda labs
|
||||
- epochs: 2 (i.e. 3,300 steps using our batch size)
|
||||
- effective batch size: 128 (4 per gpu, 8 grad accumulation steps)
|
||||
- warmup ratio: 0.03
|
||||
- weight decay: 0.01
|
||||
- learning rate: Forgot, will come back to add later.
|
||||
- learning rate scheduler: cosine
|
||||
- final training loss: 0.6432
|
||||
|
||||
|
||||
|Benchmark|Score|
|
||||
|---------|-----|
|
||||
|AIME25|0%|
|
||||
|GPQA|24.8%|
|
||||
|GSM8K|54.2%|
|
||||
|IFBench|18.3%|
|
||||
|IFEval|55%|
|
||||
|MMLU-Pro|22.8%|
|
||||
|Multi-IF|32.5%|
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from transformers import pipeline
|
||||
|
||||
pipe = pipeline(
|
||||
"text-generation",
|
||||
model="lino-levan/qwen-3-1.7b-smoltalk",
|
||||
)
|
||||
messages = [
|
||||
{"role": "user", "content": "What is 3 * 8?"},
|
||||
]
|
||||
output = pipe(messages)
|
||||
print(output[0]["generated_text"][-1]["content"])
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user