初始化项目,由ModelHub XC社区提供模型

Model: lino-levan/qwen3-1.7b-smoltalk
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-10 23:56:07 +08:00
commit 1f2b5e385b
14 changed files with 152253 additions and 0 deletions

48
README.md Normal file
View File

@@ -0,0 +1,48 @@
---
library_name: transformers
base_model:
- Qwen/Qwen3-1.7B-Base
---
# Qwen3 1.7B Smoltalk
Qwen3 1.7B just SFT'd on [smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk).
Trained using TRL with:
- 4x H100 from lambda labs
- epochs: 2 (i.e. 3,300 steps using our batch size)
- effective batch size: 128 (4 per gpu, 8 grad accumulation steps)
- warmup ratio: 0.03
- weight decay: 0.01
- learning rate: Forgot, will come back to add later.
- learning rate scheduler: cosine
- final training loss: 0.6432
|Benchmark|Score|
|---------|-----|
|AIME25|0%|
|GPQA|24.8%|
|GSM8K|54.2%|
|IFBench|18.3%|
|IFEval|55%|
|MMLU-Pro|22.8%|
|Multi-IF|32.5%|
## Quick Start
```python
from transformers import pipeline
pipe = pipeline(
"text-generation",
model="lino-levan/qwen-3-1.7b-smoltalk",
)
messages = [
{"role": "user", "content": "What is 3 * 8?"},
]
output = pipe(messages)
print(output[0]["generated_text"][-1]["content"])
```