初始化项目,由ModelHub XC社区提供模型

Model: Raghav-Singhal/tulu3sft-normal-smollm-1p7b-500B-30n-2048sl-960gbsz
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-04-28 05:47:08 +08:00
commit ca9203e0b5
11 changed files with 294159 additions and 0 deletions

41
README.md Normal file
View File

@@ -0,0 +1,41 @@
---
language: en
license: apache-2.0
tags:
- smollm
- llama
- causal-lm
- sft
- tulu
model_type: llama
pipeline_tag: text-generation
---
# tulu3sft-normal-smollm-1p7b-500B-30n-2048sl-960gbsz
This is a supervised fine-tuned (SFT) checkpoint for a SmolLM2-style 1.7B model,
trained on the `allenai/tulu-3-sft-mixture` dataset. It is based on the 500B-token
pretrained base checkpoint and exported in Hugging Face `LlamaForCausalLM` format.
## Details
- Base model: `normal-smollm-1p7b-500B-30n-2048sl-960gbsz`
- SFT dataset: `allenai/tulu-3-sft-mixture`
- Context length: 2048
- Vocab size: 49152
- Architecture: Llama (RMSNorm, SwiGLU, RoPE)
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "REPLACE_WITH_OWNER/tulu3sft-normal-smollm-1p7b-500B-30n-2048sl-960gbsz"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
```
## Notes
This is an SFT model intended for chat-style use. For preference tuning, run DPO
on top of this checkpoint.