初始化项目,由ModelHub XC社区提供模型

Model: sudhisrk1982/slm-125m-instruct
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-08-03 03:55:17 +08:00
commit a786622aef
9 changed files with 81197 additions and 0 deletions

35
README.md Normal file
View File

@@ -0,0 +1,35 @@
---
license: apache-2.0
base_model: thesreedath/slm-125m-base
pipeline_tag: text-generation
---
# slm-125m-instruct
Instruction-tuned from [`thesreedath/slm-125m-base`](thesreedath/slm-125m-base) on 9,071 legal/financial
Q&A pairs generated from a case-law + SEC-filing corpus.
## Training
- **Base**: 125.8M params (12L / 768d / 16,384 vocab)
- **Data**: 9,071 Q&A pairs, LLM-judged (90.7% pass rate) from 19,775 raw
- **Method**: full SFT, prompt tokens masked from the loss (answer-only training)
- **Hyperparameters**: lr 2e-5, cosine decay, 3 epochs, bf16, 1xA100
- **Best val loss**: 1.914
## Chat format
```
<|bos|><|system|>{system}<|user|>{question}<|assistant|>{answer}<|eos|>
```
## Note on token ids
The base repo's `config.json` declares `eos_token_id: 2`, but id 2 is `<|pad|>`
in the tokenizer -- `<|eos|>` is id 1. This model ships the **corrected** ids
(`bos=0, eos=1, pad=2`), so generation terminates properly.
## Limitations
125M parameters trained on ~9k pairs. It answers in the right form and stops
cleanly, but factual accuracy is limited -- verify anything load-bearing.