b3f76f74e6
Model: Raghav-Singhal/sdsp-smollm-1p7b-100B-30n-2048sl-960gbsz-judgemental-a1_0p0-a2_1p0 Source: Original Platform
26 lines
521 B
Markdown
26 lines
521 B
Markdown
---
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- llama
|
|
- causal-lm
|
|
- bfloat16
|
|
---
|
|
|
|
# sdsp-smollm-1p7b-100B-30n-2048sl-960gbsz-judgemental-a1_0p0-a2_1p0
|
|
|
|
Converted Hugging Face base checkpoint from the Model Raising pretraining run.
|
|
|
|
## Details
|
|
|
|
- Architecture: `LlamaForCausalLM`
|
|
- Base model size: `1.7B`
|
|
- Precision on disk: `bfloat16`
|
|
- Source Megatron checkpoint iteration: `50863`
|
|
- Model kind: `normal`
|
|
- Config vocab size: `49152`
|
|
|
|
## Tokenizer
|
|
|
|
Use the bundled tokenizer from this repository.
|