skhotta/slm-125m-base
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
slm-125m-base/README.md
ModelHub XC f759d1e95e 初始化项目，由ModelHub XC社区提供模型 
Model: skhotta/slm-125m-base
Source: Original Platform
2026-08-08 04:51:17 +08:00

741 B
Raw Permalink Blame History

license, language, library_name, pipeline_tag, tags
license language library_name pipeline_tag tags
apache-2.0
en
transformers text-generation
llama
legal
finance
small-language-model

skhotta/slm-125m-base

A ~125.8M parameter Llama-architecture base language model trained from scratch on a legal/financial corpus (US case law + SEC filings + a fineweb-edu slice), with a custom 16,384-token byte-level BPE tokenizer.

  • Params: ~125.8M (12L / 768d / 12h, context 1024)
  • Vocab: 16384 (byte-level BPE)
  • Training data: ~2.0B tokens (~40% case law / ~40% SEC / ~20% web), deduplicated and decontaminated against CaseHOLD/LexGLUE.
  • Objective: causal language modeling.

This is a base model (no instruction tuning). It is a research artifact.