初始化项目,由ModelHub XC社区提供模型
Model: atenareply/smollm2-135m-noval Source: Original Platform
This commit is contained in:
49
README.md
Normal file
49
README.md
Normal file
@@ -0,0 +1,49 @@
|
||||
---
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
language:
|
||||
- en
|
||||
base_model:
|
||||
- HuggingFaceTB/SmolLM2-135M
|
||||
datasets:
|
||||
- atenareply/noval-corp-training-corpus
|
||||
tags:
|
||||
- smollm2
|
||||
- continued-pretraining
|
||||
- domain-adaptation
|
||||
license: apache-2.0
|
||||
---
|
||||
|
||||
# SmolLM2-135M Noval — Domain CPT
|
||||
|
||||
HuggingFaceTB/SmolLM2-135M after Continued Pre-Training on the fictional OMC + Mars Express domain corpus. First-round (135M) proof of the pipeline.
|
||||
|
||||
## Overview
|
||||
|
||||
- **Stage**: Continued Pre-Training (full fine-tune, CLM)
|
||||
- **Lineage**: SmolLM2-135M → **CPT** (this model)
|
||||
- **Method**: Continued Pre-Training (next-token, full fine-tune) on a 30/70 mars/orbital interleaved corpus (tokenized + packed).
|
||||
- **Domain**: fictional — Orbital Mining Corporation (OMC) technical docs + Mars Express telemetry.
|
||||
|
||||
## Training
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Corpus | noval-corp-training-corpus (30% Mars / 70% OMC, packed input_ids) |
|
||||
| LR / schedule | 5e-4 cosine, warmup 200 steps |
|
||||
| Duration / seq | max_steps 5000, seq 1024, eff_batch 32, T4 fp16 |
|
||||
|
||||
## Evaluation
|
||||
|
||||
_No task metrics for this checkpoint (intermediate / no-training stage)._
|
||||
|
||||
## Intended use & limitations
|
||||
|
||||
Domain backbone for the 135M line (knows the domain; not instruction-tuned).
|
||||
|
||||
**Limitations:**
|
||||
- 135M is intrinsically limited — expectations should be modest.
|
||||
- Fictional domain; no chat template / instruction following.
|
||||
|
||||
---
|
||||
_Card generated by `noval-corp/scripts/gen_model_cards.py` (standardized across the noval-corp model family)._
|
||||
Reference in New Issue
Block a user