初始化项目，由ModelHub XC社区提供模型

Model: AliceYin/l20-edu-135m
Source: Original Platform

docs/training_recipe.md

@@ -0,0 +1,130 @@
+# Training Recipe
+
+This document records the exact recipe used for the released
+`l20-edu-135m` base checkpoint.
+
+## Model
+
+| Field | Value |
+| --- | --- |
+| Run name | `l20-edu-135m-deepthin` |
+| Parameters | 134,515,008 |
+| Architecture | Llama-style decoder-only Transformer |
+| Layers | 30 |
+| Hidden size | 576 |
+| FFN size | 1536 |
+| Attention heads | 9 query heads, 3 key/value heads |
+| Context length | 2048 |
+| Tokenizer | `HuggingFaceTB/SmolLM2-135M` |
+| Attention implementation | PyTorch SDPA |
+| Tied embeddings | yes |
+
+## Data
+
+| Field | Value |
+| --- | --- |
+| Dataset | `HuggingFaceFW/fineweb-edu` |
+| Config | `sample-10BT` |
+| Split | `train` |
+| Streaming | yes |
+| Text filter | `min_chars=300`, `max_chars=50000` |
+| Quality filter | `min_score=3.0`, `min_int_score=3` |
+| Packing | EOS-joined documents packed into 2048-token blocks |
+| Planned token budget | 10,001,252,352 tokens |
+
+## Optimization
+
+| Field | Value |
+| --- | --- |
+| Optimizer | AdamW |
+| Learning rate | `4e-4` peak |
+| LR schedule | linear warmup + cosine decay to `0.1 * peak_lr` |
+| Warmup steps | 1000 |
+| Min LR ratio | 0.1 |
+| Weight decay | 0.1 |
+| Adam beta1 / beta2 | 0.9 / 0.95 |
+| Gradient clip | 1.0 |
+| Precision | bfloat16 |
+| Torch compile | enabled |
+| Gradient checkpointing | enabled |
+
+## Batch And Token Accounting
+
+| Field | Value |
+| --- | ---: |
+| Micro batch size | 6 sequences |
+| Gradient accumulation | 43 |
+| Global batch size | 258 sequences |
+| Sequence length | 2048 tokens |
+| Tokens per optimizer step | 528,384 |
+| Max steps | 18,928 |
+| Planned tokens | 10,001,252,352 |
+
+## Checkpointing And Evaluation
+
+| Field | Value |
+| --- | --- |
+| Log interval | 10 steps |
+| Eval interval | 500 steps |
+| Eval batches | 64 |
+| Save interval | 1000 steps |
+| Checkpoints retained | last 2 |
+| Final checkpoint | `runs/l20-edu-135m-deepthin/step-018928` |
+| Published checkpoint | `AliceYin/l20-edu-135m` |
+
+Each regular checkpoint represents about 528.4M training tokens. The final
+checkpoint was saved at step 18,928 rather than an even 1000-step boundary.
+
+## Runtime And Hardware
+
+| Field | Value |
+| --- | --- |
+| GPU | NVIDIA L20 |
+| Reported GPU memory | 46,068 MiB total |
+| Driver | 550.163.01 |
+| Mean logged throughput | 38,541 tokens/s |
+| Mean logged throughput after step 1000 | 38,587 tokens/s |
+| Estimated train time from throughput | about 72.0 hours |
+| Final checkpoint mtime | 2026-05-19 05:04:22 +0800 |
+
+The JSON training log did not record an exact wall-clock launch timestamp or
+peak VRAM. Peak GPU memory should therefore be treated as **not measured** for
+this release. Future runs should log `nvidia-smi --query-gpu=memory.used` during
+training.
+
+Cost was not available from billing logs. A reproducible estimate is:
+
+```text
+estimated_cost = 72 GPU-hours * L20_hourly_rate
+```
+
+Examples:
+
+| L20 Hourly Rate | Estimated GPU Cost |
+| ---: | ---: |
+| $0.60 / hour | $43 |
+| $1.00 / hour | $72 |
+| $1.50 / hour | $108 |
+
+This excludes storage, network egress, idle time, and engineering time.
+
+## Known Issues During The Run
+
+- The dataset mirror produced transient `Read timed out` errors near the end of
+  training. The run recovered through retry and continued.
+- The final perplexity command printed `loss=2.8731 perplexity=17.69`, then hit
+  a Python finalization crash. The metric is usable because it was printed before
+  process teardown, but the crash is documented.
+- After the final checkpoint was written, the training process printed
+  `terminate called without an active exception`. The checkpoint was complete
+  and load-tested with `AutoModelForCausalLM`.
+- Peak VRAM and exact cloud cost were not logged.
+
+## Reproduction Command
+
+```bash
+python -m l20_pretrain.train configs/l20_135m_deepthin.yaml
+```
+
+The config file is the source of truth for this recipe:
+`configs/l20_135m_deepthin.yaml`.