初始化项目，由ModelHub XC社区提供模型

Model: Fordentinc/book-builder-bookwriter-v1
Source: Original Platform

resume/RESUME.md

@@ -0,0 +1,111 @@
+# How to resume training from step 5000
+
+This guide assumes you stopped training at step 5000 (or any saved checkpoint) and now want to continue without restarting from scratch.
+
+## What you need
+
+1. **A GPU** with at least 80 GB VRAM (B200 / H200 / 2×H100 80GB).
+2. **The full checkpoint folder** — published on this repo on the branch `resumable-step-5000`. Contains:
+   - `adapter_model.safetensors` (LoRA weights, ~155 MB)
+   - `adapter_config.json`
+   - `optimizer.pt` (8-bit AdamW state, ~80 MB)
+   - `scheduler.pt` (cosine LR schedule state)
+   - `rng_state.pth` (random state — needed for shuffle resume)
+   - `trainer_state.json` (step counter, loss history)
+   - `training_args.bin` (config snapshot)
+   - tokenizer files
+3. **The training data** — 7.9 GB JSONL of `(bible→chapter)` examples, NOT published here (copyright). The owner has it staged at:
+   - workstation: `/home/fordoilcorp/booktrain/training/train_quality.jsonl`
+   - or wherever the owner has cold-storage backups
+4. **The exact pinned Python stack** — see `requirements.txt` (also on this branch).
+5. **The training script** — `train_qlora_full.py` (also on this branch).
+
+## Step-by-step resume
+
+```bash
+# 1. Set up the host (assumes Ubuntu 22.04 + CUDA 12.8 base image)
+apt-get update && apt-get install -y python3-pip rsync git curl
+pip install --upgrade pip
+
+# 2. Install the pinned stack
+pip install hf_transfer huggingface-hub==0.36.2
+pip install torch==2.12.0 --index-url https://download.pytorch.org/whl/cu128
+pip install -r requirements.txt
+
+# 3. CRITICAL env var for bitsandbytes to find libnvJitLink.so.13
+export LD_LIBRARY_PATH=$(python3 -c "import torch; import os; print(os.path.dirname(torch.__file__))")/../nvidia/cu13/lib:$LD_LIBRARY_PATH
+export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+export HF_HUB_ENABLE_HF_TRANSFER=1
+export HF_TOKEN=<your hf token>
+
+# 4. Pull the resumable checkpoint
+mkdir -p /workspace/checkpoints
+huggingface-cli download Fordentinc/book-builder-bookwriter-v1 \
+  --revision resumable-step-5000 \
+  --local-dir /workspace/checkpoints/checkpoint-5000
+
+# 5. Pull the training script
+huggingface-cli download Fordentinc/book-builder-bookwriter-v1 \
+  --revision resumable-step-5000 \
+  --include "train_qlora_full.py" \
+  --local-dir /workspace/training
+
+# 6. Get the training data onto the host
+#    Option A: rsync from your workstation:
+#      rsync -av /home/fordoilcorp/booktrain/training/train_quality.jsonl <pod>:/workspace/data/
+#      rsync -av /home/fordoilcorp/booktrain/training/eval_quality.jsonl <pod>:/workspace/data/
+#    Option B: pull from a private HF dataset if one is set up later.
+
+# 7. Resume training
+cd /workspace/training
+python3 train_qlora_full.py \
+  --base_model Qwen/Qwen2.5-7B \
+  --train_jsonl /workspace/data/train_quality.jsonl \
+  --eval_jsonl /workspace/data/eval_quality.jsonl \
+  --output_dir /workspace/checkpoints \
+  --max_seq 2048 \
+  --per_device_train_batch_size 4 \
+  --grad_accum 8 \
+  --lr 2e-4 \
+  --num_epochs 1.0 \
+  --warmup_ratio 0.03 \
+  --logging_steps 10 \
+  --save_steps 500 \
+  --eval_steps 500 \
+  --save_total_limit 3 \
+  --hub_repo_id Fordentinc/book-builder-bookwriter-v1 \
+  --hub_token $HF_TOKEN \
+  --resume_from_checkpoint /workspace/checkpoints/checkpoint-5000
+```
+
+The trainer will pick up at step 5001, with the optimizer momentum, the LR schedule position, and the RNG state restored exactly. Expected continuation: step 5001 → 9697 = 4696 steps × ~4.78s = **~6h 14m** of training, ~$26 on a $4.17/hr B200.
+
+## What was tested
+
+- Mechanism validated on this repo using `resumable-test-4500` branch (proved a full checkpoint folder uploads and re-downloads cleanly with all files intact).
+
+## Things that are NOT in this branch (and where they live)
+
+- **Base model `Qwen/Qwen2.5-7B`** — pulled from `huggingface.co/Qwen/Qwen2.5-7B` (Apache 2.0, public, will remain available).
+- **Training corpus** — see point 3 above. NOT on HF.
+- **Tokenize cache** — regenerated automatically on first run (~7 min on B200). Not worth shipping.
+
+## Verifying the checkpoint before launching real training
+
+After step 5 above, sanity check by loading the checkpoint:
+
+```python
+from transformers import AutoModelForCausalLM
+from peft import PeftModel
+import torch
+
+base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B", torch_dtype=torch.bfloat16, device_map="cuda:0")
+model = PeftModel.from_pretrained(base, "/workspace/checkpoints/checkpoint-5000")
+print("adapter loaded OK, trainable params:", sum(p.numel() for p in model.parameters() if p.requires_grad)/1e6, "M")
+```
+
+Expect: `~40.4 M`. If you see that, you can resume safely.
+
+---
+
+Last updated: 2026-05-28