96 lines
4.7 KiB
Markdown
96 lines
4.7 KiB
Markdown
# Implementation Plan: Fine-Tuning GPT-2 on High-Quality SFT Dataset (Revised)
|
|
|
|
We will perform a new fine-tuning run using the high-quality **`HuggingFaceH4/no_robots`** SFT dataset. We will first train on a subset to verify training dynamics and output coherence, and then execute training on the full dataset.
|
|
|
|
---
|
|
|
|
## User Review Required
|
|
|
|
> [!IMPORTANT]
|
|
> **Please review the following clarifications and confirm approval to begin the autonomous run:**
|
|
|
|
### 1. Continual Pre-training (FineWeb) vs. Direct SFT (no_robots)
|
|
You asked: *Is it a good idea to SFT on a high quality document dataset, like fineweb-2, then IF on no_robots?*
|
|
- **Our Recommendation**: We recommend **direct SFT on `no_robots`** from the base GPT-2 model.
|
|
- **Why**: Continual pre-training (on raw document datasets like FineWeb) is highly compute-intensive and requires billions of tokens to avoid "catastrophic forgetting" of the model's base language modeling capability. For a 124M model like GPT-2 under a 12-hour limit, training on FineWeb documents first would consume significant time and risk degrading the model's coherence or causing it to forget basic grammar before SFT even starts. Instruction-following (IF) alignment directly on `no_robots` is the most reliable way to teach it conversational structure within our timeframe.
|
|
|
|
### 2. Disk Space Cleanup Strategy
|
|
- **Baseline**: Currently, we have 21 GB of free disk space. However, `/home/maxgn/.cache/huggingface/` contains **35 GB** of cached data (29 GB in `hub`, 6.2 GB in `datasets`).
|
|
- **Action**: Before and during training, we will monitor disk space. If free space drops below **12 GB**, we will clean up old caches (specifically unused hub models and datasets) to keep the host healthy.
|
|
|
|
### 3. Updated Test Split & Verification Formats
|
|
- **Test Split**: Adjusted to `train[:2500]` per your feedback.
|
|
- **Coherence Verification Formats**: Added **`IQ3_XXS`** to our test formats alongside `IQ4_NL` and `Q8_0`.
|
|
|
|
---
|
|
|
|
## Proposed Changes
|
|
|
|
### Configuration Changes
|
|
|
|
#### [MODIFY] [axolotl-gpt2-chatml-fp32.yml](file:///home/maxgn/gpt2-chatml-web-chat/axolotl-gpt2-chatml-fp32.yml)
|
|
Update the dataset configuration to point to `HuggingFaceH4/no_robots`, map its standard fields, and direct outputs to a new dedicated folder:
|
|
|
|
```yaml
|
|
datasets:
|
|
- path: HuggingFaceH4/no_robots
|
|
split: train[:2500] # Set to train[:2500] for test split, train for full run
|
|
type: chat_template
|
|
field_messages: messages
|
|
message_property_mappings:
|
|
role: role
|
|
content: content
|
|
roles_to_train:
|
|
- assistant
|
|
train_on_eos: turn
|
|
default_system_message: "You are a helpful, concise assistant. Keep answers short and clear."
|
|
|
|
output_dir: /home/maxgn/outputs/gpt2-chatml-no-robots
|
|
```
|
|
|
|
---
|
|
|
|
## Execution Plan (Autonomous Execution)
|
|
|
|
Once approved, we will execute the following steps:
|
|
|
|
### Phase 1: Cache Setup & Pre-Flight Check
|
|
1. Verify free disk space. If needed, clear out old Hugging Face hub checkpoints to free up extra headroom.
|
|
2. Update `axolotl-gpt2-chatml-fp32.yml` to target the `train[:2500]` split.
|
|
|
|
### Phase 2: Test Split Training
|
|
1. Launch training using the background command:
|
|
```bash
|
|
export AXOLOTL_DO_NOT_TRACK=1 && source /home/maxgn/axolotl/.venv/bin/activate && accelerate launch --module --dynamo_backend no axolotl.cli.train /home/maxgn/gpt2-chatml-web-chat/axolotl-gpt2-chatml-fp32.yml --debug=False --debug-text-only=False --debug-num-examples=0 --shard=False
|
|
```
|
|
2. Monitor validation loss in `debug.log`.
|
|
|
|
### Phase 3: Test Split GGUF Quantization & Verification
|
|
1. Convert the test checkpoint to GGUF format: `gpt2-f32.gguf`.
|
|
2. Generate an imatrix file on the `calibration-data.txt` file.
|
|
3. Quantize the model into **`Q8_0`**, **`IQ4_NL`**, and **`IQ3_XXS`**.
|
|
4. Run inference tests on:
|
|
- *Prompt A*: "What is the capital of France?"
|
|
- *Prompt B*: "Why is the sky blue? Explain in one short sentence."
|
|
5. **Evaluation**:
|
|
- If output quality is equal/better than our previous run, proceed to **Phase 5**.
|
|
- If output quality is worse (e.g. repetitive loops or structure loss), proceed to **Phase 4** (Troubleshooting).
|
|
|
|
### Phase 4: Troubleshooting (If needed)
|
|
1. Adjust hyperparameters (e.g., reduce learning rate to `2e-5`, increase epochs, or tune repetition penalty).
|
|
2. Re-run Phase 2 and 3 until verified.
|
|
|
|
### Phase 5: Full Dataset Training & Final Quantization
|
|
1. Update `axolotl-gpt2-chatml-fp32.yml` to `split: train`.
|
|
2. Run full training (~1.5 hours).
|
|
3. Convert final checkpoint to GGUF and quantize into **`Q8_0`**, **`IQ4_NL`**, and **`IQ3_XXS`** using imatrix.
|
|
4. Perform final coherence verification.
|
|
|
|
---
|
|
|
|
## Verification Plan
|
|
|
|
### Automated Tests
|
|
- Run `llama-cli -st` on the quantized models and log responses to confirm coherence.
|
|
- Output files and sizes will be verified and logged in `walkthrough.md`.
|