--- library_name: transformers license: llama3.1 base_model: meta-llama/Llama-3.1-8B-Instruct tags: - axolotl - generated_from_trainer datasets: - cfierro/othello-snake-llama3-multi-format - cfierro/othello-snake-llama3-multi-format - cfierro/othello-snake-llama3-multi-format - cfierro/othello-snake-llama3-multi-format - cfierro/tulu3-sft-replay model-index: - name: llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5 results: [] --- [Built with Axolotl](https://github.com/axolotl-ai-cloud/axolotl)
See axolotl config axolotl version: `0.10.0` ```yaml base_model: meta-llama/Llama-3.1-8B-Instruct model_type: LlamaForCausalLM tokenizer_type: AutoTokenizer load_in_8bit: false load_in_4bit: false # ============================================================================ # Full FT of the Othello-snake task WITH a chat-data replay mix, to fight the # catastrophic forgetting of instruction-following seen in the game-only run # (the model collapsed to emitting only game moves). Two data sources are mixed: # 1. The game, as a raw `input_output` continuation document (masked prefix), # here in the MULTI-FORMAT variant: ~70% space-separated moves plus ~5% each # of several alternative notations (Player 1/2, black/white, turn N, comma, # "(black)/(white)"), so the model is robust to how a game is written down. # 2. A general chat replay set (Tulu-3 subset) as `type: chat_template`. # The Llama-3 role tokens on the chat data teach the model a context switch: # game-like raw context -> moves; chat prompt with an instruction -> language. # Replay built by data/push_replay_subset.py at a ~1:1 trained-token ratio vs the # game (full-FT forgets hard, so a generous chat share). # ============================================================================ chat_template: llama3 datasets: # --- Game shards (template-free, loss-masked continuation, multi-format) --- # The first shard is sliced to hold out its last 500 games for a game-ONLY # validation set (see test_datasets below). The other three shards are full. - path: cfierro/othello-snake-llama3-multi-format type: input_output split: games_0_5k[:-500] - path: cfierro/othello-snake-llama3-multi-format type: input_output split: games_5k_10k - path: cfierro/othello-snake-llama3-multi-format type: input_output split: games_10k_25k - path: cfierro/othello-snake-llama3-multi-format type: input_output split: games_25k_50k # --- Replay: general chat data (Llama-3 chat-template formatted) --- - path: cfierro/tulu3-sft-replay type: chat_template field_messages: messages split: train train_on_inputs: false # game: segment labels apply; chat: mask user, train assistant dataset_prepared_path: /scratch/project/eu-26-55/knowledge-ft/axolotl/datasets/llama-3.1-8b/othello-snake-multi-format-replay output_dir: /scratch/project/eu-26-55/knowledge-ft/axolotl/models/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5 sequence_len: 1024 # games are short; replay was filtered to <=1024 tokens sample_packing: true # pack many short docs per sequence (block-diagonal attn) eval_sample_packing: false # --- Validation loss on GAME DATA ONLY --- # val_set_size: 0 disables the default carve (which would mix game + chat into the # val set). test_datasets points eval exclusively at a held-out slice of game data, # so eval_loss == game performance and drives load_best_model_at_end below. # NOTE: the split-slicing syntax (`[:-500]` / `[-500:]`) requires your Axolotl to # forward `split` to load_dataset (verify once with `axolotl preprocess ... --debug`). # Fallback if slicing is unsupported in 0.10.0: hold out a WHOLE shard instead -- # drop `games_25k_50k` from datasets above and use it (full) as the test_dataset. val_set_size: 0 test_datasets: - path: cfierro/othello-snake-llama3-multi-format type: input_output split: games_0_5k[-500:] wandb_project: othello-snake-ft wandb_entity: cfierro wandb_watch: wandb_name: llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5 wandb_log_model: "false" # --- Multi-GPU: 4x A40 (48GB) per node --- # Full FT of an 8B model needs ZeRO-3 to shard weights+grads+optimizer. # Effective batch = 4 GPUs * micro 4 * grad_accum 1 = 16 packed sequences. # Adding the ~1:1 replay roughly doubles the token count vs the game-only run, so # expect ~2x the steps/epoch. Recompute exact steps with `axolotl preprocess`. gradient_accumulation_steps: 1 micro_batch_size: 4 num_epochs: 2 # lowered from 3: replay + fewer epochs both reduce forgetting save_strategy: epoch eval_strategy: epoch # must match save_strategy for load_best_model_at_end # --- Auto-upload the BEST (lowest game eval_loss) checkpoint to the Hub --- # eval_loss here is GAME-ONLY (see test_datasets), so "best" is selected on game # performance, not on how chatty the model is. Needs a write-scoped HF token on the # cluster (HF_TOKEN or `huggingface-cli login`). load_best_model_at_end: true metric_for_best_model: eval_loss greater_is_better: false hub_model_id: cfierro/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5 hub_strategy: end hub_private_repo: false optimizer: adamw_bnb_8bit lr_scheduler: cosine learning_rate: 1e-5 # lowered from 2e-5 to reduce drift away from the base model bf16: auto tf32: false gradient_checkpointing: true resume_from_checkpoint: logging_steps: 1 flash_attention: true warmup_ratio: 0.03 save_total_limit: 3 # keep all per-epoch checkpoints; best is always retained weight_decay: 0.0 special_tokens: pad_token: <|end_of_text|> # DeepSpeed ZeRO Stage 3 — shards weights, gradients, and optimizer across GPUs deepspeed: deepspeed_configs/zero3.json # Verify the data mix + loss mask before training: # axolotl preprocess axolotl_configs/fullft-othello-snake-8b-multi-format-replay.yaml --debug # Confirm: game rows show the masked prefix (-100) + trained moves/<|end_of_text|>; # chat rows show masked user/system turns + trained assistant turns; and the # heterogeneous input_output + chat_template mix concatenates without error. ```

# llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5 This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format and the cfierro/tulu3-sft-replay datasets. It achieves the following results on the evaluation set: - Loss: 0.2137 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 1e-05 - train_batch_size: 4 - eval_batch_size: 4 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - total_train_batch_size: 16 - total_eval_batch_size: 16 - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments - lr_scheduler_type: cosine - lr_scheduler_warmup_steps: 54 - training_steps: 1824 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:----:|:---------------:| | No log | 0 | 0 | 1.1501 | | 0.5652 | 1.0 | 912 | 0.2181 | | 0.3346 | 2.0 | 1824 | 0.2137 | ### Framework versions - Transformers 4.57.3 - Pytorch 2.9.0+cu128 - Datasets 3.5.0 - Tokenizers 0.22.2