llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5/README.md

---
library_name: transformers
license: llama3.1
base_model: meta-llama/Llama-3.1-8B-Instruct
tags:
- axolotl
- generated_from_trainer
datasets:
- cfierro/othello-snake-llama3-multi-format
- cfierro/othello-snake-llama3-multi-format
- cfierro/othello-snake-llama3-multi-format
- cfierro/othello-snake-llama3-multi-format
- cfierro/tulu3-sft-replay
model-index:
- name: llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

[<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
<details><summary>See axolotl config</summary>

axolotl version: `0.10.0`
```yaml
base_model: meta-llama/Llama-3.1-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false

# ============================================================================
# Full FT of the Othello-snake task WITH a chat-data replay mix, to fight the
# catastrophic forgetting of instruction-following seen in the game-only run
# (the model collapsed to emitting only game moves). Two data sources are mixed:
#   1. The game, as a raw `input_output` continuation document (masked prefix),
#      here in the MULTI-FORMAT variant: ~70% space-separated moves plus ~5% each
#      of several alternative notations (Player 1/2, black/white, turn N, comma,
#      "(black)/(white)"), so the model is robust to how a game is written down.
#   2. A general chat replay set (Tulu-3 subset) as `type: chat_template`.
# The Llama-3 role tokens on the chat data teach the model a context switch:
# game-like raw context -> moves; chat prompt with an instruction -> language.
# Replay built by data/push_replay_subset.py at a ~1:1 trained-token ratio vs the
# game (full-FT forgets hard, so a generous chat share).
# ============================================================================
chat_template: llama3
datasets:
  # --- Game shards (template-free, loss-masked continuation, multi-format) ---
  # The first shard is sliced to hold out its last 500 games for a game-ONLY
  # validation set (see test_datasets below). The other three shards are full.
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_0_5k[:-500]
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_5k_10k
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_10k_25k
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_25k_50k
  # --- Replay: general chat data (Llama-3 chat-template formatted) ---
  - path: cfierro/tulu3-sft-replay
    type: chat_template
    field_messages: messages
    split: train
train_on_inputs: false          # game: segment labels apply; chat: mask user, train assistant
dataset_prepared_path: /scratch/project/eu-26-55/knowledge-ft/axolotl/datasets/llama-3.1-8b/othello-snake-multi-format-replay
output_dir: /scratch/project/eu-26-55/knowledge-ft/axolotl/models/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5

sequence_len: 1024              # games are short; replay was filtered to <=1024 tokens
sample_packing: true            # pack many short docs per sequence (block-diagonal attn)
eval_sample_packing: false

# --- Validation loss on GAME DATA ONLY ---
# val_set_size: 0 disables the default carve (which would mix game + chat into the
# val set). test_datasets points eval exclusively at a held-out slice of game data,
# so eval_loss == game performance and drives load_best_model_at_end below.
# NOTE: the split-slicing syntax (`[:-500]` / `[-500:]`) requires your Axolotl to
# forward `split` to load_dataset (verify once with `axolotl preprocess ... --debug`).
# Fallback if slicing is unsupported in 0.10.0: hold out a WHOLE shard instead --
# drop `games_25k_50k` from datasets above and use it (full) as the test_dataset.
val_set_size: 0
test_datasets:
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_0_5k[-500:]

wandb_project: othello-snake-ft
wandb_entity: cfierro
wandb_watch:
wandb_name: llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5
wandb_log_model: "false"

# --- Multi-GPU: 4x A40 (48GB) per node ---
# Full FT of an 8B model needs ZeRO-3 to shard weights+grads+optimizer.
# Effective batch = 4 GPUs * micro 4 * grad_accum 1 = 16 packed sequences.
# Adding the ~1:1 replay roughly doubles the token count vs the game-only run, so
# expect ~2x the steps/epoch. Recompute exact steps with `axolotl preprocess`.
gradient_accumulation_steps: 1
micro_batch_size: 4
num_epochs: 2                   # lowered from 3: replay + fewer epochs both reduce forgetting
save_strategy: epoch
eval_strategy: epoch            # must match save_strategy for load_best_model_at_end

# --- Auto-upload the BEST (lowest game eval_loss) checkpoint to the Hub ---
# eval_loss here is GAME-ONLY (see test_datasets), so "best" is selected on game
# performance, not on how chatty the model is. Needs a write-scoped HF token on the
# cluster (HF_TOKEN or `huggingface-cli login`).
load_best_model_at_end: true
metric_for_best_model: eval_loss
greater_is_better: false
hub_model_id: cfierro/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5
hub_strategy: end
hub_private_repo: false

optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 1e-5             # lowered from 2e-5 to reduce drift away from the base model

bf16: auto
tf32: false

gradient_checkpointing: true
resume_from_checkpoint:
logging_steps: 1
flash_attention: true

warmup_ratio: 0.03
save_total_limit: 3             # keep all per-epoch checkpoints; best is always retained
weight_decay: 0.0
special_tokens:
   pad_token: <|end_of_text|>

# DeepSpeed ZeRO Stage 3 — shards weights, gradients, and optimizer across GPUs
deepspeed: deepspeed_configs/zero3.json

# Verify the data mix + loss mask before training:
#   axolotl preprocess axolotl_configs/fullft-othello-snake-8b-multi-format-replay.yaml --debug
# Confirm: game rows show the masked prefix (-100) + trained moves/<|end_of_text|>;
# chat rows show masked user/system turns + trained assistant turns; and the
# heterogeneous input_output + chat_template mix concatenates without error.

```

</details><br>

# llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5

This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format and the cfierro/tulu3-sft-replay datasets.
It achieves the following results on the evaluation set:
- Loss: 0.2137

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- total_train_batch_size: 16
- total_eval_batch_size: 16
- optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 54
- training_steps: 1824

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| No log        | 0     | 0    | 1.1501          |
| 0.5652        | 1.0   | 912  | 0.2181          |
| 0.3346        | 2.0   | 1824 | 0.2137          |


### Framework versions

- Transformers 4.57.3
- Pytorch 2.9.0+cu128
- Datasets 3.5.0
- Tokenizers 0.22.2