llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5/README.md

library_name, license, base_model, tags, datasets, model-index

library_name	license	base_model	tags	datasets	model-index
transformers	llama3.1	meta-llama/Llama-3.1-8B-Instruct	axolotl generated_from_trainer	cfierro/othello-snake-llama3-multi-format cfierro/othello-snake-llama3-multi-format cfierro/othello-snake-llama3-multi-format cfierro/othello-snake-llama3-multi-format cfierro/tulu3-sft-replay	name results llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5

See axolotl config

axolotl version: 0.10.0

base_model: meta-llama/Llama-3.1-8B-Instruct
model_type: LlamaForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false

# ============================================================================
# Full FT of the Othello-snake task WITH a chat-data replay mix, to fight the
# catastrophic forgetting of instruction-following seen in the game-only run
# (the model collapsed to emitting only game moves). Two data sources are mixed:
#   1. The game, as a raw `input_output` continuation document (masked prefix),
#      here in the MULTI-FORMAT variant: ~70% space-separated moves plus ~5% each
#      of several alternative notations (Player 1/2, black/white, turn N, comma,
#      "(black)/(white)"), so the model is robust to how a game is written down.
#   2. A general chat replay set (Tulu-3 subset) as `type: chat_template`.
# The Llama-3 role tokens on the chat data teach the model a context switch:
# game-like raw context -> moves; chat prompt with an instruction -> language.
# Replay built by data/push_replay_subset.py at a ~1:1 trained-token ratio vs the
# game (full-FT forgets hard, so a generous chat share).
# ============================================================================
chat_template: llama3
datasets:
  # --- Game shards (template-free, loss-masked continuation, multi-format) ---
  # The first shard is sliced to hold out its last 500 games for a game-ONLY
  # validation set (see test_datasets below). The other three shards are full.
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_0_5k[:-500]
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_5k_10k
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_10k_25k
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_25k_50k
  # --- Replay: general chat data (Llama-3 chat-template formatted) ---
  - path: cfierro/tulu3-sft-replay
    type: chat_template
    field_messages: messages
    split: train
train_on_inputs: false          # game: segment labels apply; chat: mask user, train assistant
dataset_prepared_path: /scratch/project/eu-26-55/knowledge-ft/axolotl/datasets/llama-3.1-8b/othello-snake-multi-format-replay
output_dir: /scratch/project/eu-26-55/knowledge-ft/axolotl/models/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5

sequence_len: 1024              # games are short; replay was filtered to <=1024 tokens
sample_packing: true            # pack many short docs per sequence (block-diagonal attn)
eval_sample_packing: false

# --- Validation loss on GAME DATA ONLY ---
# val_set_size: 0 disables the default carve (which would mix game + chat into the
# val set). test_datasets points eval exclusively at a held-out slice of game data,
# so eval_loss == game performance and drives load_best_model_at_end below.
# NOTE: the split-slicing syntax (`[:-500]` / `[-500:]`) requires your Axolotl to
# forward `split` to load_dataset (verify once with `axolotl preprocess ... --debug`).
# Fallback if slicing is unsupported in 0.10.0: hold out a WHOLE shard instead --
# drop `games_25k_50k` from datasets above and use it (full) as the test_dataset.
val_set_size: 0
test_datasets:
  - path: cfierro/othello-snake-llama3-multi-format
    type: input_output
    split: games_0_5k[-500:]

wandb_project: othello-snake-ft
wandb_entity: cfierro
wandb_watch:
wandb_name: llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5
wandb_log_model: "false"

# --- Multi-GPU: 4x A40 (48GB) per node ---
# Full FT of an 8B model needs ZeRO-3 to shard weights+grads+optimizer.
# Effective batch = 4 GPUs * micro 4 * grad_accum 1 = 16 packed sequences.
# Adding the ~1:1 replay roughly doubles the token count vs the game-only run, so
# expect ~2x the steps/epoch. Recompute exact steps with `axolotl preprocess`.
gradient_accumulation_steps: 1
micro_batch_size: 4
num_epochs: 2                   # lowered from 3: replay + fewer epochs both reduce forgetting
save_strategy: epoch
eval_strategy: epoch            # must match save_strategy for load_best_model_at_end

# --- Auto-upload the BEST (lowest game eval_loss) checkpoint to the Hub ---
# eval_loss here is GAME-ONLY (see test_datasets), so "best" is selected on game
# performance, not on how chatty the model is. Needs a write-scoped HF token on the
# cluster (HF_TOKEN or `huggingface-cli login`).
load_best_model_at_end: true
metric_for_best_model: eval_loss
greater_is_better: false
hub_model_id: cfierro/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5
hub_strategy: end
hub_private_repo: false

optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 1e-5             # lowered from 2e-5 to reduce drift away from the base model

bf16: auto
tf32: false

gradient_checkpointing: true
resume_from_checkpoint:
logging_steps: 1
flash_attention: true

warmup_ratio: 0.03
save_total_limit: 3             # keep all per-epoch checkpoints; best is always retained
weight_decay: 0.0
special_tokens:
   pad_token: <|end_of_text|>

# DeepSpeed ZeRO Stage 3 — shards weights, gradients, and optimizer across GPUs
deepspeed: deepspeed_configs/zero3.json

# Verify the data mix + loss mask before training:
#   axolotl preprocess axolotl_configs/fullft-othello-snake-8b-multi-format-replay.yaml --debug
# Confirm: game rows show the masked prefix (-100) + trained moves/<|end_of_text|>;
# chat rows show masked user/system turns + trained assistant turns; and the
# heterogeneous input_output + chat_template mix concatenates without error.

llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5

This model is a fine-tuned version of meta-llama/Llama-3.1-8B-Instruct on the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format, the cfierro/othello-snake-llama3-multi-format and the cfierro/tulu3-sft-replay datasets. It achieves the following results on the evaluation set:

• Loss: 0.2137

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 16
• total_eval_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 54
• training_steps: 1824

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0	0	1.1501
0.5652	1.0	912	0.2181
0.3346	2.0	1824	0.2137

Framework versions

• Transformers 4.57.3
• Pytorch 2.9.0+cu128
• Datasets 3.5.0
• Tokenizers 0.22.2