2.3 KiB
2.3 KiB
library_name, pipeline_tag, base_model, tags
| library_name | pipeline_tag | base_model | tags | ||||
|---|---|---|---|---|---|---|---|
| transformers | text-generation |
|
|
Meta-Llama-3-8B-Instruct-fedavg-v0
This repository contains a full merged model produced by applying a PEFT LoRA adapter to its base model. It is intended for reproducible evaluation without requiring a separate adapter loading path.
Quick Start
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "wvnvwn/Meta-Llama-3-8B-Instruct-fedavg-v0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
Run the bundled FSL evaluation wrapper:
bash fsl/src/evaluation/overall_eval.sh \
--model_name wvnvwn/Meta-Llama-3-8B-Instruct-fedavg-v0 \
--model_type instruct \
--max_samples 100 \
--output_root fsl/results/evaluation/Meta-Llama-3-8B-Instruct-fedavg-v0
Training Procedure
- Procedure: Federated LoRA fine-tuning followed by adapter aggregation. The resulting PEFT LoRA adapter was merged into the base model for reproducible evaluation.
- Algorithm:
fedavg - Training data:
data_hetero_with_4_tasks - Number of clients:
8 - Communication round/checkpoint:
3 - Local epochs:
3 - Local batch size:
256 - Local micro batch size:
16 - Local learning rate:
0.0003
Original training command:
Not specified
Merge Metadata
- Base model:
meta-llama/Meta-Llama-3-8B-Instruct - Adapter source:
/NHNHOME/0226010080_A/BASE/jongbokwon/FLS/fsl/outputs/fedavg/8/2 - PEFT type:
LORA - Task type:
CAUSAL_LM - LoRA rank:
16 - LoRA alpha:
16 - Target modules:
up_proj,v_proj,gate_proj,q_proj,k_proj,o_proj,down_proj - Merged at UTC:
2026-05-22T05:24:31.213169+00:00
Framework Versions
- python:
3.10.20 - platform:
Linux-6.8.0-100-generic-x86_64-with-glibc2.39 - torch:
2.11.0+cu130 - cuda:
13.0 - transformers:
4.57.1 - peft:
0.19.1 - accelerate:
1.13.0 - huggingface_hub:
0.36.2 - safetensors:
0.7.0 - vllm:
0.20.2 - wandb:
0.27.0
Notes
- This is a merged full-weight model, not an adapter-only checkpoint.
- Redistribution/access should follow the base model license and access policy.
merge_info.jsonin this repository records the adapter and merge configuration.