初始化项目,由ModelHub XC社区提供模型

Model: wvnvwn/Meta-Llama-3-8B-Instruct-fedavg-v0
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-25 21:22:20 +08:00
commit deec78727a
14 changed files with 2608 additions and 0 deletions

85
README.md Normal file
View File

@@ -0,0 +1,85 @@
---
library_name: transformers
pipeline_tag: text-generation
base_model:
- meta-llama/Meta-Llama-3-8B-Instruct
tags:
- peft
- lora
- merged
---
# Meta-Llama-3-8B-Instruct-fedavg-v0
This repository contains a full merged model produced by applying a PEFT LoRA adapter to its base model. It is intended for reproducible evaluation without requiring a separate adapter loading path.
## Quick Start
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "wvnvwn/Meta-Llama-3-8B-Instruct-fedavg-v0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
```
Run the bundled FSL evaluation wrapper:
```bash
bash fsl/src/evaluation/overall_eval.sh \
--model_name wvnvwn/Meta-Llama-3-8B-Instruct-fedavg-v0 \
--model_type instruct \
--max_samples 100 \
--output_root fsl/results/evaluation/Meta-Llama-3-8B-Instruct-fedavg-v0
```
## Training Procedure
- Procedure: Federated LoRA fine-tuning followed by adapter aggregation. The resulting PEFT LoRA adapter was merged into the base model for reproducible evaluation.
- Algorithm: `fedavg`
- Training data: `data_hetero_with_4_tasks`
- Number of clients: `8`
- Communication round/checkpoint: `3`
- Local epochs: `3`
- Local batch size: `256`
- Local micro batch size: `16`
- Local learning rate: `0.0003`
Original training command:
Not specified
## Merge Metadata
- Base model: `meta-llama/Meta-Llama-3-8B-Instruct`
- Adapter source: `/NHNHOME/0226010080_A/BASE/jongbokwon/FLS/fsl/outputs/fedavg/8/2`
- PEFT type: `LORA`
- Task type: `CAUSAL_LM`
- LoRA rank: `16`
- LoRA alpha: `16`
- Target modules: `up_proj`, `v_proj`, `gate_proj`, `q_proj`, `k_proj`, `o_proj`, `down_proj`
- Merged at UTC: `2026-05-22T05:24:31.213169+00:00`
## Framework Versions
- python: `3.10.20`
- platform: `Linux-6.8.0-100-generic-x86_64-with-glibc2.39`
- torch: `2.11.0+cu130`
- cuda: `13.0`
- transformers: `4.57.1`
- peft: `0.19.1`
- accelerate: `1.13.0`
- huggingface_hub: `0.36.2`
- safetensors: `0.7.0`
- vllm: `0.20.2`
- wandb: `0.27.0`
## Notes
- This is a merged full-weight model, not an adapter-only checkpoint.
- Redistribution/access should follow the base model license and access policy.
- `merge_info.json` in this repository records the adapter and merge configuration.