初始化项目,由ModelHub XC社区提供模型

Model: thefabdev/llama-3-docker-ft
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-07-20 05:23:09 +08:00
commit 82161ab018
8 changed files with 199 additions and 0 deletions

92
README.md Normal file
View File

@@ -0,0 +1,92 @@
---
library_name: transformers
base_model: meta-llama/Meta-Llama-3-8B-Instruct
license: llama3
datasets:
- MattCoddity/dockerNLcommands
language:
- en
tags:
- lora
- docker
- text-generation
pipeline_tag: text-generation
---
# llama-3-docker-ft
LoRA fine-tune of `meta-llama/Meta-Llama-3-8B-Instruct` that translates natural-language
requests into Docker CLI commands. Merged adapter weights (base + LoRA), not an
adapter-only checkpoint.
This is a learning-lab artifact ([source notebook and writeup](https://github.com/Fakorede/llm-alignment-lab/tree/main/01_lora_sft)),
not a production model. Treat it as a first fine-tuning exercise, not a benchmarked release.
## Model Details
- **Base model:** `meta-llama/Meta-Llama-3-8B-Instruct`, loaded in 8-bit (`BitsAndBytesConfig(load_in_8bit=True)`)
- **Fine-tuning method:** LoRA (`peft`), `r=16`, `lora_alpha=32`, dropout `0.05`,
targeting `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
(41.9M trainable params, 0.52% of 8.07B total)
- **Merged:** LoRA adapter merged into the base weights before push (`merge_and_unload()`)
- **License:** inherits the [Llama 3 Community License](https://llama.meta.com/llama3/license/) from the base model
## Training Data
[`MattCoddity/dockerNLcommands`](https://huggingface.co/datasets/MattCoddity/dockerNLcommands),
an instruction/input/output dataset pairing natural-language requests with the
corresponding Docker CLI command. Split 80/20 train/validation (seed 42).
## Training Procedure
`transformers.Trainer` + `TrainingArguments`: batch size 2, gradient accumulation 4
(effective batch size 8), `paged_adamw_8bit`, learning rate `2e-4`, 2 epochs (484 steps),
fp16, warmup steps 5, weight decay 0.01.
### Results
| Metric | Value |
|---|---|
| Train loss (last logged step) | 0.307 |
| Train loss (run average) | 0.461 |
| Eval loss | 0.341 |
| Train runtime | ~2738s (single A100 80GB) |
## Uses
**Intended use:** translating short, single-turn natural-language instructions about
containers/images into a Docker CLI command. Example:
\`\`\`python
import transformers
import torch
pipeline = transformers.pipeline(
"text-generation",
model="thefabdev/llama-3-docker-ft",
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a helpful assistant. Translate this sentence in docker command"},
{"role": "user", "content": "Display the information of the last 4 containers."},
]
result = pipeline(messages, max_new_tokens=256, temperature=0.25, top_p=1, repetition_penalty=1.2)
print(result[0]["generated_text"][-1]["content"])
# docker ps --last 4
\`\`\`
**Out of scope:** general-purpose assistant use, multi-turn conversation, any
command generation where correctness/safety of the resulting shell command isn't
independently verified before execution. Generated commands are not validated for
safety and should not be run against production systems without review.
## Limitations
- Fine-tuned on a single small, narrow dataset (Docker CLI only), will not generalize
to other CLIs or general instruction-following.
- Trained for 2 epochs on ~800 examples; not evaluated against a held-out benchmark
beyond the validation split loss above.
- No safety/red-teaming evaluation has been performed on this model.