38 lines
1.0 KiB
Markdown
38 lines
1.0 KiB
Markdown
---
|
|
library_name: transformers
|
|
pipeline_tag: text-generation
|
|
tags:
|
|
- codewraith
|
|
- code-specification
|
|
- llama
|
|
- lora
|
|
- merged
|
|
base_model: unsloth/Llama-3.1-8B-Instruct
|
|
license: mit
|
|
---
|
|
|
|
# CodeWraith Merged 8B (v8b)
|
|
|
|
Merged Llama 3.1 8B Instruct model fine-tuned for generating technical specifications from Python source code.
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("slenk/codewraith-merged-8b")
|
|
tokenizer = AutoTokenizer.from_pretrained("slenk/codewraith-merged-8b")
|
|
```
|
|
|
|
## Training
|
|
|
|
- **Base model:** unsloth/Llama-3.1-8B-Instruct
|
|
- **Method:** LoRA fine-tuning (r=16), merged into base weights
|
|
- **Dataset:** 197 training pairs (r=32, dropout=0.05) generated by Qwen2.5-Coder-14B-AWQ via vLLM
|
|
- **Evaluation:** 0.98 structural score on 34 held-out examples (24/34 perfect)
|
|
- **Training loss:** 0.11
|
|
|
|
## Project
|
|
|
|
Part of [CodeWraith](https://github.com/derekslenk/flexion-codewraith) -- a teacher-student architecture for automated Python module specification generation.
|