---
library_name: transformers
pipeline_tag: text-generation
tags:
  - codewraith
  - code-specification
  - llama
  - lora
  - merged
base_model: unsloth/Llama-3.1-8B-Instruct
license: mit
---

# CodeWraith Merged 8B (v8b)

Merged Llama 3.1 8B Instruct model fine-tuned for generating technical specifications from Python source code.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("slenk/codewraith-merged-8b")
tokenizer = AutoTokenizer.from_pretrained("slenk/codewraith-merged-8b")
```

## Training

- **Base model:** unsloth/Llama-3.1-8B-Instruct
- **Method:** LoRA fine-tuning (r=16), merged into base weights
- **Dataset:** 197 training pairs (r=32, dropout=0.05) generated by Qwen2.5-Coder-14B-AWQ via vLLM
- **Evaluation:** 0.98 structural score on 34 held-out examples (24/34 perfect)
- **Training loss:** 0.11

## Project

Part of [CodeWraith](https://github.com/derekslenk/flexion-codewraith) -- a teacher-student architecture for automated Python module specification generation.