76 lines
2.0 KiB
Markdown
76 lines
2.0 KiB
Markdown
---
|
||
library_name: gguf
|
||
base_model: Elinnos/elinnos-sv-v7-ahb-merged
|
||
tags:
|
||
- qwen2
|
||
- gguf
|
||
- quantized
|
||
- hardware-design
|
||
- systemverilog
|
||
- amba-ahb
|
||
- vlsi
|
||
- rtl
|
||
license: apache-2.0
|
||
language:
|
||
- en
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# ELINNOS SV-v7-AHB — GGUF Quantized
|
||
|
||
GGUF quantized versions of [`Elinnos/elinnos-sv-v7-ahb-merged`](https://huggingface.co/Elinnos/elinnos-sv-v7-ahb-merged), the merged ELINNOS SV-v7-AHB model fine-tuned on AMBA AHB hardware design tasks.
|
||
|
||
## Available Quantizations
|
||
|
||
| File | Size | Quant | Description |
|
||
|---|---|---|---|
|
||
| `elinnos-sv-v7-ahb-Q4_K_M.gguf` | ~4.4 GB | Q4_K_M | **Recommended** — best quality/size trade-off |
|
||
| `elinnos-sv-v7-ahb-Q8_0.gguf` | ~7.6 GB | Q8_0 | Near-lossless, use if VRAM allows |
|
||
| `elinnos-sv-v7-ahb-f16.gguf` | ~15 GB | F16 | Full precision reference |
|
||
|
||
## Quick Start (Ollama)
|
||
|
||
```bash
|
||
ollama run pkelinnos/elinnos-sv-v7-ahb
|
||
```
|
||
|
||
## Quick Start (llama.cpp)
|
||
|
||
```bash
|
||
./llama-cli -m elinnos-sv-v7-ahb-Q4_K_M.gguf \
|
||
--ctx-size 8192 \
|
||
--temp 0 \
|
||
-p "Design an AHB-Lite slave with 4 read/write registers."
|
||
```
|
||
|
||
## Quick Start (Python — llama-cpp-python)
|
||
|
||
```python
|
||
from llama_cpp import Llama
|
||
|
||
llm = Llama(
|
||
model_path="elinnos-sv-v7-ahb-Q4_K_M.gguf",
|
||
n_ctx=8192,
|
||
n_gpu_layers=-1,
|
||
)
|
||
output = llm.create_chat_completion(
|
||
messages=[
|
||
{"role": "system", "content": "You are Elinnos, a hardware design assistant..."},
|
||
{"role": "user", "content": "Design an AHB bus matrix for 2 masters and 3 slaves."},
|
||
],
|
||
temperature=0,
|
||
max_tokens=2048,
|
||
)
|
||
print(output["choices"][0]["message"]["content"])
|
||
```
|
||
|
||
## Model Details
|
||
|
||
See the full model card at [`Elinnos/elinnos-sv-v7-ahb`](https://huggingface.co/Elinnos/elinnos-sv-v7-ahb).
|
||
|
||
- **Base**: Qwen2.5-7B-Instruct
|
||
- **Adapter chain**: v3 → v4 → v5 → v6 → v7 (merged)
|
||
- **LoRA**: r=96, α=192, target: q/k/v/o/gate/up/down proj
|
||
- **Training**: 4 epochs, best eval_loss=0.5408
|
||
- **Conversion**: llama.cpp `convert_hf_to_gguf.py` + `llama-quantize`
|