76 lines
2.0 KiB
Markdown
76 lines
2.0 KiB
Markdown
|
---
|
|||
|
|
library_name: gguf
|
|||
|
|
base_model: Elinnos/elinnos-sv-v7-ahb-merged
|
|||
|
|
tags:
|
|||
|
|
- qwen2
|
|||
|
|
- gguf
|
|||
|
|
- quantized
|
|||
|
|
- hardware-design
|
|||
|
|
- systemverilog
|
|||
|
|
- amba-ahb
|
|||
|
|
- vlsi
|
|||
|
|
- rtl
|
|||
|
|
license: apache-2.0
|
|||
|
|
language:
|
|||
|
|
- en
|
|||
|
|
pipeline_tag: text-generation
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
# ELINNOS SV-v7-AHB — GGUF Quantized
|
|||
|
|
|
|||
|
|
GGUF quantized versions of [`Elinnos/elinnos-sv-v7-ahb-merged`](https://huggingface.co/Elinnos/elinnos-sv-v7-ahb-merged), the merged ELINNOS SV-v7-AHB model fine-tuned on AMBA AHB hardware design tasks.
|
|||
|
|
|
|||
|
|
## Available Quantizations
|
|||
|
|
|
|||
|
|
| File | Size | Quant | Description |
|
|||
|
|
|---|---|---|---|
|
|||
|
|
| `elinnos-sv-v7-ahb-Q4_K_M.gguf` | ~4.4 GB | Q4_K_M | **Recommended** — best quality/size trade-off |
|
|||
|
|
| `elinnos-sv-v7-ahb-Q8_0.gguf` | ~7.6 GB | Q8_0 | Near-lossless, use if VRAM allows |
|
|||
|
|
| `elinnos-sv-v7-ahb-f16.gguf` | ~15 GB | F16 | Full precision reference |
|
|||
|
|
|
|||
|
|
## Quick Start (Ollama)
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
ollama run pkelinnos/elinnos-sv-v7-ahb
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Quick Start (llama.cpp)
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
./llama-cli -m elinnos-sv-v7-ahb-Q4_K_M.gguf \
|
|||
|
|
--ctx-size 8192 \
|
|||
|
|
--temp 0 \
|
|||
|
|
-p "Design an AHB-Lite slave with 4 read/write registers."
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Quick Start (Python — llama-cpp-python)
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from llama_cpp import Llama
|
|||
|
|
|
|||
|
|
llm = Llama(
|
|||
|
|
model_path="elinnos-sv-v7-ahb-Q4_K_M.gguf",
|
|||
|
|
n_ctx=8192,
|
|||
|
|
n_gpu_layers=-1,
|
|||
|
|
)
|
|||
|
|
output = llm.create_chat_completion(
|
|||
|
|
messages=[
|
|||
|
|
{"role": "system", "content": "You are Elinnos, a hardware design assistant..."},
|
|||
|
|
{"role": "user", "content": "Design an AHB bus matrix for 2 masters and 3 slaves."},
|
|||
|
|
],
|
|||
|
|
temperature=0,
|
|||
|
|
max_tokens=2048,
|
|||
|
|
)
|
|||
|
|
print(output["choices"][0]["message"]["content"])
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Model Details
|
|||
|
|
|
|||
|
|
See the full model card at [`Elinnos/elinnos-sv-v7-ahb`](https://huggingface.co/Elinnos/elinnos-sv-v7-ahb).
|
|||
|
|
|
|||
|
|
- **Base**: Qwen2.5-7B-Instruct
|
|||
|
|
- **Adapter chain**: v3 → v4 → v5 → v6 → v7 (merged)
|
|||
|
|
- **LoRA**: r=96, α=192, target: q/k/v/o/gate/up/down proj
|
|||
|
|
- **Training**: 4 epochs, best eval_loss=0.5408
|
|||
|
|
- **Conversion**: llama.cpp `convert_hf_to_gguf.py` + `llama-quantize`
|