30 lines
880 B
Markdown
30 lines
880 B
Markdown
|
---
|
||
|
|
language:
|
||
|
|
- en
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- qwen
|
||
|
|
- qwen2.5
|
||
|
|
- text-generation
|
||
|
|
- gguf
|
||
|
|
- local-ai
|
||
|
|
- apple-silicon
|
||
|
|
---
|
||
|
|
|
||
|
|
# Qwen 2.5 0.5B Spliced GGUF
|
||
|
|
|
||
|
|
A highly optimized, 15-layer spliced variant of the **Qwen 2.5 0.5B** architecture.
|
||
|
|
|
||
|
|
## Model Description
|
||
|
|
This repository contains the spliced mixed-precision weights of Qwen 2.5 0.5B, surgically pruned from 16 layers to 15 layers. It features standard configs and tokenizer support for seamless local execution on mobile and edge platforms.
|
||
|
|
|
||
|
|
* **Format**: Safetensors / GGUF (`Q4_K_M`, `IQ4_XS`)
|
||
|
|
* **Size**: 284 MB (GGUF) / 257 MB (IQ4_XS)
|
||
|
|
* **Target Platforms**: Apple Silicon MacBooks (M1/M2/M3/M4) and standard CPU/GPU local runtimes.
|
||
|
|
|
||
|
|
## Local Quickstart
|
||
|
|
Run the model natively via standard `llama-cli`:
|
||
|
|
```bash
|
||
|
|
llama-cli -ngl 100 -m ./gemma_spliced_fresh_q4km.gguf -p "The mathematical beauty of wavelets lies in" -n 128
|
||
|
|
```
|