qwen2.5-0.5b-spliced/README.md

language, license, tags

language	license	tags
en	apache-2.0	qwen qwen2.5 text-generation gguf local-ai apple-silicon

Qwen 2.5 0.5B Spliced GGUF

A highly optimized, 15-layer spliced variant of the Qwen 2.5 0.5B architecture.

Model Description

This repository contains the spliced mixed-precision weights of Qwen 2.5 0.5B, surgically pruned from 16 layers to 15 layers. It features standard configs and tokenizer support for seamless local execution on mobile and edge platforms.

• Format: Safetensors / GGUF (Q4_K_M, IQ4_XS)
• Size: 284 MB (GGUF) / 257 MB (IQ4_XS)
• Target Platforms: Apple Silicon MacBooks (M1/M2/M3/M4) and standard CPU/GPU local runtimes.

Local Quickstart

Run the model natively via standard llama-cli:

llama-cli -ngl 100 -m ./gemma_spliced_fresh_q4km.gguf -p "The mathematical beauty of wavelets lies in" -n 128