GLM-Z1-9B-0414-GGUF/README.md

license, language, pipeline_tag, library_name, base_model

license	language	pipeline_tag	library_name	base_model
mit	zh en	text-generation	transformers	THUDM/GLM-Z1-9B-0414

Melvin56/GLM-Z1-9B-0414-GGUF

Original Model : THUDM/GLM-Z1-9B-0414

Llama.cpp build: ced44be3 (5199)

I used imatrix to create all these quants using this Dataset.

---

Update-02

• [Fixed Quant] Re-quantized all quants to fix this issue. #13099 #13140

---

	CPU (AVX2)	CPU (ARM NEON)	Metal	cuBLAS	rocBLAS	SYCL	CLBlast	Vulkan	Kompute
K-quants	✅	✅	✅	✅	✅	✅	✅ 🐢5	✅ 🐢5	❌
I-quants	✅ 🐢4	✅ 🐢4	✅ 🐢4	✅	✅	Partial¹	❌	❌	❌

✅: feature works
🚫: feature does not work
❓: unknown, please contribute if you can test it youself
🐢: feature is slow
¹: IQ3_S and IQ1_S, see #5886
²: Only with -ngl 0
³: Inference is 50% slower
⁴: Slower than K-quants of comparable size
⁵: Slower than cuBLAS/rocBLAS on similar cards
⁶: Only q8_0 and iq4_nl