rwkv7-g1g-7.2b-GGUF/README.md

license, language, tags, base_model, pipeline_tag

license	language	tags	base_model	pipeline_tag
apache-2.0	en zh fr es de pt ru it ja ko vi ar	text-generation causal-lm rwkv gguf llama.cpp lmstudio	BlinkDL/rwkv7-g1	text-generation

RWKV7-G1G 7.2B - GGUF Quantizations

GGUF conversions of BlinkDL/rwkv7-g1g-7.2b (Apache 2.0)
for use with llama.cpp, LM Studio, and other GGUF-compatible runtimes.

All quants converted from BF16 base for maximum lossless quality.
Static quantizations — no importance matrix (imatrix) was used. Check out this for imatrix quantizations: RemySkye/rwkv7-g1g-7.2b-i1-GGUF

S/M/L K-quant variants had no effect on size for RWKV architecture — only _M variants are included.

See UncheatableEval for benchmarks.

Note: Q4_0, Q3_K_M, and Q2_K were tested but produced completely unintelligible / broken output without an importance matrix. For lower BPW quantizations (below ~4.5 BPW), an imatrix would be required — these quants currently use no imatrix at all. Stick to Q4_K_M or higher for usable results.