commit 490edbb0a48e7d125f3e97f4b334ad8ff5b348f5 Author: ModelHub XC Date: Sun Sep 27 05:51:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..99faee7 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,60 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Qwen2.5-3B-Instruct-Flux.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ1_M.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ1_M.gguf new file mode 100644 index 0000000..9316278 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2453e3d3030180b2f8abd949b3c345a5d56ddb7ffbfe30de1102f46abc52d075 +size 850028192 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ1_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ1_S.gguf new file mode 100644 index 0000000..0e327e5 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9b7b8bc6a7ca5bb5d7d1f15abbf269df0690cc8ec2a1b5548a3ddaf5456a1ae6 +size 791094944 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ2_M.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_M.gguf new file mode 100644 index 0000000..d26df3c --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de856ffba99cdf76c8c08c0acf1ef4414fbb24b75eecccedaa1bd76af2c170fe +size 1140516512 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ2_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_S.gguf new file mode 100644 index 0000000..4fd69de --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f44dc3592cb9d1fc7437de0230b301e4d45560d0a8c5a0d4a780226f739cac26 +size 1061938848 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XS.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XS.gguf new file mode 100644 index 0000000..0950b18 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55bc4701cc52e11c318fdc27378f1dde0b3ee3329df922e15e0374b78789c8d8 +size 1031546528 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XXS.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..9f9e9ff --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:64e7da26d2c8be5d53a78a2fe918acc0e8e817b5060ed9822c9a73b8c3478a99 +size 948250272 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ3_M.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_M.gguf new file mode 100644 index 0000000..e7f3c99 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7403cb355e8c86e8785a487f5466fa4959b479de6a42018f9aec8e89801df1d +size 1488895648 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ3_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_S.gguf new file mode 100644 index 0000000..8f7bfa1 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b83cefb3245ff210f64bdc7453f347f7816413924aa16383692bbb42a84ab1f +size 1456864928 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XS.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XS.gguf new file mode 100644 index 0000000..bb8dd3f --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0a420b689ed811bf722349c2628df69f09d2bd27deed6c9c66be7543814ccd22 +size 1391836832 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XXS.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..57d3741 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd77901bc61c7f88f6f18b4f54decb2f6e6f9db6036c13eadb9e8bb00ad2d6c2 +size 1282827936 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ4_NL.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ4_NL.gguf new file mode 100644 index 0000000..ff68a4b --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ4_NL.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:174eb61346f74e3f484105bf61d2bb66099945a3c4bf3dcebf0f5285bf4848f5 +size 1825210016 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-IQ4_XS.gguf b/Qwen2.5-3B-Instruct-Flux.i1-IQ4_XS.gguf new file mode 100644 index 0000000..7b4da2c --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d37881825736bb5bb617632a04d24b35e59ea93c7538bf35124a9a46b0eb936 +size 1739095712 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q2_K.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q2_K.gguf new file mode 100644 index 0000000..ed2e12d --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f44691b8cbbcee36c27ca8a6a1743d7baf85b9a4170e1e53ec405e0fbc9ec06b +size 1274756768 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q2_K_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q2_K_S.gguf new file mode 100644 index 0000000..8670388 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc5ef06395fc83323e68f7017e04351f9cd6307b1d55519c1d5fba12da23d3de +size 1198128800 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_L.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_L.gguf new file mode 100644 index 0000000..f676782 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:50dc0c17e2c6aaf926d334655a0db134a0d76a27b1a166e047da6845f1aec756 +size 1707392672 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_M.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_M.gguf new file mode 100644 index 0000000..1d9f273 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:de854f231cf7bd0e3d178443ff85d6e2b991646b250a97091e3d8202465f5117 +size 1590476448 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_S.gguf new file mode 100644 index 0000000..b217729 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28b814982bb0021ad2691a25796588d2d24b9096d705b438a2616bc3d2d87259 +size 1454358176 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q4_0.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q4_0.gguf new file mode 100644 index 0000000..9c3addf --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b9bf9e7045754ab536f99e4976e3b8082272b0e46e7874b308470ea4a439e8f8 +size 1828486816 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q4_1.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q4_1.gguf new file mode 100644 index 0000000..100a0ed --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ffcca69fc30f96c74d9696053d9494e87e0b927741828bff12aaeb22fc6ffe6 +size 1996258976 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_M.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_M.gguf new file mode 100644 index 0000000..ae71cae --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ce52e97d8aedc9cb0642df9ca2dc12aba0d72727d439079daba5a50b594009c4 +size 1929903776 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_S.gguf new file mode 100644 index 0000000..c22aea4 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bd5257351840a18031b28813eb9745dd7ca1a1f39ebfc13b913020fce9fc85f9 +size 1834385056 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_M.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_M.gguf new file mode 100644 index 0000000..bcc41e6 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:58cb70c20a0692221b42be155a42929498f26b2befca2a58f6e504728be90bcb +size 2224815776 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_S.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_S.gguf new file mode 100644 index 0000000..1e7c9a3 --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a695ca8f116f1cb7b62fb19a885d0a3c06c5d89e73e26e89d31bca2351186881 +size 2169667232 diff --git a/Qwen2.5-3B-Instruct-Flux.i1-Q6_K.gguf b/Qwen2.5-3B-Instruct-Flux.i1-Q6_K.gguf new file mode 100644 index 0000000..d452bdf --- /dev/null +++ b/Qwen2.5-3B-Instruct-Flux.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:488b18c0b0906a8b7b061e8e4c23009e94f859fd1615586786463a9097d21cad +size 2538159776 diff --git a/README.md b/README.md new file mode 100644 index 0000000..de7fea7 --- /dev/null +++ b/README.md @@ -0,0 +1,81 @@ +--- +base_model: mrkrak3n/Qwen2.5-3B-Instruct-Flux +datasets: +- kadirnar/fluxdev_controlnet_16k +language: +- en +library_name: transformers +quantized_by: mradermacher +tags: +- unsloth +- trl +- sft +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/mrkrak3n/Qwen2.5-3B-Instruct-Flux + + +static quants are available at https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ1_S.gguf) | i1-IQ1_S | 0.9 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ1_M.gguf) | i1-IQ1_M | 1.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 1.0 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ2_XS.gguf) | i1-IQ2_XS | 1.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ2_S.gguf) | i1-IQ2_S | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ2_M.gguf) | i1-IQ2_M | 1.2 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q2_K_S.gguf) | i1-Q2_K_S | 1.3 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q2_K.gguf) | i1-Q2_K | 1.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 1.4 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ3_XS.gguf) | i1-IQ3_XS | 1.5 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_S.gguf) | i1-Q3_K_S | 1.6 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ3_S.gguf) | i1-IQ3_S | 1.6 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ3_M.gguf) | i1-IQ3_M | 1.6 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_M.gguf) | i1-Q3_K_M | 1.7 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.8 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.8 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-IQ4_NL.gguf) | i1-IQ4_NL | 1.9 | prefer IQ4_XS | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q4_0.gguf) | i1-Q4_0 | 1.9 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.9 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q4_K_M.gguf) | i1-Q4_K_M | 2.0 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q4_1.gguf) | i1-Q4_1 | 2.1 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_S.gguf) | i1-Q5_K_S | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q5_K_M.gguf) | i1-Q5_K_M | 2.3 | | +| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-3B-Instruct-Flux-i1-GGUF/resolve/main/Qwen2.5-3B-Instruct-Flux.i1-Q6_K.gguf) | i1-Q6_K | 2.6 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..a3fb1db --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bf40324fdff2d9c925f28467b81a2e23a01a2cf23e7bc65be7fe474599682df0 +size 3362977