commit 52604206d8a0ff060534206d05bd3437249854d7 Author: ModelHub XC Date: Fri Oct 16 15:51:17 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Austral-GLM4-Winton-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..065dd1d --- /dev/null +++ b/.gitattributes @@ -0,0 +1,58 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-Winton.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Austral-GLM4-Winton.i1-IQ1_M.gguf b/Austral-GLM4-Winton.i1-IQ1_M.gguf new file mode 100644 index 0000000..01b78ba --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ddbc9c1334783e73aea9205162f632781fbbd19d492380117af9a5b4b2c2b56 +size 7907190112 diff --git a/Austral-GLM4-Winton.i1-IQ1_S.gguf b/Austral-GLM4-Winton.i1-IQ1_S.gguf new file mode 100644 index 0000000..cb84cde --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5f3f076f851b9934b48ab1e07d7c13191bf0f6fbc443e3597ef2acce15b5b6e0 +size 7267046752 diff --git a/Austral-GLM4-Winton.i1-IQ2_M.gguf b/Austral-GLM4-Winton.i1-IQ2_M.gguf new file mode 100644 index 0000000..280171c --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:72d21a83c64b9a8619790d3ed061dd685defddcf0875eb723c3fe2c1ac41f30b +size 11271994592 diff --git a/Austral-GLM4-Winton.i1-IQ2_S.gguf b/Austral-GLM4-Winton.i1-IQ2_S.gguf new file mode 100644 index 0000000..9c502ee --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2f824db955c2aed2df90bab617cb0730cc24160cf7b8fdc60632681dad72d7d0 +size 10418470112 diff --git a/Austral-GLM4-Winton.i1-IQ2_XS.gguf b/Austral-GLM4-Winton.i1-IQ2_XS.gguf new file mode 100644 index 0000000..a753afb --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7ac96bc4dda80685e3b0e8f60f024def35f69215712f7abc1600bbc50c5125e0 +size 9899578720 diff --git a/Austral-GLM4-Winton.i1-IQ2_XXS.gguf b/Austral-GLM4-Winton.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..8bf9078 --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:88545df51eae2ef0d4ccf5d6be647dcf8942f8e5173c895727d582e4d76f73ae +size 8974095712 diff --git a/Austral-GLM4-Winton.i1-IQ3_M.gguf b/Austral-GLM4-Winton.i1-IQ3_M.gguf new file mode 100644 index 0000000..d39ba56 --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:78cb78974ceb3487a253321f0617180307761378ffe879f1e492743981d4d7b4 +size 14820256032 diff --git a/Austral-GLM4-Winton.i1-IQ3_S.gguf b/Austral-GLM4-Winton.i1-IQ3_S.gguf new file mode 100644 index 0000000..be0bf8f --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4ef32cb6c80d5101eec55a5d9990d486907db65a076144c89be7a95f2c8afb66 +size 14382827808 diff --git a/Austral-GLM4-Winton.i1-IQ3_XS.gguf b/Austral-GLM4-Winton.i1-IQ3_XS.gguf new file mode 100644 index 0000000..f129af2 --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6bbae9e5d392e6772be0db99e6232d07cc782dfd166377ac20257dd36b60c1dd +size 13659924768 diff --git a/Austral-GLM4-Winton.i1-IQ3_XXS.gguf b/Austral-GLM4-Winton.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..3191cda --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa28b464b6f9425d79ed264bdead113f653e60963426d3d60c98ad26dbd0ce1c +size 12782681312 diff --git a/Austral-GLM4-Winton.i1-IQ4_XS.gguf b/Austral-GLM4-Winton.i1-IQ4_XS.gguf new file mode 100644 index 0000000..dbfeb78 --- /dev/null +++ b/Austral-GLM4-Winton.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9011da36e368797491fd6ac662ca41f0632b6bd3d1c0d20d0ed5ee32ea238859 +size 17597718656 diff --git a/Austral-GLM4-Winton.i1-Q2_K.gguf b/Austral-GLM4-Winton.i1-Q2_K.gguf new file mode 100644 index 0000000..a46b071 --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c47244ec886580698f29939884d682b1d0d1f7bf0b3d8de94d0bcf3eda1c8139 +size 12290789792 diff --git a/Austral-GLM4-Winton.i1-Q2_K_S.gguf b/Austral-GLM4-Winton.i1-Q2_K_S.gguf new file mode 100644 index 0000000..fccf230 --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7275a3ed4ded27bcf999e456414f64db834cf67556276d8273ca9c95c17437b5 +size 11412173216 diff --git a/Austral-GLM4-Winton.i1-Q3_K_L.gguf b/Austral-GLM4-Winton.i1-Q3_K_L.gguf new file mode 100644 index 0000000..91d617a --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:949f1bd86daeb94bb8d4e8bcf7398d0d0aa686ffd729fdbfe4e85372e439fdc0 +size 17214695712 diff --git a/Austral-GLM4-Winton.i1-Q3_K_M.gguf b/Austral-GLM4-Winton.i1-Q3_K_M.gguf new file mode 100644 index 0000000..c958e83 --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:83d67cc679d0aab5ac2b3e5b4770ee64f5d367444dd4f24c275245d40cbfb449 +size 15888967968 diff --git a/Austral-GLM4-Winton.i1-Q3_K_S.gguf b/Austral-GLM4-Winton.i1-Q3_K_S.gguf new file mode 100644 index 0000000..58b2b7e --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3d82f8e35f50d1aad850baf069f4eba545ab625efdaa1c205b5ac1ddae19332 +size 14370085152 diff --git a/Austral-GLM4-Winton.i1-Q4_0.gguf b/Austral-GLM4-Winton.i1-Q4_0.gguf new file mode 100644 index 0000000..5ebe1bd --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0723e1d5a11b0e27238912ce2f8e9040cb95f02b40b338ad22246d616d54e22d +size 18633164096 diff --git a/Austral-GLM4-Winton.i1-Q4_1.gguf b/Austral-GLM4-Winton.i1-Q4_1.gguf new file mode 100644 index 0000000..adb619b --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:91ad3da06038806dd2b8fd389f768901c5b0e85de11751bd1d6b95e919483dda +size 20548243136 diff --git a/Austral-GLM4-Winton.i1-Q4_K_M.gguf b/Austral-GLM4-Winton.i1-Q4_K_M.gguf new file mode 100644 index 0000000..b14ca9a --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2999ec396119959230bf44c0b43ba27aa5c6bdf6b7d218e36a00161dc2dfd314 +size 19678258496 diff --git a/Austral-GLM4-Winton.i1-Q4_K_S.gguf b/Austral-GLM4-Winton.i1-Q4_K_S.gguf new file mode 100644 index 0000000..375a037 --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d079e17ab21169f6ebb0ade5c69c2041cac050dfcdba1221a3826016a5ce0499 +size 18695882048 diff --git a/Austral-GLM4-Winton.i1-Q5_K_M.gguf b/Austral-GLM4-Winton.i1-Q5_K_M.gguf new file mode 100644 index 0000000..c62edcb --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:39c5a07b77316d936a3121d116c1fa1b97ad5687ca663dfff4e37ee05dab1a12 +size 23095539776 diff --git a/Austral-GLM4-Winton.i1-Q5_K_S.gguf b/Austral-GLM4-Winton.i1-Q5_K_S.gguf new file mode 100644 index 0000000..0d11ae8 --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d4a19904dee22049c3a3c54c2f1754c06ab9520c79c0965a424bcacc1aecdbf8 +size 22525253696 diff --git a/Austral-GLM4-Winton.i1-Q6_K.gguf b/Austral-GLM4-Winton.i1-Q6_K.gguf new file mode 100644 index 0000000..fa2c1f9 --- /dev/null +++ b/Austral-GLM4-Winton.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:639cc1680e9da19f694ec6267156a092905dacaf952470f2f82aad8b8d09c9c1 +size 26726401152 diff --git a/README.md b/README.md new file mode 100644 index 0000000..63fb9c4 --- /dev/null +++ b/README.md @@ -0,0 +1,110 @@ +--- +base_model: Delta-Vector/Austral-32B-GLM4-Winton +datasets: +- Delta-Vector/Tauri-Rep-Remover-KTO +- Delta-Vector/Orion-LN-V1-ShareGPT +- Delta-Vector/Orion-Personamaxx-RP +- Delta-Vector/Orion-Co-Writer-51K +- Delta-Vector/Orion-Praxis-Co-Writer +- Delta-Vector/Orion-Shoujo-AI-Filtered-ShareGPT +- Delta-Vector/Orion-PIPPA-Cleaned-V2 +- Delta-Vector/Orion-Alpindale-LN-ShareGPT +- Delta-Vector/Orion-Deepseek-V3-RP-Filtered +- Delta-Vector/Orion-Books-V2-ShareGPT +- Delta-Vector/Orion-Light-Novels-Roleplay-Logs-Books-Oh-My-duplicate-turns-removed +- Delta-Vector/Orion-RP-Guild +- Delta-Vector/Orion-Creative_Writing-Complexity +- Delta-Vector/Orion-Deepseek-R1-RP-Filtered +- Delta-Vector/Orion-Storium-Prefixed-Clean +- Delta-Vector/Orion-Misc-Sharegpt-Prefixed +- Delta-Vector/Orion-LIMARP-Complexity +- Delta-Vector/Orion-BlueSky-10K-Complexity +- Delta-Vector/Orion-OpenCAI-ShareGPT +- Delta-Vector/Orion-Roleplay-Logs-Sharegpt-Ngram-cleaned +- Delta-Vector/Orion-vanilla-backrooms-claude-sharegpt +language: +- en +library_name: transformers +license: apache-2.0 +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +tags: +- roleplay +- finetune +- axolotl +- adventure +- creative-writing +- GLM4 +- 32B +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/Delta-Vector/Austral-32B-GLM4-Winton + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Austral-GLM4-Winton-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Austral-GLM4-Winton-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ1_S.gguf) | i1-IQ1_S | 7.4 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ1_M.gguf) | i1-IQ1_M | 8.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.1 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.0 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ2_S.gguf) | i1-IQ2_S | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ2_M.gguf) | i1-IQ2_M | 11.4 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q2_K_S.gguf) | i1-Q2_K_S | 11.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q2_K.gguf) | i1-Q2_K | 12.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.8 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ3_S.gguf) | i1-IQ3_S | 14.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ3_M.gguf) | i1-IQ3_M | 14.9 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.0 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.7 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q4_0.gguf) | i1-Q4_0 | 18.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q4_K_M.gguf) | i1-Q4_K_M | 19.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q4_1.gguf) | i1-Q4_1 | 20.6 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.6 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.2 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-Winton-i1-GGUF/resolve/main/Austral-GLM4-Winton.i1-Q6_K.gguf) | i1-Q6_K | 26.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..c6790ba --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:18780b2c9689ec486e7adaca9b5cff88b3718830e5ba3f42872190cd4a114d42 +size 13129554