commit a99a970b8a59e9d2c2f7f29db6e50c4595459058 Author: ModelHub XC Date: Sun May 10 01:52:11 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: mradermacher/Austral-GLM4-SFT-i1-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..601cda0 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,59 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +imatrix.dat filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text +Austral-GLM4-SFT.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Austral-GLM4-SFT.i1-IQ1_M.gguf b/Austral-GLM4-SFT.i1-IQ1_M.gguf new file mode 100644 index 0000000..4965104 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ1_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11820b99ca8ba2353330cc531798168c35032cf1cbc4d3d8bd856214b39acb02 +size 7907190112 diff --git a/Austral-GLM4-SFT.i1-IQ1_S.gguf b/Austral-GLM4-SFT.i1-IQ1_S.gguf new file mode 100644 index 0000000..66bd059 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ1_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9e955d47324c373ac1396377af84503efcfd4ac5041aa826430f6a4b8007d900 +size 7267046752 diff --git a/Austral-GLM4-SFT.i1-IQ2_M.gguf b/Austral-GLM4-SFT.i1-IQ2_M.gguf new file mode 100644 index 0000000..ec3d1d1 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ2_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:74bffe49fecdb380e629e2551a5aa7eb35aad63d16f713d44a08b85dd26905c6 +size 11271994592 diff --git a/Austral-GLM4-SFT.i1-IQ2_S.gguf b/Austral-GLM4-SFT.i1-IQ2_S.gguf new file mode 100644 index 0000000..7af7bff --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ2_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:269d009f55b5753f6298557088f128768dd9aaee02d186dcecf2980a41348b03 +size 10418470112 diff --git a/Austral-GLM4-SFT.i1-IQ2_XS.gguf b/Austral-GLM4-SFT.i1-IQ2_XS.gguf new file mode 100644 index 0000000..c805c27 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ2_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:68ad62b89af82a671cb9dad2412cc2f3519cd9622e0ae59c47f4b343dfec9a33 +size 9899578720 diff --git a/Austral-GLM4-SFT.i1-IQ2_XXS.gguf b/Austral-GLM4-SFT.i1-IQ2_XXS.gguf new file mode 100644 index 0000000..6fbf55c --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ2_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d3e0906a184d5f6caaec7f7728ec48eb4d70dad637c4910342d1040d1e883228 +size 8974095712 diff --git a/Austral-GLM4-SFT.i1-IQ3_M.gguf b/Austral-GLM4-SFT.i1-IQ3_M.gguf new file mode 100644 index 0000000..bd743f4 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ3_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d2f0300b2732247832242eef9f936b289d6fb46e098edfc7dc8f17abdaa5003c +size 14820256032 diff --git a/Austral-GLM4-SFT.i1-IQ3_S.gguf b/Austral-GLM4-SFT.i1-IQ3_S.gguf new file mode 100644 index 0000000..eb041ce --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ3_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:49951b7d32dda9483a796adcad051a1a9088906442c83a106aabf767d81e186e +size 14382827808 diff --git a/Austral-GLM4-SFT.i1-IQ3_XS.gguf b/Austral-GLM4-SFT.i1-IQ3_XS.gguf new file mode 100644 index 0000000..2ffbc4f --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ3_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aa014c965eee26c0b4f0ade772290e293b478253b5b2098476b836c0950da1ed +size 13659924768 diff --git a/Austral-GLM4-SFT.i1-IQ3_XXS.gguf b/Austral-GLM4-SFT.i1-IQ3_XXS.gguf new file mode 100644 index 0000000..9e75d60 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ3_XXS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:991431fd3845723ccfb6e99fb80af35f7d4df01d6f60e119467566b6a974cbf4 +size 12782681312 diff --git a/Austral-GLM4-SFT.i1-IQ4_XS.gguf b/Austral-GLM4-SFT.i1-IQ4_XS.gguf new file mode 100644 index 0000000..e7b20f7 --- /dev/null +++ b/Austral-GLM4-SFT.i1-IQ4_XS.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ac6dbf92205e306dad610130a6cd40008499eb7a012c4746b0d42cabad0983af +size 17597718656 diff --git a/Austral-GLM4-SFT.i1-Q2_K.gguf b/Austral-GLM4-SFT.i1-Q2_K.gguf new file mode 100644 index 0000000..bd993a6 --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89c1dd63ac054bba6ea08d10bb00c5dad138b7bb718f9ab1f7f34c3d3b4eed4f +size 12290789792 diff --git a/Austral-GLM4-SFT.i1-Q2_K_S.gguf b/Austral-GLM4-SFT.i1-Q2_K_S.gguf new file mode 100644 index 0000000..0ce00d5 --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q2_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:51add4252ee57bdc7f6e2a4e37e4c274f1023b26cd63eb4e2e721d7579cd9dcf +size 11412173216 diff --git a/Austral-GLM4-SFT.i1-Q3_K_L.gguf b/Austral-GLM4-SFT.i1-Q3_K_L.gguf new file mode 100644 index 0000000..dff4b24 --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:43915192a03823eb4caf41584d7d770922acd9686a04dfa1fd85ae725796e2f9 +size 17214695712 diff --git a/Austral-GLM4-SFT.i1-Q3_K_M.gguf b/Austral-GLM4-SFT.i1-Q3_K_M.gguf new file mode 100644 index 0000000..0e1b0ad --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6efd66215c03a680ff8e554c37793b294feeff448706d72a53038128327f6c4a +size 15888967968 diff --git a/Austral-GLM4-SFT.i1-Q3_K_S.gguf b/Austral-GLM4-SFT.i1-Q3_K_S.gguf new file mode 100644 index 0000000..4e493e4 --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c88179fe8a860112639c46d9512610bd0ee1c20d871e2997ad261636fb0700f6 +size 14370085152 diff --git a/Austral-GLM4-SFT.i1-Q4_0.gguf b/Austral-GLM4-SFT.i1-Q4_0.gguf new file mode 100644 index 0000000..1f9a9e9 --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q4_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:14b1df6775553975910f54eb5f6967da9b07fa87af7b7abf1232cbe02d2687f6 +size 18633164096 diff --git a/Austral-GLM4-SFT.i1-Q4_1.gguf b/Austral-GLM4-SFT.i1-Q4_1.gguf new file mode 100644 index 0000000..132453b --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q4_1.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6827dee14b11171c55323e9ff538d9e6b53b423c0b98ddd79982c0aaaca2929 +size 20548243136 diff --git a/Austral-GLM4-SFT.i1-Q4_K_M.gguf b/Austral-GLM4-SFT.i1-Q4_K_M.gguf new file mode 100644 index 0000000..b7144e3 --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7bed8ecbbb20943ed120495aa9de03e8e30dcc32879a1ce0bf9ba6648faf471 +size 19678258496 diff --git a/Austral-GLM4-SFT.i1-Q4_K_S.gguf b/Austral-GLM4-SFT.i1-Q4_K_S.gguf new file mode 100644 index 0000000..33bd8cf --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8d8a04f917883008208f150ba700fb0ed140e7743fd8e467fe2e3fe9b8ab65fe +size 18695882048 diff --git a/Austral-GLM4-SFT.i1-Q5_K_M.gguf b/Austral-GLM4-SFT.i1-Q5_K_M.gguf new file mode 100644 index 0000000..0eab8fa --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:37adbd4dc8e189e1aa4310a516324a35cb786b256a882c79d9b0fdeb39bac293 +size 23095539776 diff --git a/Austral-GLM4-SFT.i1-Q5_K_S.gguf b/Austral-GLM4-SFT.i1-Q5_K_S.gguf new file mode 100644 index 0000000..eb9c46f --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:122fa9ded486a4a1d20aa24e5439c0407a94896ee38d1982ed99b1c4dd736c28 +size 22525253696 diff --git a/Austral-GLM4-SFT.i1-Q6_K.gguf b/Austral-GLM4-SFT.i1-Q6_K.gguf new file mode 100644 index 0000000..be7249b --- /dev/null +++ b/Austral-GLM4-SFT.i1-Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:80dc473f88c478fcf0bc2ffe853a5bbd76565a9e4be5377686a745eb54ac324e +size 26726401152 diff --git a/README.md b/README.md new file mode 100644 index 0000000..4cdea9c --- /dev/null +++ b/README.md @@ -0,0 +1,100 @@ +--- +base_model: Delta-Vector/Austral-GLM4-SFT +datasets: +- Delta-Vector/Orion-LN-V1-ShareGPT +- Delta-Vector/Orion-Personamaxx-RP +- Delta-Vector/Orion-Co-Writer-51K +- Delta-Vector/Orion-Praxis-Co-Writer +- Delta-Vector/Orion-Shoujo-AI-Filtered-ShareGPT +- Delta-Vector/Orion-PIPPA-Cleaned-V2 +- Delta-Vector/Orion-Alpindale-LN-ShareGPT +- Delta-Vector/Orion-Deepseek-V3-RP-Filtered +- Delta-Vector/Orion-Books-V2-ShareGPT +- Delta-Vector/Orion-Light-Novels-Roleplay-Logs-Books-Oh-My-duplicate-turns-removed +- Delta-Vector/Orion-RP-Guild +- Delta-Vector/Orion-Creative_Writing-Complexity +- Delta-Vector/Orion-Deepseek-R1-RP-Filtered +- Delta-Vector/Orion-Storium-Prefixed-Clean +- Delta-Vector/Orion-Misc-Sharegpt-Prefixed +- Delta-Vector/Orion-LIMARP-Complexity +- Delta-Vector/Orion-BlueSky-10K-Complexity +- Delta-Vector/Orion-OpenCAI-ShareGPT +- Delta-Vector/Orion-Roleplay-Logs-Sharegpt-Ngram-cleaned +- Delta-Vector/Orion-vanilla-backrooms-claude-sharegpt +language: +- en +library_name: transformers +mradermacher: + readme_rev: 1 +quantized_by: mradermacher +--- +## About + + + + + + +weighted/imatrix quants of https://huggingface.co/Delta-Vector/Austral-GLM4-SFT + + + +***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Austral-GLM4-SFT-i1-GGUF).*** + +static quants are available at https://huggingface.co/mradermacher/Austral-GLM4-SFT-GGUF +## Usage + +If you are unsure how to use GGUF files, refer to one of [TheBloke's +READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for +more details, including on how to concatenate multi-part files. + +## Provided Quants + +(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) + +| Link | Type | Size/GB | Notes | +|:-----|:-----|--------:|:------| +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ1_S.gguf) | i1-IQ1_S | 7.4 | for the desperate | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ1_M.gguf) | i1-IQ1_M | 8.0 | mostly desperate | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 9.1 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ2_XS.gguf) | i1-IQ2_XS | 10.0 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ2_S.gguf) | i1-IQ2_S | 10.5 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ2_M.gguf) | i1-IQ2_M | 11.4 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q2_K_S.gguf) | i1-Q2_K_S | 11.5 | very low quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q2_K.gguf) | i1-Q2_K | 12.4 | IQ3_XXS probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 12.9 | lower quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ3_XS.gguf) | i1-IQ3_XS | 13.8 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q3_K_S.gguf) | i1-Q3_K_S | 14.5 | IQ3_XS probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ3_S.gguf) | i1-IQ3_S | 14.5 | beats Q3_K* | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ3_M.gguf) | i1-IQ3_M | 14.9 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q3_K_M.gguf) | i1-Q3_K_M | 16.0 | IQ3_S probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q3_K_L.gguf) | i1-Q3_K_L | 17.3 | IQ3_M probably better | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-IQ4_XS.gguf) | i1-IQ4_XS | 17.7 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q4_0.gguf) | i1-Q4_0 | 18.7 | fast, low quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q4_K_S.gguf) | i1-Q4_K_S | 18.8 | optimal size/speed/quality | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q4_K_M.gguf) | i1-Q4_K_M | 19.8 | fast, recommended | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q4_1.gguf) | i1-Q4_1 | 20.6 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q5_K_S.gguf) | i1-Q5_K_S | 22.6 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q5_K_M.gguf) | i1-Q5_K_M | 23.2 | | +| [GGUF](https://huggingface.co/mradermacher/Austral-GLM4-SFT-i1-GGUF/resolve/main/Austral-GLM4-SFT.i1-Q6_K.gguf) | i1-Q6_K | 26.8 | practically like static Q6_K | + +Here is a handy graph by ikawrakow comparing some lower-quality quant +types (lower is better): + +![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png) + +And here are Artefact2's thoughts on the matter: +https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9 + +## FAQ / Model Request + +See https://huggingface.co/mradermacher/model_requests for some answers to +questions you might have and/or if you want some other model quantized. + +## Thanks + +I thank my company, [nethype GmbH](https://www.nethype.de/), for letting +me use its servers and providing upgrades to my workstation to enable +this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to. + + diff --git a/imatrix.dat b/imatrix.dat new file mode 100644 index 0000000..50cb5be --- /dev/null +++ b/imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8669e6d10980a5f4e37e1c5d92448b5c7f145fc4aad4dd0b17d1d6aa7803e00d +size 13129554