初始化项目，由ModelHub XC社区提供模型
Model: mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
60
.gitattributes
vendored
Normal file
60
.gitattributes
vendored
Normal file
@@ -0,0 +1,60 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:69750756aa39716297e89dc3f453a91d807178240c6dbff78ada1f97e28e5a24
|
||||
size 1704888864
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:38a3665b3872ad3f308fe2b1daa0dd17d76ad034cc60039b964b8690a50b4c79
|
||||
size 1595187744
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:59eeb8eb74b340e5ea2c6e9a120bb24e5e4cb188ec0e06b5c15ce70f070693c1
|
||||
size 2334062624
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5d6434b4efd67f9c9d20b38b3a79f18bdd8fdb6bf4cbd7df33edfdae3f508bbd
|
||||
size 2187794464
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5850c5dbe8a13e67586ac034f824b0034a89c7a1057d4df5cc1a49c348c49a60
|
||||
size 2052014624
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6a4dfdd1dc649b05409921e1ae634d331cf102b39ca58c2e2ea8468952abf434
|
||||
size 1887724064
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:af8359105af1bb923e1317a189b8a41bbe9b40ce3360847f49b45512c90e79b8
|
||||
size 2994099744
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c5ca87b8e49dd20bd87520d3f9b2587bced0ec583f8941aefe70b952374891e6
|
||||
size 2897659424
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b360a0594a3b3f3e3b0052fb96a944033b3e7eaacbe9ab57bea325577f7fa353
|
||||
size 2765809184
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0185dedf3b247b2ae4617236123b6c08c593e1ae5c88c27f726139867224d724
|
||||
size 2574538784
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5649af96bd8895237433a8f6e3b6737f15c7ac9f5020e7a824c4fa63a9acbf06
|
||||
size 3685420064
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6c5de945625be602eac519d3e0713f9f90edc011d142a5c1ed2f68f1d1271326
|
||||
size 3504305184
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:de0355dff693c35241f7957fcf473d9b106b293eea9b4ed8f026679dde9e3fb2
|
||||
size 2506719264
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:abf67edb70b906f68c0377152b829f96af6198b1846e28ab1330a2a0718aef44
|
||||
size 2344056864
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bcf4dffe122a5eca784b058b5fd1f32c075eaa0f353b19a480497009d5b51c31
|
||||
size 3417493024
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:316723d46b6b88fb9e694276a0d94b9218a7065c429dd4d21be3cf0422c13fb9
|
||||
size 3166162464
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:df09de12325db4a7b8695d7c5feaf457581f9505903f04f63800a9c11b8033d8
|
||||
size 2878510624
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a377b9b192dabe6ebd0072f91bfa06e0b346d808d7327d293a302701486e83ef
|
||||
size 3676736544
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:be153ca5f31ce868dc506fb61377a5bb25eab91aa294b9fed2bacaa34c52e072
|
||||
size 4038638624
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:bb0eb6d5365cdd0487cb2c9036cb1e0571088c202ff9f339cfe544665a780a68
|
||||
size 3858998304
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:68bb993b4e749c1170ac2143fe1bbdc2c448f6739ebcf9d33a658fb0987230ee
|
||||
size 3687386144
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f637f9deb46bcd74219e30dfb2afad7e4f1f7c7947d92d0ba72837232336a061
|
||||
size 4508582944
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:469307404e7242cc63ec65383ef9f3e8c69618a41eb4680eda328d1309d8ccd4
|
||||
size 4409879584
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f264cb0dc67868f1af1c4a03f405b4e9ca3ca9af7cbb02f028e77c1257e299d1
|
||||
size 5198766624
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0251b6fc2a2f2a1186c8f64f00ba59cf0aa4d6a4d5b88a08734e9cd2aa2c64ac
|
||||
size 5916480
|
||||
112
README.md
Normal file
112
README.md
Normal file
@@ -0,0 +1,112 @@
|
||||
---
|
||||
base_model: DavidAU/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
license: apache-2.0
|
||||
mradermacher:
|
||||
readme_rev: 1
|
||||
quantized_by: mradermacher
|
||||
tags:
|
||||
- programming
|
||||
- code generation
|
||||
- code
|
||||
- coding
|
||||
- coder
|
||||
- chat
|
||||
- code
|
||||
- chat
|
||||
- brainstorm
|
||||
- qwen
|
||||
- instruct
|
||||
- qwen3
|
||||
- qwencoder
|
||||
- brainstorm 20x
|
||||
- creative
|
||||
- all uses cases
|
||||
- Jan-V1
|
||||
- Stargate
|
||||
- SG1
|
||||
- horror
|
||||
- science fiction
|
||||
- fantasy
|
||||
- finetune
|
||||
- thinking
|
||||
- reasoning
|
||||
- unsloth
|
||||
---
|
||||
## About
|
||||
|
||||
<!-- ### quantize_version: 2 -->
|
||||
<!-- ### output_tensor_quantised: 1 -->
|
||||
<!-- ### convert_type: hf -->
|
||||
<!-- ### vocab_type: -->
|
||||
<!-- ### tags: nicoboss -->
|
||||
<!-- ### quants: -->
|
||||
<!-- ### quants_skip: -->
|
||||
<!-- ### skip_mmproj: -->
|
||||
weighted/imatrix quants of https://huggingface.co/DavidAU/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B
|
||||
|
||||
<!-- provided-files -->
|
||||
|
||||
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF).***
|
||||
|
||||
static quants are available at https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-GGUF
|
||||
## Usage
|
||||
|
||||
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||
more details, including on how to concatenate multi-part files.
|
||||
|
||||
## Provided Quants
|
||||
|
||||
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||
|
||||
| Link | Type | Size/GB | Notes |
|
||||
|:-----|:-----|--------:|:------|
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ1_S.gguf) | i1-IQ1_S | 1.7 | for the desperate |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ1_M.gguf) | i1-IQ1_M | 1.8 | mostly desperate |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.0 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.2 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_S.gguf) | i1-IQ2_S | 2.3 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ2_M.gguf) | i1-IQ2_M | 2.4 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 2.4 | very low quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q2_K.gguf) | i1-Q2_K | 2.6 | IQ3_XXS probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 2.7 | lower quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 2.9 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.0 | IQ3_XS probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_S.gguf) | i1-IQ3_S | 3.0 | beats Q3_K* |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ3_M.gguf) | i1-IQ3_M | 3.1 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 3.3 | IQ3_S probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 3.5 | IQ3_M probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 3.6 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_0.gguf) | i1-Q4_0 | 3.8 | fast, low quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 3.8 | prefer IQ4_XS |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 3.8 | optimal size/speed/quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 4.0 | fast, recommended |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q4_1.gguf) | i1-Q4_1 | 4.1 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 4.5 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 4.6 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B-i1-GGUF/resolve/main/Qwen3-SG1-10S-III-INSTRUCT-Float32-256k-ctx-6B.i1-Q6_K.gguf) | i1-Q6_K | 5.3 | practically like static Q6_K |
|
||||
|
||||
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||
types (lower is better):
|
||||
|
||||
|
||||
|
||||
And here are Artefact2's thoughts on the matter:
|
||||
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||
|
||||
## FAQ / Model Request
|
||||
|
||||
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||
questions you might have and/or if you want some other model quantized.
|
||||
|
||||
## Thanks
|
||||
|
||||
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||
me use its servers and providing upgrades to my workstation to enable
|
||||
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||
|
||||
<!-- end -->
|
||||
Reference in New Issue
Block a user