mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-11 22:06:35 +08:00
commit 92b6bf0da6
27 changed files with 221 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b739b8491c7c4ae9685ee7c56c38de9c3bc692edd874c55afa98024b1b6b0540
size 2161973184

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a792cb3d3e4a9f01d57ffb77239a5bc9509da8b8bd688d713453cce6b1bf48e7
size 2019628992

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:12ba4b5412da5f4a6d9761229797a39060398c659d68dd6c81f8d309ab193e39
size 2948282304

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c8f8d9a7a4c68b7af89a6f4a3ad0682313143c71c5dd749c78e9b870e95c7f42
size 2758490048

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9b4825a40b7a63c2ad5b0e572bab4292516558fd0d769b05835a78a8ba7d47bf
size 2605782976

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:24e42dba12b0a6110feadb39084cd2367f2dae38ddaadc4bb20218c2bd89cf49
size 2399213504

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:52d7edaee01cc09fc06afd2b810d9e7740cafc8f4c4d6787bc54cf937c854b77
size 3784824768

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:41c455c3ecfe9e345bb54e1305463fc42542a1a5acc431279f9798f9c79630b3
size 3682326464

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:128ab5a823a047ed87bc4348081d8d17b4098411878ebf8340588c1a02bb19cf
size 3518748608

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9d63c12ae0d29cc8b01a42508474926ccd2332aeeb507e32745f5a7e60c6593d
size 3274913728

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dbfd9fa20ae7fe88b9b3798e818b3cde3c7511965572f118a7a47733f2545852
size 4447664064

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:665b27957d70efad699dd55f00285e0e26414d0823dea819ce6b2eaeb6cf7f8e
size 3179132864

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b15aa57b597c454b5e7eeda89f16eac82bec4b331ad4ea4fa545d03f97968191
size 4321957824

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:41856398853c13c5a712afc5c9a12e580463331f58472c6f05414e2140011240
size 4018919360

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:83889445bf17fe33ca4a0e53f8cae08425b36825402915a64123d63718dfa349
size 3664500672

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:61e11845f494369d41eb8437a20bed14d2d787707ad37a49d41ccfb577d1cb33
size 4675893184

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_4_4.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3666967e1abc979c5b651e25ff7829a79f8d1546534d71da7ce1e838ad83f958
size 4661213120

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_4_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b5553fed0bd921f053360236c80f270133c4fe2f141c4aecb6c92efbf1158634
size 4661213120

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_8_8.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:43d6b9da50adf8bf22c7673e3180026dfcf658632def461837efbf1730130074
size 4661213120

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:44aa7f50ddbb6350baabff84a761f3fc27860a017eca703eb7aa08e8311120e9
size 4920735680

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:95a62344636b72ea881dc3b384d23282eac6b96146b0732fccdb2ee8f5841db7
size 4692670400

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b68ca921c284e572f042b8d9cf9a6b8a28857c1a0e2c1535e9d9c13016873df8
size 5732988864

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:49cc8b3896600403f87fc5046284805b665916667bb65f37b950c971a821938a
size 5599295424

3
Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0ce2db12baad0261727d274feddb3fac64e6fb2cc65207637f6e6da15dba0595
size 6596007872

86
README.md Normal file
View File

@@ -0,0 +1,86 @@
---
base_model: tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.2
datasets:
- lmsys/lmsys-chat-1m
- tokyotech-llm/lmsys-chat-1m-synth
- argilla/magpie-ultra-v0.1
- tokyotech-llm/swallow-magpie-ultra-v0.1
- tokyotech-llm/swallow-gemma-magpie-v0.1
language:
- en
- ja
library_name: transformers
license:
- llama3.1
- gemma
model_type: llama
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.2
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 4.8 | fast on arm, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 4.8 | fast on arm+i8mm, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 4.8 | fast on arm+sve, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-Swallow-8B-Instruct-v0.2-i1-GGUF/resolve/main/Llama-3.1-Swallow-8B-Instruct-v0.2.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:77249c492ee8954e0812be99ce5efc0fed952c717840458264b60a5547b57c66
size 4988157