mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-12 18:19:24 +08:00
commit 5602f7dc72
27 changed files with 211 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6f316b4fae8234b5eb30fad029a8132c6e3ab5afa7c4b6c5fd9ea659cf837aaf
size 2165189632

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:67089299370d3853e9f9e72a41365ca8cf9f236358b1cba994866fb28b03c337
size 2022845440

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22556f49a4700569c29d677ac9917fc45b3c50c80f54cc20a3ff02dc49ef6e84
size 2951818240

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7a9b82b3f1dfda8ad5e04d258f9e4f6fd2073851bbde2cb25e3c5d012cfe5514
size 2762025984

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13cec00ee0eed29015e05aceec37563632d5025cd290a2a122e02257dcdc59bc
size 2608999424

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:166d810d30017cc384316c63a0a65ecf30c11a3a98dbd40cf8ec54ad3acfbdf3
size 2402429952

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2a88f3f26aef6da69649b007c088efab76fdfbaa93de434d4a47a87591881483
size 3788778496

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:64c3dc4e2ac68f0b9bc642a42f427e411e3ee0878578f1cc534befc1db737e16
size 3686280192

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:af4984da0affd66aac95ce961e3616409d79a56104775a61086e8ce115cfa131
size 3522702336

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:176a058f2249a6ede82ed06bac452b80fa11e17856f4c2702d7e1946732e8b34
size 3278449664

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8e167864a384f5e81836c12c17b8477242020a04d719ac160215f2d39b3fde9f
size 4682361856

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:47ac5d680e7df135a5efc3a3a54870614d1863f910024f4cf809138014c5b174
size 4451937280

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a9a352940f6338cdd35d688367491bdd0a7a57a2417d639cedf1ad9c6052703f
size 3182767104

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b8b04322743ae37a6ccd7bd7b2f4448877400a2bf0a24864a48569017519aaaf
size 2992450560

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2ce34e22e8f3279096c774f16972a47aaf2a28a5c223d5d94929c81fb3204aa1
size 4325911552

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c24d944a5cd5b944278218748bc4e7054f8da36da6649c1f3efb9188d7e1be2c
size 4022873088

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3c8c7aa640b4cda4d706661258f0c04f7184c19e939d926671af75e804713725
size 3668454400

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:86cf8457714fe23d953ba3ba0c695ce3d88430b30ea178a9f90f880da2b6f63a
size 4680264704

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:92e70a9b7c6a31565582a349a44861880d3fb70d090b50fb266cc4d5cefd8b3b
size 5134822400

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6738cf4023449c45c499295ed1de320fa881d3d041fa0ab5b57595cf73a1545f
size 4925107200

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:792f0f232c8a74c1485ce6861952f369256d156f64906ad721aa876444dd75ca
size 4697041920

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9c63899cc0852b4eae02b2fd5d03eaaa8015f2ed956c9097e93e591dfbd04296
size 5737753600

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:665de4d7204e2595cf67eb57ea20fa917b3c90fc1bef87fb54fa85fb177807be
size 5604060160

3
Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e40b58059a50e27de9d2bdf41b83fb328965788f3086c1936deab0450bf23bdb
size 6601190400

76
README.md Normal file
View File

@@ -0,0 +1,76 @@
---
base_model: nvidia/Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct
language:
- en
library_name: transformers
license: cc-by-nc-4.0
quantized_by: mradermacher
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: hf -->
<!-- ### vocab_type: -->
<!-- ### tags: nicoboss -->
weighted/imatrix quants of https://huggingface.co/nvidia/Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ2_M.gguf) | i1-IQ2_M | 3.1 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q2_K_S.gguf) | i1-Q2_K_S | 3.1 | very low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.6 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-IQ4_NL.gguf) | i1-IQ4_NL | 4.8 | prefer IQ4_XS |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q4_1.gguf) | i1-Q4_1 | 5.2 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-8B-UltraLong-1M-Instruct-i1-GGUF/resolve/main/Llama-3.1-8B-UltraLong-1M-Instruct.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dfead29dc6a754d6b8e888c3a36e084ab778bc775b877ae98e99ff7d2f56dc13
size 4988157