mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-19 11:46:39 +08:00
commit 017ffada7c
24 changed files with 200 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
.gitattributes vendored Normal file
View File

@@ -0,0 +1,57 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3-8B-Ultra-Instruct.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text

3
Llama-3-8B-Ultra-Instruct.i1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5d0865d7222927a42ddf1861d345d73dee8c2451e2b884f1a4b402540337a29a
size 2162485152

3
Llama-3-8B-Ultra-Instruct.i1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ab67752354781efea6438fa0a55ba1302509e96b7f0d14f5113d5c55a3e7471
size 2020140960

3
Llama-3-8B-Ultra-Instruct.i1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:82c392952c13befb4036d90908e140b7ac35db17378425bf5cf7a357f7c03f7f
size 2948794272

3
Llama-3-8B-Ultra-Instruct.i1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb70340c2fdb07bcd6c3905632ba27d037d8afc2dc51195ce23423832e597eb3
size 2759002016

3
Llama-3-8B-Ultra-Instruct.i1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2cdae18358c38d598d5bd28f48b68c4f6b39336e73e97aaf17893370c0b4b188
size 2606294944

3
Llama-3-8B-Ultra-Instruct.i1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:262266f1fabe97f3e697f6c726cefd7f162a70b7a4a5d3c8acbee5c3d3223a2f
size 2399725472

3
Llama-3-8B-Ultra-Instruct.i1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:46560338da8210dcbe796c9e7d43f49fd4c8416989e366b23a49ca24911c21c7
size 3785336736

3
Llama-3-8B-Ultra-Instruct.i1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d008640c4b1f850b07e1a0d1ae5c0fd529030887f41f02118294c4cb4a46168f
size 3682838432

3
Llama-3-8B-Ultra-Instruct.i1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4d3a5fc2ce45c62a99917bd4dd39cb37713ca518b0d862911273e05ebb0c0de8
size 3519260576

3
Llama-3-8B-Ultra-Instruct.i1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c75795e6c1f4525fe2fcc88341ca53f6cc59d1353dcf132eb15f0d07117358a3
size 3275425696

3
Llama-3-8B-Ultra-Instruct.i1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3199224d467ec29b3f66db4ac944318c921c9f7d7eda6a1b33325793a4d88792
size 4448176032

3
Llama-3-8B-Ultra-Instruct.i1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fb6aaddd88130df5ea83dfbbd691568545eb02b57be124aca9a5b007d560e1f9
size 3179644832

3
Llama-3-8B-Ultra-Instruct.i1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2ca18c5e631a6c84ffac9f03de7ffee1f2f91efa23e5d1e8471b6b339aee45cd
size 4322469792

3
Llama-3-8B-Ultra-Instruct.i1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5c4164e175f5531d400afbabf0bb8a9a4e9d5535c019eb75ee0465050433b539
size 4019431328

3
Llama-3-8B-Ultra-Instruct.i1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3099a9a6080baa85de9f8f12bbf52fa3756d8b92b15a7ce33c4d7758824873ae
size 3665012640

3
Llama-3-8B-Ultra-Instruct.i1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:73a104d92e294b50b4d07c063612545f4113d89c1071b9223fe8e3f288b1a613
size 4676405152

3
Llama-3-8B-Ultra-Instruct.i1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9fd87323bb8b71b6c2a6e54b462718ec43503b48b0a502867dd4c034b83f3eb1
size 4921247648

3
Llama-3-8B-Ultra-Instruct.i1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ae0776162676bec52ec6abf0524c3bdcc2988cf362672ef01d649ee2894cbc88
size 4693182368

3
Llama-3-8B-Ultra-Instruct.i1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6627486038a357e6c3188ce1043134179e36d01b74fb405dbdcde72efa5911e0
size 5733500832

3
Llama-3-8B-Ultra-Instruct.i1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b1a9d452853e3ed4d9f92bb8c6476dd058ecc23a45715d3e34dcfce61bade0bb
size 5599807392

3
Llama-3-8B-Ultra-Instruct.i1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22a55e6bee24e31e4569f5ae72a9295f014e3de8be5b48b816347f980663c6e1
size 6596519840

77
README.md Normal file
View File

@@ -0,0 +1,77 @@
---
base_model: Dampfinchen/Llama-3-8B-Ultra-Instruct
language:
- en
library_name: transformers
license: llama3
quantized_by: mradermacher
tags:
- mergekit
- merge
---
## About
<!-- ### quantize_version: 2 -->
<!-- ### output_tensor_quantised: 1 -->
<!-- ### convert_type: -->
<!-- ### vocab_type: -->
weighted/imatrix quants of https://huggingface.co/Dampfinchen/Llama-3-8B-Ultra-Instruct
You should use `--override-kv tokenizer.ggml.pre=str:llama3` and a current llama.cpp version to work around a bug in llama.cpp that made these quants. (see https://old.reddit.com/r/LocalLLaMA/comments/1cg0z1i/bpe_pretokenization_support_is_now_merged_llamacpp/?share_id=5dBFB9x0cOJi8vbr-Murh)
<!-- provided-files -->
static quants are available at https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-GGUF
## Usage
If you are unsure how to use GGUF files, refer to one of [TheBloke's
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
more details, including on how to concatenate multi-part files.
## Provided Quants
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
| Link | Type | Size/GB | Notes |
|:-----|:-----|--------:|:------|
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
| [GGUF](https://huggingface.co/mradermacher/Llama-3-8B-Ultra-Instruct-i1-GGUF/resolve/main/Llama-3-8B-Ultra-Instruct.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
And here are Artefact2's thoughts on the matter:
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
## FAQ / Model Request
See https://huggingface.co/mradermacher/model_requests for some answers to
questions you might have and/or if you want some other model quantized.
## Thanks
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
me use its servers and providing upgrades to my workstation to enable
this work in my free time.
<!-- end -->

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bfe0fd0df214dd15efaa55e6a329483ce741c42a593f7e90887d5c7745d17899
size 4988159