初始化项目,由ModelHub XC社区提供模型
Model: mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF Source: Original Platform
This commit is contained in:
60
.gitattributes
vendored
Normal file
60
.gitattributes
vendored
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0_4_4.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0_4_8.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0_8_8.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_M.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:223fd9f4f15def6f90074d03e27622f77e3e976623e9bf6e45b500fd711946a2
|
||||||
|
size 2161998144
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_S.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:49120bf6ff6834f291221e8daf75f10db841ff28ec28d076cd90d2a49dd987b5
|
||||||
|
size 2019653952
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_M.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7a0fb5bf797e2af7c080918173f3c7da4336edacee6acc0e9aacc3ecfc981173
|
||||||
|
size 2948309760
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_S.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:6b010dc7ba76eda2b7e6017854025425a7e3b6f5c8adbdf0ff72fb630fb9030a
|
||||||
|
size 2758517504
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XS.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5b686b646b14e66d17bd12095c82b7413f43589c11fdb8e6b5eb4bbd0129c724
|
||||||
|
size 2605807936
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XXS.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f1a901a3da7ba3b6ed7935f6ee0f7e1a789abf7cbc68c1d9362426257db7c4e5
|
||||||
|
size 2399238464
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_M.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1cdde8fd646eedd7e455d82dc5b3cca0c5a34b6d3d459d745c5cef3fa0704237
|
||||||
|
size 3784855488
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_S.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:cbdc52339c7244b825af22eb48740432a469fb7472024200568f046dd7bfb5c5
|
||||||
|
size 3682357184
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XS.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:2111953eb97be7bce908553fe498ae049fe07a8ef8b54f1b7241c11ccf1d4dc2
|
||||||
|
size 3518779328
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XXS.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7451602e5ae713240745d8c1f82c64af7cde8c14aa8c1b5d043c06f6f273618e
|
||||||
|
size 3274941184
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ4_XS.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:cd0f8b3a6d7ab149d32564bf2995aae3ca0227021f7f8605cdf275f3b47e40d1
|
||||||
|
size 4447697280
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q2_K.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7a60baf7a6cfbfb0c7a1e56338137a37b59129701c9096dab988b93808c37aed
|
||||||
|
size 3179161088
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_L.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:06b30174b500af1f43f8f2aae6388d61cca8db6764569be8fadf890b42750a2c
|
||||||
|
size 4321988544
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_M.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7775a7abfebd0bfbc14d936c4f111aa39e46dc92a6b7989d8f927664e9e256c1
|
||||||
|
size 4018950080
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_S.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f20735ddcc9dcadfce3ad7974fefe49593449cd7dfe96541bab3f744bc8f450d
|
||||||
|
size 3664531392
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:e1adbf4f00e901a330b4c3717b76b963f20d199ff0d99189386173154aa35d7a
|
||||||
|
size 4675927168
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:449684a7e062809d8dc7a95b8ec32e180a08a136180ef93b580705eb9d93120c
|
||||||
|
size 4661247104
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:1274e44aaeac961683491b473ab81ec5f4e64009e92de54dd412c546b02ac60b
|
||||||
|
size 4661247104
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:a7da176a8105a5a6d175d376f87bd5a709932470965342899cca4db1cf669197
|
||||||
|
size 4661247104
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_M.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0c4531fe553d00142808e1bc7348ae92d400794c5b64d2db1a974718324dfe9a
|
||||||
|
size 4920769664
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_S.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:acb8a14d4968ebb2936b41ec803775c29ddfa39d79f4f6984879e6bf83c48416
|
||||||
|
size 4692704384
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_M.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:d84e0d1ac7ffcc6ab29dd76c8fa31ea5a5477a0d6c3d1b4248d3ee38d5267143
|
||||||
|
size 5733025920
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_S.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f34ef7e59f68123e8880c5b4c1b76f0adefb21a639e5efa46b408971ae6bcf1b
|
||||||
|
size 5599332480
|
||||||
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q6_K.gguf
Normal file
3
Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b98d24b76b5c72341e19c9170c3dec4efbe80ed71b449c7656b64b2ad1cf378f
|
||||||
|
size 6596048192
|
||||||
82
README.md
Normal file
82
README.md
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
---
|
||||||
|
base_model: SE6446/Llama-3.1-SuperNova-Lite-Reflection-V1.0
|
||||||
|
datasets:
|
||||||
|
- thesven/Reflective-MAGLLAMA-v0.1.1
|
||||||
|
- SE6446/MAGllama_Sharegpt
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
license: llama3.1
|
||||||
|
quantized_by: mradermacher
|
||||||
|
tags:
|
||||||
|
- axolotl
|
||||||
|
- generated_from_trainer
|
||||||
|
---
|
||||||
|
## About
|
||||||
|
|
||||||
|
<!-- ### quantize_version: 2 -->
|
||||||
|
<!-- ### output_tensor_quantised: 1 -->
|
||||||
|
<!-- ### convert_type: hf -->
|
||||||
|
<!-- ### vocab_type: -->
|
||||||
|
<!-- ### tags: nicoboss -->
|
||||||
|
weighted/imatrix quants of https://huggingface.co/SE6446/Llama-3.1-SuperNova-Lite-Reflection-V1.0
|
||||||
|
|
||||||
|
<!-- provided-files -->
|
||||||
|
static quants are available at https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-GGUF
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||||
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||||
|
more details, including on how to concatenate multi-part files.
|
||||||
|
|
||||||
|
## Provided Quants
|
||||||
|
|
||||||
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||||
|
|
||||||
|
| Link | Type | Size/GB | Notes |
|
||||||
|
|:-----|:-----|--------:|:------|
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_S.gguf) | i1-IQ1_S | 2.1 | for the desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ1_M.gguf) | i1-IQ1_M | 2.3 | mostly desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 2.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_XS.gguf) | i1-IQ2_XS | 2.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_S.gguf) | i1-IQ2_S | 2.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ2_M.gguf) | i1-IQ2_M | 3.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q2_K.gguf) | i1-Q2_K | 3.3 | IQ3_XXS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 3.4 | lower quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_XS.gguf) | i1-IQ3_XS | 3.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_S.gguf) | i1-Q3_K_S | 3.8 | IQ3_XS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_S.gguf) | i1-IQ3_S | 3.8 | beats Q3_K* |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ3_M.gguf) | i1-IQ3_M | 3.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_M.gguf) | i1-Q3_K_M | 4.1 | IQ3_S probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q3_K_L.gguf) | i1-Q3_K_L | 4.4 | IQ3_M probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-IQ4_XS.gguf) | i1-IQ4_XS | 4.5 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0_4_4.gguf) | i1-Q4_0_4_4 | 4.8 | fast on arm, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0_4_8.gguf) | i1-Q4_0_4_8 | 4.8 | fast on arm+i8mm, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0_8_8.gguf) | i1-Q4_0_8_8 | 4.8 | fast on arm+sve, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_0.gguf) | i1-Q4_0 | 4.8 | fast, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_S.gguf) | i1-Q4_K_S | 4.8 | optimal size/speed/quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q4_K_M.gguf) | i1-Q4_K_M | 5.0 | fast, recommended |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_S.gguf) | i1-Q5_K_S | 5.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q5_K_M.gguf) | i1-Q5_K_M | 5.8 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/Llama-3.1-SuperNova-Lite-Reflection-V1.0-i1-GGUF/resolve/main/Llama-3.1-SuperNova-Lite-Reflection-V1.0.i1-Q6_K.gguf) | i1-Q6_K | 6.7 | practically like static Q6_K |
|
||||||
|
|
||||||
|
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||||
|
types (lower is better):
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
And here are Artefact2's thoughts on the matter:
|
||||||
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||||
|
|
||||||
|
## FAQ / Model Request
|
||||||
|
|
||||||
|
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||||
|
questions you might have and/or if you want some other model quantized.
|
||||||
|
|
||||||
|
## Thanks
|
||||||
|
|
||||||
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||||
|
me use its servers and providing upgrades to my workstation to enable
|
||||||
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||||
|
|
||||||
|
<!-- end -->
|
||||||
3
imatrix.dat
Normal file
3
imatrix.dat
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:dc49aa469e18b670d7848f714bbbcfbf1cd7954c456f634520b682712188ba54
|
||||||
|
size 4988157
|
||||||
Reference in New Issue
Block a user