初始化项目，由ModelHub XC社区提供模型
Model: mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF Source: Original Platform
This commit is contained in:
ModelHub XC
59
.gitattributes
vendored
Normal file
59
.gitattributes
vendored
Normal file
@@ -0,0 +1,59 @@
|
||||
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||
*.model filter=lfs diff=lfs merge=lfs -text
|
||||
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_M.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6ce5952ab777809a4d4517593bdb0afe3adcb16f666a12e17b81e96f40a42f8f
|
||||
size 5940520864
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0b89f16c8a4335178ef1f8ec07baa308ce366e1ad85cc0d0979c28b489c60b13
|
||||
size 5418330016
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_M.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:dbc76bf1ea2b0a319d8923f0057ce36c4ee91c6402b3c0445a038b2a3bacffeb
|
||||
size 8444397472
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b622b75f837f26ae1415103cf3b1bea21a53f8279411de5c1114eb99141a9134
|
||||
size 7748143008
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_XS.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f3cd6d2e6cd871f0de8b2be607c72635080a0a4910c3e18d8addf24232217e63
|
||||
size 7523870624
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e15202475eba1be2ef2f83488938cc3a4c3bf4e590aeb27244c72f199db71a66
|
||||
size 6810838944
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_M.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:d51db8604e2a013a3173196b09e2d86b774dd1e8815791b22f3405323129ef44
|
||||
size 11147143072
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:0ce410181067cd9bcafff4f9f3a565e40022fce5ecee82eb966b49013cdbcded
|
||||
size 10951059360
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_XS.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:4ad3fcb1c9778e2b46702bf0647d93a35fff94f2ebbd7195b8017ced20ded31e
|
||||
size 10391119776
|
||||
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:a4df47f56a0ed9fb929822761913d2397040a2df8264ba40ccbfe0ebe9e8ac5f
|
||||
size 9750923168
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ4_XS.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:62d5d785dda58ef33220f6ae334f77d665f49bc80bfde5f0464c2aedf20a9a18
|
||||
size 13433964448
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:e0d00370286cbf767cfba53aa4861a7fa887ff437085afec574dd0b9b414394e
|
||||
size 9302820768
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:28c4329e4ee3eaafe5aefcbbfccb818326b6bc24a053127b73d4acaea8962407
|
||||
size 8705132448
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_L.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:b8db5e19b2be81d3f1c126ce4307596fb3ad3bbf51b2f25d11523920d4b3b377
|
||||
size 13044017056
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_M.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c750ce3e5c82598c950325a1a59906791daec606725b0b819628efc080814056
|
||||
size 12080375712
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:1afea483f837f8f6b5693b5d913bb4319da4bb7ae2794d734fba1b48fb07f9d0
|
||||
size 10933233568
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_0.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:f58f75b27ed2b7f9a4420e7a0cc6bdc8ef8d843569c1d1793c8c5426192ac3df
|
||||
size 14234716064
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_1.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:c0c38eb8db9e7ea3cde21107f3096ca580144244c4498aff752618a41100480b
|
||||
size 15702001568
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_M.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5778a9601183cde0d25ac4b565a5dec6c4969f01b38575535899f9071e3f03b1
|
||||
size 15162181536
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:23b5bca216dd0e4b4cfb31706a6a9488013cc6a2d31f922709e0d4b7307f456d
|
||||
size 14295533472
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_M.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:5c0cb115e4137c4df9f45b509136bf4a820a23370b28daa278e1d5cd61f02e0a
|
||||
size 17736042400
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_S.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:63d3944df5669275d81377d0e019b72aa955358cd072d0b0b56077d070ca7233
|
||||
size 17228007328
|
||||
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q6_K.gguf
Normal file
3
L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:6f8ed3c66ae0cce08454776eb8fa084cd704f057060b3340e9b67d14aa88a933
|
||||
size 20470769568
|
||||
85
README.md
Normal file
85
README.md
Normal file
@@ -0,0 +1,85 @@
|
||||
---
|
||||
base_model: DavidAU/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32
|
||||
language:
|
||||
- en
|
||||
library_name: transformers
|
||||
quantized_by: mradermacher
|
||||
tags:
|
||||
- mergekit
|
||||
- mixture of experts
|
||||
- moe
|
||||
- 4x8B
|
||||
- float32
|
||||
- llama-3
|
||||
- llama3
|
||||
- 32 bit enhanced
|
||||
- LLama MOE
|
||||
- merge
|
||||
---
|
||||
## About
|
||||
|
||||
<!-- ### quantize_version: 2 -->
|
||||
<!-- ### output_tensor_quantised: 1 -->
|
||||
<!-- ### convert_type: hf -->
|
||||
<!-- ### vocab_type: -->
|
||||
<!-- ### tags: nicoboss -->
|
||||
weighted/imatrix quants of https://huggingface.co/DavidAU/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32
|
||||
|
||||
<!-- provided-files -->
|
||||
static quants are available at https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-GGUF
|
||||
## Usage
|
||||
|
||||
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||
more details, including on how to concatenate multi-part files.
|
||||
|
||||
## Provided Quants
|
||||
|
||||
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||
|
||||
| Link | Type | Size/GB | Notes |
|
||||
|:-----|:-----|--------:|:------|
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_S.gguf) | i1-IQ1_S | 5.5 | for the desperate |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ1_M.gguf) | i1-IQ1_M | 6.0 | mostly desperate |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 6.9 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_XS.gguf) | i1-IQ2_XS | 7.6 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_S.gguf) | i1-IQ2_S | 7.8 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ2_M.gguf) | i1-IQ2_M | 8.5 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K_S.gguf) | i1-Q2_K_S | 8.8 | very low quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q2_K.gguf) | i1-Q2_K | 9.4 | IQ3_XXS probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 9.9 | lower quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_XS.gguf) | i1-IQ3_XS | 10.5 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_S.gguf) | i1-Q3_K_S | 11.0 | IQ3_XS probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_S.gguf) | i1-IQ3_S | 11.1 | beats Q3_K* |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ3_M.gguf) | i1-IQ3_M | 11.2 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_M.gguf) | i1-Q3_K_M | 12.2 | IQ3_S probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q3_K_L.gguf) | i1-Q3_K_L | 13.1 | IQ3_M probably better |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-IQ4_XS.gguf) | i1-IQ4_XS | 13.5 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_0.gguf) | i1-Q4_0 | 14.3 | fast, low quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_S.gguf) | i1-Q4_K_S | 14.4 | optimal size/speed/quality |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_K_M.gguf) | i1-Q4_K_M | 15.3 | fast, recommended |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q4_1.gguf) | i1-Q4_1 | 15.8 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_S.gguf) | i1-Q5_K_S | 17.3 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q5_K_M.gguf) | i1-Q5_K_M | 17.8 | |
|
||||
| [GGUF](https://huggingface.co/mradermacher/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32-i1-GGUF/resolve/main/L3-Grand-Story-Darkness-MOE-4X8-24.9B-e32.i1-Q6_K.gguf) | i1-Q6_K | 20.6 | practically like static Q6_K |
|
||||
|
||||
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||
types (lower is better):
|
||||
|
||||
|
||||
|
||||
And here are Artefact2's thoughts on the matter:
|
||||
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||
|
||||
## FAQ / Model Request
|
||||
|
||||
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||
questions you might have and/or if you want some other model quantized.
|
||||
|
||||
## Thanks
|
||||
|
||||
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||
me use its servers and providing upgrades to my workstation to enable
|
||||
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||
|
||||
<!-- end -->
|
||||
3
imatrix.dat
Normal file
3
imatrix.dat
Normal file
@@ -0,0 +1,3 @@
|
||||
version https://git-lfs.github.com/spec/v1
|
||||
oid sha256:ea9f5f4ec4c424f895e4b47210aad636588d102ca7cded8a416a02a1f1a06d0c
|
||||
size 14164883
|
||||
Reference in New Issue
Block a user