eaddario/Dolphin-Mistral-24B-Venice-Edition-pruned-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Ed Addario
2025-07-01 09:07:31 +01:00
parent fde40b8129
commit c2d8908143
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -132,7 +132,7 @@ The process to generate these models is roughly as follows:
| [Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S.gguf) | 18.199918 ±0.142513 | 75.20% | 1.160040 ±0.004229 | 36.220 ±0.088 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S.gguf) | 18.199918 ±0.142513 | 75.20% | 1.160040 ±0.004229 | 36.220 ±0.088 |
| [Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K.gguf) | 18.213825 ±0.142965 | 75.05% | 1.158026 ±0.004262 | 36.219 ±0.088 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K.gguf) | 18.213825 ±0.142965 | 75.05% | 1.158026 ±0.004262 | 36.219 ±0.088 |
| [Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0.gguf) | 18.203515 ±0.142826 | 75.02% | 1.158351 ±0.004265 | 36.227 ±0.088 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0.gguf) | 18.203515 ±0.142826 | 75.02% | 1.158351 ±0.004265 | 36.227 ±0.088 |
| [Dolphin-Mistral-24B-Venice-Edition-pruned-F16](./Dolphin-Mistral-24B-Venice-Edition-pruned-F16.gguf) | 6.180577 ±0.041038 | 100% | N/A | N/A | | [Dolphin-Mistral-24B-Venice-Edition-pruned-F16](./Dolphin-Mistral-24B-Venice-Edition-F16.gguf) | 6.180577 ±0.041038 | 100% | N/A | N/A |
### ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores ### ARC, HellaSwag, MMLU, Truthful QA and WinoGrande scores
Scores generated using [llama-perplexity][ppl] with 750 tasks per test, and a context size of 768 tokens. Scores generated using [llama-perplexity][ppl] with 750 tasks per test, and a context size of 768 tokens.
@@ -154,7 +154,7 @@ For the test data used in the generation of these scores, follow the appropriate
| [Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S.gguf) | 67.3333 ±1.7137 | 81.47 | 44.2667 ±1.8149 | 38.6667 ±1.7794 | 74.2667 ±1.5974 | 61.20 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q5_K_S.gguf) | 67.3333 ±1.7137 | 81.47 | 44.2667 ±1.8149 | 38.6667 ±1.7794 | 74.2667 ±1.5974 | 61.20 |
| [Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K.gguf) | 67.4667 ±1.7119 | 81.07 | 44.5333 ±1.8160 | 39.6000 ±1.7870 | 73.8667 ±1.6054 | 61.31 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q6_K.gguf) | 67.4667 ±1.7119 | 81.07 | 44.5333 ±1.8160 | 39.6000 ±1.7870 | 73.8667 ±1.6054 | 61.31 |
| [Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0.gguf) | 68.1333 ±1.7026 | 81.33 | 44.9333 ±1.8176 | 38.2667 ±1.7759 | 74.4000 ±1.5947 | 61.41 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0](./Dolphin-Mistral-24B-Venice-Edition-pruned-Q8_0.gguf) | 68.1333 ±1.7026 | 81.33 | 44.9333 ±1.8176 | 38.2667 ±1.7759 | 74.4000 ±1.5947 | 61.41 |
| [Dolphin-Mistral-24B-Venice-Edition-pruned-F16](./Dolphin-Mistral-24B-Venice-Edition-pruned-F16.gguf) | 70.8000 ±1.6614 | 84.53 | 45.3333 ±1.8190 | 38.1333 ±1.7748 | 80.2667 ±1.4542 | 63.81 | | [Dolphin-Mistral-24B-Venice-Edition-pruned-F16](./Dolphin-Mistral-24B-Venice-Edition-F16.gguf) | 70.8000 ±1.6614 | 84.53 | 45.3333 ±1.8190 | 38.1333 ±1.7748 | 80.2667 ±1.4542 | 63.81 |
### Tokens per Second - Benchmarks ### Tokens per Second - Benchmarks
Scores generated using [llama-bench][bch]. Naive (`llama-quantize` with no optimization) Q4_K_M quantization included for comparison. Scores generated using [llama-bench][bch]. Naive (`llama-quantize` with no optimization) Q4_K_M quantization included for comparison.