Lewdiculous/Erosumika-7B-v2-GGUF-IQ-Imatrix
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: Lewdiculous/Erosumika-7B-v2-GGUF-IQ-Imatrix
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-16 11:18:47 +08:00
commit 7cee579aad
15 changed files with 2514 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
.gitattributes vendored Normal file
View File

@@ -0,0 +1,47 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-F16.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-IQ3_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-IQ3_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-IQ3_XXS-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-IQ4_XS-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-Q4_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-Q4_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-Q5_K_M-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-Q5_K_S-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-Q6_K-imat.gguf filter=lfs diff=lfs merge=lfs -text
Erosumika-7B-v2-Q8_0-imat.gguf filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text

3
Erosumika-7B-v2-F16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:364f31375bbb6fc153e436b1fbb5cb702ccd7af7500b9ab15743bf7effe420bf
size 14484731616

3
Erosumika-7B-v2-IQ3_M-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ed7601a8da76c0e5bf00dd851f17e576fb05004f42dd4ab290c47d4d49d146b3
size 3284891392

3
Erosumika-7B-v2-IQ3_S-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:169078dc83c7a470ab763966744281ec80aa879de4c159ed921899878d5bf2f3
size 3182393088

3
Erosumika-7B-v2-IQ3_XXS-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:931baa1465af8d9a5ef4e6947078208bfe2a94d245e198582a824716f7ba453c
size 2827343616

3
Erosumika-7B-v2-IQ4_XS-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f0e80eb8f2a96cb553c4857e20379b01aa889d79f35b6c9fd3e6acee7c145b11
size 3907688192

3
Erosumika-7B-v2-Q4_K_M-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ff4af1c659353ad1db49554730b43d20b59a92fb86f0b0162743f182811c2446
size 4368439040

3
Erosumika-7B-v2-Q4_K_S-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50429ff0ab74a85707a0201e113d134a783dcba431d77be576a0b847fc39bcfa
size 4140373760

3
Erosumika-7B-v2-Q5_K_M-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2ae9d23e45013a661356d39710f64f6d08f4385eb020bea8b7c04f1e7e00ba1d
size 5131409152

3
Erosumika-7B-v2-Q5_K_S-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:340543c7b07c5ef1205a99b79bcf25ab3bad0d839132bd285145aac2047065b8
size 4997715712

3
Erosumika-7B-v2-Q6_K-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:38204892e4f2d197c58fbd5733eff9dfe83d26145641cc28c58bd724a6961d42
size 5942064896

3
Erosumika-7B-v2-Q8_0-imat.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3b15fb890018d138d293c2ef15e6e2fa00c26a47c39bc04060cf51a6bfd81fd6
size 7695857408

85
README.md Normal file
View File

@@ -0,0 +1,85 @@
---
language:
- en
pipeline_tag: text-generation
tags:
- text-generation-inference
- instruct
- conversational
- roleplay
- sillytavern
- gguf
- anime
- quantized
- mistral
license: cc-by-4.0
---
# **THIS VERSION IS NOW DEPRECATED. USE V3-0.2. V2 HAS PROBLEMS WITH ALIGNMENT AND THE NEW VERSION IS A SUBSTANTIAL IMPROVMENT!**
This repository hosts deprecated GGUF-IQ-Imatrix quants for [localfultonextractor/Erosumika-7B-v2](https://huggingface.co/localfultonextractor/Erosumika-7B-v2).
*"Better, smarter erosexika!!"*
[Quantized as per user request.](https://huggingface.co/Lewdiculous/Model-Requests/discussions/19)
Quants:
```python
quantization_options = [
"Q4_K_M", "Q4_K_S", "IQ4_XS", "Q5_K_M", "Q5_K_S",
"Q6_K", "Q8_0", "IQ3_M", "IQ3_S", "IQ3_XXS"
]
```
**What does "Imatrix" mean?**
It stands for **Importance Matrix**, a technique used to improve the quality of quantized models.
The **Imatrix** is calculated based on calibration data, and it helps determine the importance of different model activations during the quantization process.
The idea is to preserve the most important information during quantization, which can help reduce the loss of model performance, especially when the calibration data is diverse.
[[1]](https://github.com/ggerganov/llama.cpp/discussions/5006) [[2]](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
For imatrix data generation, kalomaze's `groups_merged.txt` with added roleplay chats was used, you can find it [here](https://huggingface.co/Lewdiculous/Datura_7B-GGUF-Imatrix/blob/main/imatrix-with-rp-format-data.txt). This was just to add a bit more diversity to the data.
**Steps:**
```
Base⇢ GGUF(F16)⇢ Imatrix-Data(F16)⇢ GGUF(Imatrix-Quants)
```
*Using the latest llama.cpp at the time.*
# Original model information:
<h1 style="text-align: center">Erosumika-7B-v2</h1>
![image/gif](https://cdn-uploads.huggingface.co/production/uploads/65d4cf2693a0a3744a27536c/jkrt-bDxaI9Z-V-9fBTbx.gif)
## Model Details
A DARE TIES merge between Nitral's [Kunocchini-7b](https://huggingface.co/Nitral-AI/Kunocchini-7b-128k-test), Epiculous' [Mika-7B](https://huggingface.co/Epiculous/Mika-7B) and my [FlatErosAlpha](https://huggingface.co/localfultonextractor/FlatErosAlpha), a flattened(in order to keep the vocab size 32000) version of tavtav's [eros-7B-ALPHA](https://huggingface.co/tavtav/eros-7B-ALPHA). In my brief testing, v2 is a significant improvement over the original Erosumika; I guess it won the DARE TIES lottery. Alpaca and Mistral seem to work best. Chat-ML might also work but I expect it to never end generations. Anything goes!
Due to it being an experimental model, there are some quirks...
- Rare occasion to misspell words
- Very rare occasion to have random formatting artifact at the end of generations
[GGUF quants](https://huggingface.co/localfultonextractor/Erosumika-7B-v2-GGUF)
## Limitations and biases
The intended use-case for this model is fictional writing for entertainment purposes. Any other sort of usage is out of scope.
It may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Outputs might often be factually wrong or misleading.
```yaml
base_model: localfultonextractor/FlatErosAlpha
models:
- model: localfultonextractor/FlatErosAlpha
- model: Epiculous/Mika-7B
parameters:
density: 0.5
weight: 0.25
- model: Nitral-AI/Kunocchini-7b
parameters:
density: 0.5
weight: 0.75
merge_method: dare_ties
dtype: bfloat16
```

2346
imatrix-with-rp-data.txt Normal file
View File

File diff suppressed because it is too large Load Diff

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd86ff91a511460dd671779b92f0e5bb818f21317fecc931cb7203f8aa289b1c
size 4988126