InferenceIllusionist/mathstral-7B-v0.1-iMat-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: InferenceIllusionist/mathstral-7B-v0.1-iMat-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-30 08:42:16 +08:00
commit 5781b814fe
23 changed files with 150 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
.gitattributes vendored Normal file
View File

@@ -0,0 +1,56 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-iMat-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
mathstral-7B-v0.1-f16.imatrix filter=lfs diff=lfs merge=lfs -text
ggml-model-f16.gguf filter=lfs diff=lfs merge=lfs -text

31
README.md Normal file
View File

@@ -0,0 +1,31 @@
---
base_model: mistralai/mathstral-7B-v0.1
library_name: transformers
quantized_by: InferenceIllusionist
tags:
- iMat
- gguf
- Mistral
- Math
license: apache-2.0
---
<img src="https://i.imgur.com/P68dXux.png" width="400"/>
# mathstral-7B-v0.1-iMat-GGUF
Quantized from fp16.
* Weighted quantizations were creating using fp16 GGUF and groups_merged.txt in 105 chunks and n_ctx=512
* Static fp16 also included in repo
For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
<i>All quants are verified working prior to uploading to repo for your safety and convenience</i>
<b>KL-Divergence Reference Chart</b>
(Click on image to view in full size)
[<img src="https://i.imgur.com/mV0nYdA.png" width="920"/>](https://i.imgur.com/mV0nYdA.png)
<b>Tips:</b> There's no need to download the entire repo, just pick one of the GGUF files. As with smaller 7b models, Q6 or larger is recommended for best results. On quants smaller than Q3, repetition penalty = 1.05 - 1.3 and min P = 0.05 mitigated some issues, but set your expectations accordingly
Original model card can be found [here](https://huggingface.co/mistralai/mathstral-7B-v0.1)

3
ggml-model-f16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50596e9b225cb88a83e1a0c513aea17920040a8d91a75e4465f3ad839c9151c1
size 14497336928

3
mathstral-7B-v0.1-f16.imatrix Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:773f179c3611de14c55eef060da320bffa4862e7d340a639b748692953a89add
size 4988161

3
mathstral-7B-v0.1-iMat-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8f21f63f7b9f73db1f9f6618b8d2e6d4eeaa9578688d32da29f728c3bcfe05e5
size 1757663072

3
mathstral-7B-v0.1-iMat-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:22442b9f3b3df5894710bb7cefc87100befd516dcfdd0afd132247aa77f4c0a4
size 2504249184

3
mathstral-7B-v0.1-iMat-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5287472cad16f3a1725525b40486ffa244de61460812b4b5249782a352961ed3
size 2314456928

3
mathstral-7B-v0.1-iMat-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6eed0cf578275b06d6d75e2859943e13372e14860a43f804ebb9a3eb8da38c18
size 2201472864

3
mathstral-7B-v0.1-iMat-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:97847b277161718ad9b041bb779a2ec72c83b2d90fcd0793041142458459077c
size 1994903392

3
mathstral-7B-v0.1-iMat-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b1177dfbe1c23df8713f7e9800433bc46db7b1928b42076cc2004e66d5605faa
size 3288846176

3
mathstral-7B-v0.1-iMat-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:eaa44182e27709da520898ac6c2f3ad5cd17f36fd81940cfbc0201209abf8097
size 3186347872

3
mathstral-7B-v0.1-iMat-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:efdd1a914d426707769d3d6a87482b0c1bc69e97d24785e5dd817776465b5c54
size 3022770016

3
mathstral-7B-v0.1-iMat-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4daccb50765e82ba0943afd24fb69cf27b6bccdc3bb989a8b687b55882582043
size 2830880608

3
mathstral-7B-v0.1-iMat-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:901b1d51c954fe8462bccfdb5637a4af35ed4ade91eeacab773637794c3dd26c
size 3911962464

3
mathstral-7B-v0.1-iMat-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7fb84486d29d1f935dd38c05e920bcbce57a3b3351e15b9f889c04ded582dcc9
size 2722877280

3
mathstral-7B-v0.1-iMat-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ecd983b792d94dc421bbb074a6a9ef6a874d43c0f25c463b76e0e980a88f762d
size 3522940768

3
mathstral-7B-v0.1-iMat-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:012b4f879a120b4da4b3999c70ad595675503db1cf4ebddf975e0947ca161816
size 3168522080

3
mathstral-7B-v0.1-iMat-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3ba94b7a8283ffa319c9ce23657f91ecf221ceada167c1253906cf56d72e8f90
size 4372811616

3
mathstral-7B-v0.1-iMat-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f4649a61163550f0c002b77d61a8ab9b3bd0b72f3f29ad6339392794d0496ba4
size 4144746336

3
mathstral-7B-v0.1-iMat-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0baa648a9394668169adc79d997520a9a367d93b11424fe54beb8410b79e6560
size 5136174944

3
mathstral-7B-v0.1-iMat-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:8f43cd07c3764bdf103d1a9856582c46bc98e835d496b62c2225d6858c18a506
size 5002481504

3
mathstral-7B-v0.1-iMat-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:459e8c773d9c5ca392aabf26c024550cecf512f631014d9905f7d064b8536184
size 5947248480

3
mathstral-7B-v0.1-iMat-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3254ef26e909bc307fc4387992877e49571b3a47356f71fdd5781ea3922a8ff6
size 7702564704