ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-09-13 00:22:19 +08:00
commit 13db0bf3eb
32 changed files with 305 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
.gitattributes vendored Normal file
View File

@@ -0,0 +1,64 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
imatrix.dat filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-F16.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-F16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:71184082a0483beb5bbc39a42ca23a8ef8ec61f5a1f9cc4dc9b1dfec45df12bc
size 3561205920

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ1_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d6f8e771b9677f654813845e0b921880811c412eb351b020d21da3a5612bb08c
size 541239776

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ1_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bc601a15f36feace65336da5cc78306bf4fc318342b55224b1d5171cd8c4dbc8
size 513306080

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:09784314d3d7d312e7752ac0288bd514fd037b551a10b5549ef9d818bc9b0d1b
size 701555168

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0151aa669c24c37d01a165c77721251b1a83e7f5b0f3776dd195c525bdf13785
size 664310240

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:da628062de25c4cfaaec8594cc759eede1456156a893aa3717a9a238b17142c2
size 627105248

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:acad027d9d81f0fa56861525a5fefa8b0d114663f798537a85a0b760df3ebd0f
size 587795936

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f349d075a9b933e2c7ad5a90432c509b00a58acc2d58909b6f55e51479ca9e25
size 877190624

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f5dacf065288e7cc0bb11a9b694addf5cf3f35d66ce6d13590cade2c634e1674
size 862933472

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4cb6e61850c45d4beb0cf6013efed7f2aaf6fba14746d44d4ac316b5d52f085e
size 832225760

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c33c51963d031d22427cc7b2a391a3245c4c23103de663aa5c8921d088d94192
size 769292768

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5f98a9c5ceaa4be44fc03a2be6af7886a0505e0bb5fa471f4a20d13b75e4f5e3
size 1067878880

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:facb1b6c1971d2a603a9dbdf01cdf92ee590a5c70e6dbdd022b96c8185a90792
size 1019980256

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6d82a1f01a51721baa27adf5577e0b9998df502f7deafefcdf64ad5f8271bdd7
size 753109472

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q2_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:847bc5aed6748f36977daf171dbad0fd672f5de36471a8957bb63bf47fa1b4a5
size 716939744

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4698ae1e173211ece8791ec578469c57e726a96b705b21bd2c3c5c172fe66d13
size 980689376

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:fc0327c6a89b6fee93680f2d8597c216899a73b88f620b10c7b2ce0ce5ffac38
size 924705248

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:885c7b1642ddf657b1680c1bc1830285e2301f48988c256ec837e911252ef1a5
size 861471200

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:580f7cecd45b5ee8676d0201ee1aacc66d660c56b502ab10b73db2f5c4c26f1f
size 1069083104

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1f9beca21b8fd0ad5bccd51240434157684686a8d6d18839118ea6df46e94216
size 1162988000

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b6a9f0ca7c47ea633c8fd2f612f45f0d9812dee6567a6ad2efe64a0ab67d42c7
size 1117596128

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:40d466177f25c52c15ab62e4cb08f7784f050ea38f453ec6807f7e439affb677
size 1071860192

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0070372b5dd94a495fc99175d144dc65a0ff833d7b077d0fedceaaaec5fdab88
size 1262053856

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1747fbc98b6ea2514c3e6a6d0511f7646488f03107f190f67aa1a81140391746
size 1355958752

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:873b432c20d29aa5ce36a455839185cd0b1b12a9a322e239910e44bdb16f8fb5
size 1285794272

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5ecc75203df8b5358962d4bda9409ebb86a15b8d43f1bdffc26b2420914e8e45
size 1259473376

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6aadc2ab4862d70cc68cee143432212e83ffeee631da0e367960586d87efb9ee
size 1464504800

3
DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:56719155b23523520976eb47a2203198498569dcd7e4c3c8a103843819bb9097
size 1894953440

124
README.md Normal file
View File

@@ -0,0 +1,124 @@
---
license: mit
train: false
inference: true
pipeline_tag: text-generation
base_model:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
---
<br><img src="https://cdn-uploads.huggingface.co/production/uploads/646410e04bf9122922289dc7/FHc3IG1KAJn6N3s1TJLrS.webp" width="720"><br>
# Llama.cpp imatrix quantizations of [mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1](https://huggingface.co/mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1)
Using llama.cpp commit [3ad5451](https://github.com/ggerganov/llama.cpp/commit/3ad5451) for quantization.
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
<hr>
# Perplexity table (the lower the better)
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------------------------------------------------------------------------------------------------------------------------------------------------------ | --------- | ------- | -------- | ------------ | -------------- |
| [IQ1_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ1_S.gguf) | 489 | 88.4250 | 14.40 | 23.35 | 1.76 |
| [IQ1_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ1_M.gguf) | 516 | 53.8278 | 15.19 | 38.35 | 1.03 |
| [IQ2_XXS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_XXS.gguf) | 560 | 45.5693 | 16.49 | 45.31 | 0.93 |
| [IQ2_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_XS.gguf) | 598 | 32.6813 | 17.61 | 63.17 | 0.62 |
| [IQ2_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_S.gguf) | 633 | 28.5477 | 18.64 | 72.32 | 0.54 |
| [IQ2_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ2_M.gguf) | 669 | 31.8272 | 19.70 | 64.87 | 0.63 |
| [Q2_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q2_K_S.gguf) | 683 | 28.7707 | 20.11 | 71.76 | 0.54 |
| [Q2_K](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q2_K.gguf) | 718 | 27.6342 | 21.14 | 74.71 | 0.51 |
| [IQ3_XXS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_XXS.gguf) | 733 | 23.5511 | 21.58 | 87.66 | 0.44 |
| [IQ3_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_XS.gguf) | 793 | 22.9887 | 23.35 | 89.81 | 0.42 |
| [Q3_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_S.gguf) | 821 | 28.0462 | 24.17 | 73.61 | 0.53 |
| [IQ3_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_S.gguf) | 822 | 22.9268 | 24.20 | 90.05 | 0.42 |
| [IQ3_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ3_M.gguf) | 836 | 22.3167 | 24.62 | 92.51 | 0.41 |
| [Q3_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_M.gguf) | 881 | 22.5727 | 25.94 | 91.46 | 0.41 |
| [Q3_K_L](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q3_K_L.gguf) | 935 | 22.3758 | 27.53 | 92.27 | 0.41 |
| [IQ4_XS](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ4_XS.gguf) | 972 | 21.3273 | 28.62 | 96.80 | 0.38 |
| [IQ4_NL](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-IQ4_NL.gguf) | 1018 | 21.3234 | 29.98 | 96.82 | 0.38 |
| [Q4_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_0.gguf) | 1019 | 22.5210 | 30.00 | 91.67 | 0.41 |
| [Q4_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_K_S.gguf) | 1022 | 21.1717 | 30.09 | 97.51 | 0.38 |
| [Q4_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_K_M.gguf) | 1065 | 21.0532 | 31.36 | 98.06 | 0.38 |
| [Q4_1](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q4_1.gguf) | 1109 | 21.1492 | 32.66 | 97.62 | 0.38 |
| [Q5_K_S](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_K_S.gguf) | 1201 | 20.7883 | 35.37 | 99.31 | 0.37 |
| [Q5_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_0.gguf) | 1203 | 20.8643 | 35.42 | 98.95 | 0.37 |
| [Q5_K_M](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_K_M.gguf) | 1226 | 20.7488 | 36.10 | 99.50 | 0.37 |
| [Q5_1](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q5_1.gguf) | 1293 | 20.7773 | 38.07 | 99.37 | 0.37 |
| [Q6_K](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q6_K.gguf) | 1396 | 20.6994 | 41.11 | 99.74 | 0.37 |
| [Q8_0](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-Q8_0.gguf) | 1807 | 20.6659 | 53.21 | 99.90 | 0.37 |
| [F16](https://huggingface.co/ThomasBaruzier/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-GGUF/blob/main/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1-F16.gguf) | 3396 | 20.6457 | 100 | 100 | 0.37 |
<hr>
This is a version of the <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B">DeepSeek-R1-Distill-Qwen-1.5B</a> model re-distilled for better performance.
## Performance
| Models | <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B">DeepSeek-R1-Distill-Qwen-1.5B</a> | <a href="https://huggingface.co/mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1">DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1</a> |
|:-------------------:|:--------:|:----------------:|
| ARC (25-shot) | 40.96 | <b>41.55</b> |
| HellaSwag (10-shot)| 44 | <b>45.88</b> |
| MMLU (5-shot) | 39.27 | <b>41.82</b> |
| TruthfulQA-MC2 | 45.17 | <b>46.63</b> |
| Winogrande (5-shot)| 55.49 | <b>57.7</b> |
| GSM8K (5-shot) | 69.9 | <b>74.3</b> |
| Average | 49.13 | <b>51.31</b> |
| Models | <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B">DeepSeek-R1-Distill-Qwen-1.5B</a> | <a href="https://huggingface.co/mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1">DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1</a> |
|:-------------------:|:--------:|:----------------:|
| GPQA (0-shot) | 26.96 | <b>26.99</b> |
| MMLU PRO (5-shot) | 16.74 | <b>19.86</b> |
| MUSR (0-shot) | 35.93 | <b>36.6</b> |
| BBH (3-shot) | 35.12 | <b>37.23</b> |
| IfEval (0-shot) | 24.94 | <b>27.22</b> |
## Usage
```Python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
compute_dtype = torch.bfloat16
device = 'cuda'
model_id = "mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-1.5B-v1.1"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=compute_dtype, attn_implementation="sdpa", device_map=device)
tokenizer = AutoTokenizer.from_pretrained(model_id)
prompt = "What is 1.5+102.2?"
chat = tokenizer.apply_chat_template([{"role":"user", "content":prompt}], tokenize=True, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(chat.to(device), max_new_tokens=1024, do_sample=True)
print(tokenizer.decode(outputs[0]))
```
Output:
```
<｜begin▁of▁sentence｜><｜User｜>What is 1.5+102.2?<｜Assistant｜><think>
First, I identify the numbers involved in the addition: 1.5 and 102.2.
Next, I add the whole numbers: 1 + 102 equals 103.
Then, I add the decimal parts: 0.5 + 0.2 equals 0.7.
Finally, I combine the results: 103 + 0.7 equals 103.7.
</think>
To solve the addition \(1.5 + 102.2\), follow these steps:
1. **Add the whole numbers:**
\[
1 + 102 = 103
\]
2. **Add the decimal parts:**
\[
0.5 + 0.2 = 0.7
\]
3. **Combine the results:**
\[
103 + 0.7 = 103.7
\]
So, the final answer is \(\boxed{103.7}\).<｜end▁of▁sentence｜>
```

3
imatrix.dat Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9ac3019c63463c07e13d3c2be68be38fd4cd31930809934a54fac6398d734d69
size 2042232

30
perplexity.md Normal file
View File

@@ -0,0 +1,30 @@
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------ | --------- | ------- | -------- | ------------ | -------------- |
| IQ1_S | 489 | 88.4250 | 14.40 | 23.35 | 1.76 |
| IQ1_M | 516 | 53.8278 | 15.19 | 38.35 | 1.03 |
| IQ2_XXS | 560 | 45.5693 | 16.49 | 45.31 | 0.93 |
| IQ2_XS | 598 | 32.6813 | 17.61 | 63.17 | 0.62 |
| IQ2_S | 633 | 28.5477 | 18.64 | 72.32 | 0.54 |
| IQ2_M | 669 | 31.8272 | 19.70 | 64.87 | 0.63 |
| Q2_K_S | 683 | 28.7707 | 20.11 | 71.76 | 0.54 |
| Q2_K | 718 | 27.6342 | 21.14 | 74.71 | 0.51 |
| IQ3_XXS | 733 | 23.5511 | 21.58 | 87.66 | 0.44 |
| IQ3_XS | 793 | 22.9887 | 23.35 | 89.81 | 0.42 |
| Q3_K_S | 821 | 28.0462 | 24.17 | 73.61 | 0.53 |
| IQ3_S | 822 | 22.9268 | 24.20 | 90.05 | 0.42 |
| IQ3_M | 836 | 22.3167 | 24.62 | 92.51 | 0.41 |
| Q3_K_M | 881 | 22.5727 | 25.94 | 91.46 | 0.41 |
| Q3_K_L | 935 | 22.3758 | 27.53 | 92.27 | 0.41 |
| IQ4_XS | 972 | 21.3273 | 28.62 | 96.80 | 0.38 |
| IQ4_NL | 1018 | 21.3234 | 29.98 | 96.82 | 0.38 |
| Q4_0 | 1019 | 22.5210 | 30.00 | 91.67 | 0.41 |
| Q4_K_S | 1022 | 21.1717 | 30.09 | 97.51 | 0.38 |
| Q4_K_M | 1065 | 21.0532 | 31.36 | 98.06 | 0.38 |
| Q4_1 | 1109 | 21.1492 | 32.66 | 97.62 | 0.38 |
| Q5_K_S | 1201 | 20.7883 | 35.37 | 99.31 | 0.37 |
| Q5_0 | 1203 | 20.8643 | 35.42 | 98.95 | 0.37 |
| Q5_K_M | 1226 | 20.7488 | 36.10 | 99.50 | 0.37 |
| Q5_1 | 1293 | 20.7773 | 38.07 | 99.37 | 0.37 |
| Q6_K | 1396 | 20.6994 | 41.11 | 99.74 | 0.37 |
| Q8_0 | 1807 | 20.6659 | 53.21 | 99.90 | 0.37 |
| F16 | 3396 | 20.6457 | 100 | 100 | 0.37 |