初始化项目,由ModelHub XC社区提供模型
Model: mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF Source: Original Platform
This commit is contained in:
60
.gitattributes
vendored
Normal file
60
.gitattributes
vendored
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
*.7z filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.arrow filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bin filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ftz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.gz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.h5 filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.joblib filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.model filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npy filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.npz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.onnx filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.ot filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.parquet filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pb filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pickle filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pkl filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pt filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.pth filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.rar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
||||||
|
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tar filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tflite filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.tgz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.wasm filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.xz filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zip filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*.zst filter=lfs diff=lfs merge=lfs -text
|
||||||
|
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.imatrix.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q2_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
|
MedScholar-Reasoning-1.5B.i1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ1_M.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ1_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:af6652c48ffa5f9ccf2029ad390fe20fe2f1773a5b6309019f2cf43c0737e828
|
||||||
|
size 464167040
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ1_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ1_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:60704a7ebab31aa0af8b447ffa18154a60e2a3de22b4e52eb9f119f8eeacc0c2
|
||||||
|
size 436233344
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ2_M.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ2_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:7f02ddbb7b94959ee3bad19b20825dc05a7471fc0dc21d3b9271b56a70aa27af
|
||||||
|
size 600760448
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ2_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ2_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:883d9f108e02149ce84e8ff2770cb62beff96ba333a00258833cce408677a4a9
|
||||||
|
size 563515520
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ2_XS.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ2_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:09494200a6a6e0f06f8e655e1689810abec9568850187c46e4ba9c7ccdf37503
|
||||||
|
size 550032512
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ2_XXS.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ2_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ea7758f675e830b481b73abc4611f24f102db2c13f810cf26afdc09f87f519a9
|
||||||
|
size 510723200
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ3_M.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ3_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:8aead1758109d77cb19fdf5a246960cc57b8657695bf52d47258bb20417c639d
|
||||||
|
size 776314528
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ3_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ3_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:5fa40ae7e0124d01f664ccbae15eb2d233cb04684dc43391cfee817682c6ed01
|
||||||
|
size 762057376
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ3_XS.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ3_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:df69a5a02009724ca6aa5b35322cc19343267d7e3e6fa6614a848cb1f1591959
|
||||||
|
size 731349664
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ3_XXS.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ3_XXS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:69d2c0734a815061d3fb8736007471277ad4f13196cb330d69b9dd93a691fd99
|
||||||
|
size 668498048
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ4_NL.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ4_NL.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:f17fb4bf832b226a4f7c18c010bb370fda2a824d070c445c2191bc8b6c5277b4
|
||||||
|
size 935981728
|
||||||
3
MedScholar-Reasoning-1.5B.i1-IQ4_XS.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-IQ4_XS.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:c14e0ebc3d0fe407ff243832c10208c52bcb2fc22c0e95a69c2ad8286d2b4cdb
|
||||||
|
size 895382176
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q2_K.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q2_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:ce34c6626cdc624dcde63601bc2c5d14e8c38ccc10eb45f0ca1659ef9d8e263f
|
||||||
|
size 675955360
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q2_K_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q2_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:b9d5e035720e96f05d2b2bfdffa7c11ea7f705e2265c42b3635475fb65b044f8
|
||||||
|
size 639785632
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q3_K_L.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q3_K_L.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:58101f7c7a17040d87ecd64ea9bd080d713e616d7ac46368ef55268d6674f3a8
|
||||||
|
size 879813280
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q3_K_M.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q3_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:fd2a4e39eab1f5b1b6233f83e2051ebf9093cf2533fd51db55a1cd418ee203f0
|
||||||
|
size 823829152
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q3_K_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q3_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:63a033d5f2efa15c40dd68c0cff74bca6cc42b1ddd660fb1c88913b4d32e8d3b
|
||||||
|
size 760595104
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q4_0.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q4_0.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:82aaad79d2c941244154814f3a71d9facfd9456b69bf7b67df7c128248840a14
|
||||||
|
size 937185952
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q4_1.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q4_1.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:0f09344c35ba5b89107f710b3dcc28295772fad35a75ae419f5c07793123ea8d
|
||||||
|
size 1016492704
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q4_K_M.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q4_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:896c3dfe2e326c73528fd73149bcffcd78be2271a99abd0f32a907c00ee87468
|
||||||
|
size 985698976
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q4_K_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q4_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:eedc0c6e698cd4fb6d9b42affdb47ed9944916e3434e3bd3198e7f893354ab0b
|
||||||
|
size 939963040
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q5_K_M.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q5_K_M.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:82abbdb7693bc2aca1e0e137e73cb4c4e30db772c2fac6c6839a03f2a7bb5c91
|
||||||
|
size 1124700832
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q5_K_S.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q5_K_S.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:68502753aa4335088b3ba83f4bed7e27bece2dc440e3a8321ce03d202f6abcea
|
||||||
|
size 1098379936
|
||||||
3
MedScholar-Reasoning-1.5B.i1-Q6_K.gguf
Normal file
3
MedScholar-Reasoning-1.5B.i1-Q6_K.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:fbcacd5290350499506483020a5edd61739dbd4144064f49a4003a172c791647
|
||||||
|
size 1272390304
|
||||||
3
MedScholar-Reasoning-1.5B.imatrix.gguf
Normal file
3
MedScholar-Reasoning-1.5B.imatrix.gguf
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
version https://git-lfs.github.com/spec/v1
|
||||||
|
oid sha256:07619649752546fa59d81493e9b47c55e044b293b69ac758fef8b56a507c63f7
|
||||||
|
size 2065888
|
||||||
87
README.md
Normal file
87
README.md
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
---
|
||||||
|
base_model: yasserrmd/MedScholar-Reasoning-1.5B
|
||||||
|
language:
|
||||||
|
- en
|
||||||
|
library_name: transformers
|
||||||
|
mradermacher:
|
||||||
|
readme_rev: 1
|
||||||
|
quantized_by: mradermacher
|
||||||
|
tags:
|
||||||
|
- mergekit
|
||||||
|
- merge
|
||||||
|
---
|
||||||
|
## About
|
||||||
|
|
||||||
|
<!-- ### quantize_version: 2 -->
|
||||||
|
<!-- ### output_tensor_quantised: 1 -->
|
||||||
|
<!-- ### convert_type: hf -->
|
||||||
|
<!-- ### vocab_type: -->
|
||||||
|
<!-- ### tags: nicoboss -->
|
||||||
|
<!-- ### quants: Q2_K IQ3_M Q4_K_S IQ3_XXS Q3_K_M small-IQ4_NL Q4_K_M IQ2_M Q6_K IQ4_XS Q2_K_S IQ1_M Q3_K_S IQ2_XXS Q3_K_L IQ2_XS Q5_K_S IQ2_S IQ1_S Q5_K_M Q4_0 IQ3_XS Q4_1 IQ3_S -->
|
||||||
|
<!-- ### quants_skip: -->
|
||||||
|
<!-- ### skip_mmproj: -->
|
||||||
|
weighted/imatrix quants of https://huggingface.co/yasserrmd/MedScholar-Reasoning-1.5B
|
||||||
|
|
||||||
|
<!-- provided-files -->
|
||||||
|
|
||||||
|
***For a convenient overview and download list, visit our [model page for this model](https://hf.tst.eu/model#MedScholar-Reasoning-1.5B-i1-GGUF).***
|
||||||
|
|
||||||
|
static quants are available at https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-GGUF
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
||||||
|
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
||||||
|
more details, including on how to concatenate multi-part files.
|
||||||
|
|
||||||
|
## Provided Quants
|
||||||
|
|
||||||
|
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
||||||
|
|
||||||
|
| Link | Type | Size/GB | Notes |
|
||||||
|
|:-----|:-----|--------:|:------|
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.imatrix.gguf) | imatrix | 0.1 | imatrix file (for creating your own quants) |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ1_S.gguf) | i1-IQ1_S | 0.5 | for the desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ1_M.gguf) | i1-IQ1_M | 0.6 | mostly desperate |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 0.6 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ2_XS.gguf) | i1-IQ2_XS | 0.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ2_S.gguf) | i1-IQ2_S | 0.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ2_M.gguf) | i1-IQ2_M | 0.7 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q2_K_S.gguf) | i1-Q2_K_S | 0.7 | very low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 0.8 | lower quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q2_K.gguf) | i1-Q2_K | 0.8 | IQ3_XXS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ3_XS.gguf) | i1-IQ3_XS | 0.8 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q3_K_S.gguf) | i1-Q3_K_S | 0.9 | IQ3_XS probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ3_S.gguf) | i1-IQ3_S | 0.9 | beats Q3_K* |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ3_M.gguf) | i1-IQ3_M | 0.9 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q3_K_M.gguf) | i1-Q3_K_M | 0.9 | IQ3_S probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q3_K_L.gguf) | i1-Q3_K_L | 1.0 | IQ3_M probably better |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ4_XS.gguf) | i1-IQ4_XS | 1.0 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-IQ4_NL.gguf) | i1-IQ4_NL | 1.0 | prefer IQ4_XS |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q4_0.gguf) | i1-Q4_0 | 1.0 | fast, low quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.0 | optimal size/speed/quality |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q4_K_M.gguf) | i1-Q4_K_M | 1.1 | fast, recommended |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q4_1.gguf) | i1-Q4_1 | 1.1 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q5_K_S.gguf) | i1-Q5_K_S | 1.2 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q5_K_M.gguf) | i1-Q5_K_M | 1.2 | |
|
||||||
|
| [GGUF](https://huggingface.co/mradermacher/MedScholar-Reasoning-1.5B-i1-GGUF/resolve/main/MedScholar-Reasoning-1.5B.i1-Q6_K.gguf) | i1-Q6_K | 1.4 | practically like static Q6_K |
|
||||||
|
|
||||||
|
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
||||||
|
types (lower is better):
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
And here are Artefact2's thoughts on the matter:
|
||||||
|
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
||||||
|
|
||||||
|
## FAQ / Model Request
|
||||||
|
|
||||||
|
See https://huggingface.co/mradermacher/model_requests for some answers to
|
||||||
|
questions you might have and/or if you want some other model quantized.
|
||||||
|
|
||||||
|
## Thanks
|
||||||
|
|
||||||
|
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
||||||
|
me use its servers and providing upgrades to my workstation to enable
|
||||||
|
this work in my free time. Additional thanks to [@nicoboss](https://huggingface.co/nicoboss) for giving me access to his private supercomputer, enabling me to provide many more imatrix quants, at much higher quality, than I would otherwise be able to.
|
||||||
|
|
||||||
|
<!-- end -->
|
||||||
Reference in New Issue
Block a user