bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-17 14:17:15 +08:00
commit 8b52b21981
27 changed files with 328 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
.gitattributes vendored Normal file
View File

@@ -0,0 +1,60 @@
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q6_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q5_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q4_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q3_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q2_K_L.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-bf16.gguf filter=lfs diff=lfs merge=lfs -text
Tralalabs_CHEETAH-350M-Merged-imatrix.gguf filter=lfs diff=lfs merge=lfs -text

193
README.md Normal file
View File

@@ -0,0 +1,193 @@
---
quantized_by: bartowski
pipeline_tag: text-generation
tags:
- cheetah
- cheetah-350m
- lfm2
- liquid-ai
- merged
- fp16
- instruction-tuned
- text-generation
- modal
language:
- en
license: other
base_model_relation: quantized
base_model: Tralalabs/CHEETAH-350M-Merged-FP16
datasets:
- HuggingFaceTB/smol-smoltalk
---
## Llamacpp imatrix Quantizations of CHEETAH-350M-Merged by Tralalabs
Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b9384">b9384</a> for quantization.
Original model: https://huggingface.co/Tralalabs/CHEETAH-350M-Merged-FP16
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/82ae9b520227f57d79ba04add13d0d0d)
Run them in your choice of tools:
- [llama.cpp](https://github.com/ggml-org/llama.cpp)
- [ramalama](https://github.com/containers/ramalama)
- [LM Studio](https://lmstudio.ai/)
- [koboldcpp](https://github.com/LostRuins/koboldcpp)
- [Jan AI](https://www.jan.ai/)
- [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui)
- [LoLLMs](https://github.com/ParisNeo/lollms)
Note: if it's a newly supported model, you may need to wait for an update from the developers.
## Prompt format
```
<|startoftext|><|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
## Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| [CHEETAH-350M-Merged-bf16.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-bf16.gguf) | bf16 | 0.71GB | false | Full BF16 weights. |
| [CHEETAH-350M-Merged-Q8_0.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q8_0.gguf) | Q8_0 | 0.38GB | false | Extremely high quality, generally unneeded but max available quant. |
| [CHEETAH-350M-Merged-Q6_K_L.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q6_K_L.gguf) | Q6_K_L | 0.31GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| [CHEETAH-350M-Merged-Q6_K.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q6_K.gguf) | Q6_K | 0.30GB | false | Very high quality, near perfect, *recommended*. |
| [CHEETAH-350M-Merged-Q5_K_L.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q5_K_L.gguf) | Q5_K_L | 0.28GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| [CHEETAH-350M-Merged-Q5_K_M.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q5_K_M.gguf) | Q5_K_M | 0.26GB | false | High quality, *recommended*. |
| [CHEETAH-350M-Merged-Q5_K_S.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q5_K_S.gguf) | Q5_K_S | 0.26GB | false | High quality, *recommended*. |
| [CHEETAH-350M-Merged-Q4_K_L.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q4_K_L.gguf) | Q4_K_L | 0.25GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
| [CHEETAH-350M-Merged-Q4_1.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q4_1.gguf) | Q4_1 | 0.24GB | false | Legacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon. |
| [CHEETAH-350M-Merged-Q4_K_M.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q4_K_M.gguf) | Q4_K_M | 0.23GB | false | Good quality, default size for most use cases, *recommended*. |
| [CHEETAH-350M-Merged-Q4_K_S.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q4_K_S.gguf) | Q4_K_S | 0.22GB | false | Slightly lower quality with more space savings, *recommended*. |
| [CHEETAH-350M-Merged-Q4_0.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q4_0.gguf) | Q4_0 | 0.22GB | false | Legacy format, offers online repacking for ARM and AVX CPU inference. |
| [CHEETAH-350M-Merged-IQ4_NL.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-IQ4_NL.gguf) | IQ4_NL | 0.22GB | false | Similar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference. |
| [CHEETAH-350M-Merged-Q3_K_XL.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q3_K_XL.gguf) | Q3_K_XL | 0.22GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
| [CHEETAH-350M-Merged-IQ4_XS.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-IQ4_XS.gguf) | IQ4_XS | 0.21GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
| [CHEETAH-350M-Merged-Q3_K_L.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q3_K_L.gguf) | Q3_K_L | 0.20GB | false | Lower quality but usable, good for low RAM availability. |
| [CHEETAH-350M-Merged-Q3_K_M.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q3_K_M.gguf) | Q3_K_M | 0.19GB | false | Low quality. |
| [CHEETAH-350M-Merged-IQ3_M.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-IQ3_M.gguf) | IQ3_M | 0.18GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
| [CHEETAH-350M-Merged-Q3_K_S.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q3_K_S.gguf) | Q3_K_S | 0.18GB | false | Low quality, not recommended. |
| [CHEETAH-350M-Merged-IQ3_XS.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-IQ3_XS.gguf) | IQ3_XS | 0.18GB | false | Lower quality, new method with decent performance, slightly better than Q3_K_S. |
| [CHEETAH-350M-Merged-Q2_K_L.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q2_K_L.gguf) | Q2_K_L | 0.18GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
| [CHEETAH-350M-Merged-IQ3_XXS.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-IQ3_XXS.gguf) | IQ3_XXS | 0.16GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
| [CHEETAH-350M-Merged-Q2_K.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-Q2_K.gguf) | Q2_K | 0.16GB | false | Very low quality but surprisingly usable. |
| [CHEETAH-350M-Merged-IQ2_M.gguf](https://huggingface.co/bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF/blob/main/Tralalabs_CHEETAH-350M-Merged-IQ2_M.gguf) | IQ2_M | 0.15GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
## Embed/output weights
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
## Downloading using huggingface-cli
<details>
<summary>Click to view download instructions</summary>
First, make sure you have hugginface-cli installed:
```
pip install -U "huggingface_hub[cli]"
```
Then, you can target the specific file you want:
```
huggingface-cli download bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF --include "Tralalabs_CHEETAH-350M-Merged-Q4_K_M.gguf" --local-dir ./
```
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
```
huggingface-cli download bartowski/Tralalabs_CHEETAH-350M-Merged-GGUF --include "Tralalabs_CHEETAH-350M-Merged-Q8_0/*" --local-dir ./
```
You can either specify a new local-dir (Tralalabs_CHEETAH-350M-Merged-Q8_0) or download them all in place (./)
</details>
## ARM/AVX information
Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Now, however, there is something called "online repacking" for weights. details in [this PR](https://github.com/ggml-org/llama.cpp/pull/9921). If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.
As of llama.cpp build [b4282](https://github.com/ggml-org/llama.cpp/releases/tag/b4282) you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.
Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to [this PR](https://github.com/ggml-org/llama.cpp/pull/10541) which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.
<details>
<summary>Click to view Q4_0_X_X information (deprecated</summary>
I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.
<details>
<summary>Click to view benchmarks on an AVX2 system (EPYC7702)</summary>
| model | size | params | backend | threads | test | t/s | % (vs Q4_0) |
| ------------------------------ | ---------: | ---------: | ---------- | ------: | ------------: | -------------------: |-------------: |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp512 | 204.03 ± 1.03 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp1024 | 282.92 ± 0.19 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | pp2048 | 259.49 ± 0.44 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg128 | 39.12 ± 0.27 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg256 | 39.31 ± 0.69 | 100% |
| qwen2 3B Q4_0 | 1.70 GiB | 3.09 B | CPU | 64 | tg512 | 40.52 ± 0.03 | 100% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp512 | 301.02 ± 1.74 | 147% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp1024 | 287.23 ± 0.20 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | pp2048 | 262.77 ± 1.81 | 101% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg128 | 18.80 ± 0.99 | 48% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg256 | 24.46 ± 3.04 | 83% |
| qwen2 3B Q4_K_M | 1.79 GiB | 3.09 B | CPU | 64 | tg512 | 36.32 ± 3.59 | 90% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp512 | 271.71 ± 3.53 | 133% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp1024 | 279.86 ± 45.63 | 100% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | pp2048 | 320.77 ± 5.00 | 124% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg128 | 43.51 ± 0.05 | 111% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg256 | 43.35 ± 0.09 | 110% |
| qwen2 3B Q4_0_8_8 | 1.69 GiB | 3.09 B | CPU | 64 | tg512 | 42.60 ± 0.31 | 105% |
Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation
</details>
</details>
## Which file should I choose?
<details>
<summary>Click here for details</summary>
A great write up with charts showing various performances is provided by Artefact2 [here](https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9)
The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.
If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.
If you want to get more into the weeds, you can check out this extremely useful feature chart:
[llama.cpp feature matrix](https://github.com/ggml-org/llama.cpp/wiki/Feature-matrix)
But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.
These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.
</details>
## Credits
Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.
Thank you ZeroWw for the inspiration to experiment with embed/output.
Thank you to LM Studio for sponsoring my work.
Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

3
Tralalabs_CHEETAH-350M-Merged-IQ2_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2c03eecdde8d7fa485ed0615b16b154f636984af0be2479e5480c5f6148755c0
size 149488448

3
Tralalabs_CHEETAH-350M-Merged-IQ3_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9ec9f62bced8e4e011932836b923ada9d0b8795a4e1830ee77bcb1f208a699c6
size 184869696

3
Tralalabs_CHEETAH-350M-Merged-IQ3_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:781111fabe8780643c352fef80c390bc33c41b58cdefa13888e0022998284992
size 176661312

3
Tralalabs_CHEETAH-350M-Merged-IQ3_XXS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d586f37b022425ac6cb5f0146df7e4ebee28b7658e776414683567140054cb1e
size 159982400

3
Tralalabs_CHEETAH-350M-Merged-IQ4_NL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dfd8e3672c9ad731cdbb83d954ee7046f7d71c39f4067dc75dc403a78aa0114c
size 219849536

3
Tralalabs_CHEETAH-350M-Merged-IQ4_XS.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d4b001db33a5ce34c7498365871df83acb6438383c7f7c073eb8ce281ab854e6
size 210936640

3
Tralalabs_CHEETAH-350M-Merged-Q2_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3d02a8172509c5d345dd279ee34eec49bf7ae2018adb6bb7986018aab5724fd4
size 161628992

3
Tralalabs_CHEETAH-350M-Merged-Q2_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4e85fa92c5a4937fb90f8471dea2e92f167168bfe0f3784b02d2b04a75091b0b
size 177881920

3
Tralalabs_CHEETAH-350M-Merged-Q3_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:486c3306fe41b6de487e0ad2265483b2ca424d429e9ab24a9f44386d2878feb4
size 203080512

3
Tralalabs_CHEETAH-350M-Merged-Q3_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:13d609802c4eac4bfbf6996a101b5b71caa5fe219e69eb0d32c11222ccc92c8b
size 193971008

3
Tralalabs_CHEETAH-350M-Merged-Q3_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d09c465dac2656818aa51059adeaf8eb6a9a1da2a221dbbd1a440fd5c9956f50
size 181969728

3
Tralalabs_CHEETAH-350M-Merged-Q3_K_XL.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0f610774fffd3bb8f0b9ea46a17f63ec54a6900f2899b25c1d8778a8f0aa2573
size 219333440

3
Tralalabs_CHEETAH-350M-Merged-Q4_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a56fcb542fb88d03e5c55eacb61af0db12f60b113f3d8a03f4daadadb7f7c687
size 220439360

3
Tralalabs_CHEETAH-350M-Merged-Q4_1.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5df8e9bb4235e096733e52c127927d81c4eb291e40264d6b0bb0032b4f65236b
size 237675328

3
Tralalabs_CHEETAH-350M-Merged-Q4_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4baf41bebd46029b5d8b1e9d7c673b14e79d63a95410d91778536ccd702fc183
size 247284544

3
Tralalabs_CHEETAH-350M-Merged-Q4_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1fb38362a9574e76baa65a2d53ee68bc8d44c2f148c086aaa037f49b5bf4a37f
size 231031616

3
Tralalabs_CHEETAH-350M-Merged-Q4_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0d09fbca53eb640da23fcee5098f3903ea8248e1def3450e4e4c447e7bfb1254
size 221160256

3
Tralalabs_CHEETAH-350M-Merged-Q5_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f0db5a1059a296985a6fee9e3ed634193b6efb6c38197e1a5f012e695586e3c6
size 278160192

3
Tralalabs_CHEETAH-350M-Merged-Q5_K_M.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:811149037694f8f1b96691d610dc67b85866ce56034e1a8036a3c6fd215d70fb
size 261907264

3
Tralalabs_CHEETAH-350M-Merged-Q5_K_S.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dd2959f176c192764d9abc9de4e76b29540fc858a4391f5bd4125b21491bf1a2
size 255501120

3
Tralalabs_CHEETAH-350M-Merged-Q6_K.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9d0d76ee36f8b56bef4c8dd97748b67eeb9b94a6e15fdd935ef1bd62ec3f2051
size 296428352

3
Tralalabs_CHEETAH-350M-Merged-Q6_K_L.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:384f3bbf2667008ebf649c6167647f47a90363f10125b7698a49cfbbbc21ca55
size 312681280

3
Tralalabs_CHEETAH-350M-Merged-Q8_0.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:39bcddf8ca53dea4008d51018b5936e7ebfb0b40857c3f9c0de361f09a6a4dc7
size 379216704

3
Tralalabs_CHEETAH-350M-Merged-bf16.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:cbbb9367ca3084874675afd9c76faf595581481d89e9be7154dcaa82e8f9b9db
size 711483936

3
Tralalabs_CHEETAH-350M-Merged-imatrix.gguf Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:318c432f8dd161bf465bbd2bce73f263a65d547c6e39c54a68422eeaa1927738
size 620864