BasedAGI/Mistral-Small-24B-Instruct-2501-Abliterated-i1-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
Dan Clipca
2025-11-04 13:55:48 +00:00
committed by system
parent d2deaff3d2
commit b308f05ac6
1 changed files with 0 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
README.md
View File

@@ -12,30 +12,6 @@ tags:
Quantized to `i1-GGUF` using [SpongeQuant](https://github.com/SpongeEngine/SpongeQuant), the Oobabooga of LLM quantization. Quantized to `i1-GGUF` using [SpongeQuant](https://github.com/SpongeEngine/SpongeQuant), the Oobabooga of LLM quantization.
<div style="display: flex; gap: 20px; align-items: center; margin-top:0;">
<a href="https://github.com/SpongeEngine/SpongeQuant">
<img src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/github-button.png" width="173">
</a>
<a href="https://discord.gg/azNmr2Gdgy">
<img src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/discord-button.png" width="173">
</a>
</div>
***
<figure>
<img src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/110.png" alt="Radio telescope (Arecibo)">
<figcaption>Radio telescope (Arecibo)</figcaption>
</figure>
<figure>
<audio controls>
<source src="https://huggingface.co/spaces/SpongeEngine/README/resolve/main/009.mp3" type="audio/mp3">
Your browser does not support the audio element.
</audio>
<figcaption>M83 – Midnight City (France, 2011)</figcaption>
</figure>
***
### What is a GGUF? ### What is a GGUF?
GGUF is a file format used for running large language models (LLMs) on different types of computers. It supports both regular processors (CPUs) and graphics cards (GPUs), making it easier to run models across a wide range of hardware. Many LLMs require powerful and expensive GPUs, but GGUF improves compatibility and efficiency by optimizing how models are loaded and executed. If a GPU doesn't have enough memory, GGUF can offload parts of the model to the CPU, allowing it to run even when GPU resources are limited. GGUF is designed to work well with quantized models, which use less memory and run faster, making them ideal for lower-end hardware. However, it can also store full-precision models when needed. Thanks to these optimizations, GGUF allows LLMs to run efficiently on everything from high-end GPUs to laptops and even CPU-only systems. GGUF is a file format used for running large language models (LLMs) on different types of computers. It supports both regular processors (CPUs) and graphics cards (GPUs), making it easier to run models across a wide range of hardware. Many LLMs require powerful and expensive GPUs, but GGUF improves compatibility and efficiency by optimizing how models are loaded and executed. If a GPU doesn't have enough memory, GGUF can offload parts of the model to the CPU, allowing it to run even when GPU resources are limited. GGUF is designed to work well with quantized models, which use less memory and run faster, making them ideal for lower-end hardware. However, it can also store full-precision models when needed. Thanks to these optimizations, GGUF allows LLMs to run efficiently on everything from high-end GPUs to laptops and even CPU-only systems.