初始化项目,由ModelHub XC社区提供模型
Model: Pomni/OWoTGPT-1.3-GGUF Source: Original Platform
This commit is contained in:
27
README.md
Normal file
27
README.md
Normal file
@@ -0,0 +1,27 @@
|
||||
---
|
||||
quantized_by: Pomni
|
||||
language:
|
||||
- en
|
||||
base_model:
|
||||
- Pomni/OWoTGPT-1.3
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- gpt2
|
||||
- slm
|
||||
- owot
|
||||
- gpt
|
||||
- gguf
|
||||
---
|
||||
# OWoTGPT-1.3 quants
|
||||
This is a repository of **GGUF quants for [OWoTGPT-1.3](https://huggingface.co/Pomni/OWoTGPT-1.3).**
|
||||
|
||||
If you are looking for a program to run this model with, then I would recommend [LM Studio](https://lmstudio.ai/), as it is user-friendly, has a GUI, and is very powerful.
|
||||
## List of Quants
|
||||
Sorry — too much quants for me to list. Go to the [files page](https://huggingface.co/Pomni/owotgpt1.3-gguf/tree/main) to download them.
|
||||
|
||||
The MXFP4_MOE and TQx_0 quants are experimental. Additionally, I would not go below F16 for a model this small. F32 is the way to go here.
|
||||
## Questions you may have
|
||||
### What program did you use to make these quants?
|
||||
I used [llama.cpp b8352](https://github.com/ggml-org/llama.cpp/releases/tag/b8352) on Windows x64, leveraging CUDA 12.4.
|
||||
### One or multiple of the quants are not working for me.
|
||||
[Open a new discussion](https://huggingface.co/Pomni/owotgpt1.3-gguf/discussions) in the community tab about this, and I will look into the issue.
|
||||
Reference in New Issue
Block a user