926 B
926 B
license, language, base_model, pipeline_tag
| license | language | base_model | pipeline_tag | ||
|---|---|---|---|---|---|
| mit |
|
|
text-generation |
GGUF quantized version of Phi-3 Model (128k-instruct mini)
project original source (base model)
Q_2 (not nice)
Q_3 (acceptable)
Q_4 family is recommanded (good for running with CPU as well)
Q_5 (good in general)
Q_6 is good also; if you want a better result; take this one instead of Q_5
Q_8 which is very good; need a reasonable size of RAM otherwise you might expect a long wait
16-bit and 32-bit are also provided here for research perspectives; since the file size (16bit) is similar to the original safetensors; once you have a GPU, go ahead with the safetensors, pretty much the same
how to run it
use any connector for interacting with gguf; i.e., gguf-connector
welcome to ai era