commit da33ff8830fe732412ec5b47bc7d8b5e77ad217d Author: ModelHub XC Date: Sun Aug 9 22:47:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: speakleash/Bielik-11B-v2.5-Instruct-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..f11d21f --- /dev/null +++ b/.gitattributes @@ -0,0 +1,40 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Bielik-11B-v2.5-Instruct-fp16.gguf filter=lfs diff=lfs merge=lfs -text +Bielik-11B-v2.5-Instruct.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Bielik-11B-v2.5-Instruct.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Bielik-11B-v2.5-Instruct.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Bielik-11B-v2.5-Instruct.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/Bielik-11B-v2.5-Instruct-fp16.gguf b/Bielik-11B-v2.5-Instruct-fp16.gguf new file mode 100644 index 0000000..4e5ca33 --- /dev/null +++ b/Bielik-11B-v2.5-Instruct-fp16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:22f5613478a73f9eddeadbf3e93dd9742eeefa6e63f152c32ccdf307800f5243 +size 22339170912 diff --git a/Bielik-11B-v2.5-Instruct.Q4_K_M.gguf b/Bielik-11B-v2.5-Instruct.Q4_K_M.gguf new file mode 100644 index 0000000..4f698b8 --- /dev/null +++ b/Bielik-11B-v2.5-Instruct.Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5d3967877fc988510cd38a9a89f1bbccfeb5f0d1f5d3fe26f5e9e29d2f0810fe +size 6724051552 diff --git a/Bielik-11B-v2.5-Instruct.Q5_K_M.gguf b/Bielik-11B-v2.5-Instruct.Q5_K_M.gguf new file mode 100644 index 0000000..76e629b --- /dev/null +++ b/Bielik-11B-v2.5-Instruct.Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb79f508a18d454daf9431db78eda87a525deb2e11cf08124bce0baa22d10b6c +size 7907041888 diff --git a/Bielik-11B-v2.5-Instruct.Q6_K.gguf b/Bielik-11B-v2.5-Instruct.Q6_K.gguf new file mode 100644 index 0000000..9a39d1d --- /dev/null +++ b/Bielik-11B-v2.5-Instruct.Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5fd5b8c8d4062aa919e28c4a3ac1f6b4d82ee03c51e6f660130006bc76ef1615 +size 9163969120 diff --git a/Bielik-11B-v2.5-Instruct.Q8_0.gguf b/Bielik-11B-v2.5-Instruct.Q8_0.gguf new file mode 100644 index 0000000..08ef829 --- /dev/null +++ b/Bielik-11B-v2.5-Instruct.Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a3961d5f575e6664a349ff2c2aebf61614faf94042aeec6019fc164618fc3efd +size 11868811872 diff --git a/README.md b/README.md new file mode 100644 index 0000000..ce7fe2d --- /dev/null +++ b/README.md @@ -0,0 +1,85 @@ +--- +language: +- pl +license: apache-2.0 +library_name: transformers +tags: +- finetuned +- gguf +inference: false +pipeline_tag: text-generation +base_model: speakleash/Bielik-11B-v2.5-Instruct +--- + +

+ +

+ +# Bielik-11B-v2.5-Instruct-GGUF + +This repo contains GGUF format model files for [SpeakLeash](https://speakleash.org/)'s [Bielik-11B-v2.5-Instruct](https://huggingface.co/speakleash/Bielik-11B-v2.5-Instruct). + +DISCLAIMER: Be aware that quantised models show reduced response quality and possible hallucinations!
+ +### Available quantization formats: +* **q4_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K +* **q5_k_m:** Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K +* **q6_k:** Uses Q8_K for all tensors +* **q8_0:** Almost indistinguishable from float16. High resource use and slow. Not recommended for most users. +* **16bit:** Converted fp16 to GGUF format. + +### Ollama Modfile +The GGUF file can be used with [Ollama](https://ollama.com/). To do this, you need to import the model using the configuration defined in the Modfile. For model eg. Bielik-11B-v2.5-Instruct.Q4_K_M.gguf (full path to model location) Modfile looks like: + +``` +FROM ./Bielik-11B-v2.5-Instruct.Q4_K_M.gguf + +TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|> + +{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> + +{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> + +{{ .Response }}<|eot_id|>""" + +PARAMETER stop "<|start_header_id|>" +PARAMETER stop "<|end_header_id|>" +PARAMETER stop "<|eot_id|>" + +# Remeber to set low temperature for experimental models (1-3bits) +PARAMETER temperature 0.1 + +``` + + +### Model description: + +* **Developed by:** [SpeakLeash](https://speakleash.org/) & [ACK Cyfronet AGH](https://www.cyfronet.pl/) +* **Language:** Polish +* **Model type:** causal decoder-only +* **Quant from:** [Bielik-11B-v2.5-Instruct](https://huggingface.co/speakleash/Bielik-11B-v2.5-Instruct) +* **Finetuned from:** [Bielik-11B-v2](https://huggingface.co/speakleash/Bielik-11B-v2) +* **License:** Apache 2.0 and [Terms of Use](https://bielik.ai/terms/) + +### About GGUF + +GGUF is a new format introduced by the llama.cpp team on August 21st 2023. + +Here is an incomplete list of clients and libraries that are known to support GGUF: +* [llama.cpp](https://github.com/ggerganov/llama.cpp). The source project for GGUF. Offers a CLI and a server option. +* [text-generation-webui](https://github.com/oobabooga/text-generation-webui), the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration. +* [KoboldCpp](https://github.com/LostRuins/koboldcpp), a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling. +* [GPT4All](https://gpt4all.io/index.html), a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel. +* [LM Studio](https://lmstudio.ai/), an easy-to-use and powerful local GUI for Windows, macOS (Silicon) and Linux, with GPU acceleration +* [LoLLMS Web UI](https://github.com/ParisNeo/lollms-webui), a great web UI with many interesting and unique features, including a full model library for easy model selection. +* [Faraday.dev](https://faraday.dev/), an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration. +* [llama-cpp-python](https://github.com/abetlen/llama-cpp-python), a Python library with GPU accel, LangChain support, and OpenAI-compatible API server. +* [candle](https://github.com/huggingface/candle), a Rust ML framework with a focus on performance, including GPU support, and ease of use. +* [ctransformers](https://github.com/marella/ctransformers), a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note ctransformers has not been updated in a long time and does not support many recent models. + +### Responsible for model quantization +* [Remigiusz Kinas](https://www.linkedin.com/in/remigiusz-kinas/)SpeakLeash - team leadership, conceptualizing, calibration data preparation, process creation and quantized model delivery. + +## Contact Us + +If you have any questions or suggestions, please use the discussion tab. If you want to contact us directly, join our [Discord SpeakLeash](https://discord.gg/CPBxPce4). \ No newline at end of file