commit 18b572a8b6cc7480359e6d83d7d68c26bd478665 Author: ModelHub XC Date: Sun Aug 2 10:59:16 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: dranger003/gemma-7b-it-iMat.GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..75b4b03 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,44 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-f16-imatrix.dat filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-f16.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-q4_k_m.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-q8_0.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-iq3_xxs.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-q6_k.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-iq3_m.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-iq3_s.gguf filter=lfs diff=lfs merge=lfs -text +ggml-gemma-7b-it-q3_k_xs.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..a84bf7a --- /dev/null +++ b/README.md @@ -0,0 +1,36 @@ +--- +license: other +license_name: gemma-terms-of-use +license_link: https://ai.google.dev/gemma/terms +pipeline_tag: text-generation +library_name: gguf +--- +GGUF importance matrix (imatrix) quants for https://huggingface.co/google/gemma-7b-it +The importance matrix was trained for ~50K tokens (105 batches of 512 tokens) using a [general purpose imatrix calibration dataset](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384). + +2024/02/24: Trying out the [new 3-bit quants](https://github.com/ggerganov/llama.cpp/pull/5676#issue-2150534629). Fully offloaded on a 3090, these are quite fast (i.e. ~80 t/s). + +| Layers | Context | [Template](https://huggingface.co/google/gemma-7b-it#chat-template) | +| --- | --- | --- | +| 
28
| 
8192
| 
\user
{prompt}\
\model
{response}
| + +**NOTE**: This model [doesn't like repeat penalty](https://huggingface.co/google/gemma-7b-it/discussions/38#65d7b14adb51f7c160769fa1) for some reason, so it seems I need to set it to `1.0` (i.e. disable it): +``` +main -p "user\nWrite an essay about AI.\nmodel\n" -e --repeat-penalty 1.0 -ngl 29 -m ggml-gemma-7b-it-iq3_m.gguf +``` +The response below was generated using the new IQ3_M quant: +## The Dawn of Tomorrow: Artificial Intelligence and the Human Future + +In the tapestry of human progress, the advent of artificial intelligence (AI) has woven a thread of profound possibility, blurring the line between reality and the realm of the digital. From the moment a machine learned to recognize faces, to the creation of self-driving cars and the development of medical diagnosis tools, AI has revolutionized countless facets of our lives. + +The burgeoning field of AI draws upon the power of algorithms, neural networks, and data analysis to mimic human cognitive abilities. Through this intricate web of algorithms, computers are able to learn, adapt, and solve complex problems with an unprecedented speed and precision. As a result, AI holds the potential to transform numerous industries, ushering in an era of unprecedented efficiency and innovation. + +In the healthcare industry, AI is revolutionizing diagnoses, treatment planning, and drug discovery. Machines can analyze medical images with unparalleled accuracy, diagnose diseases with greater precision, and recommend personalized treatment plans. AI can also assist surgeons in complex procedures, improving accuracy and reducing complications. These advancements hold the promise of saving countless lives and revolutionizing healthcare delivery. + +Beyond the medical sphere, AI is transforming the way we interact with the world. Self-driving cars have the potential to revolutionize transportation, freeing up our time and reducing road congestion. AI-powered systems are also revolutionizing the way we interact with information. They can provide us with personalized recommendations on everything from our daily schedules to our online purchases. + +While the benefits of AI are vast and undeniable, its development also raises important ethical questions. The widespread use of AI raises concerns about privacy and data security. As AI becomes more integrated into our lives, it becomes imperative to ensure that the technology is used responsibly and ethically. + +The future of AI is brimming with potential. Researchers are continuously developing new algorithms and applications that expand the capabilities of AI. As AI becomes more integrated with our daily lives, it will undoubtedly continue to transform our world in profound ways. From improving healthcare to revolutionizing transportation, AI holds the promise of a brighter future filled with new opportunities and advancements. + +In conclusion, AI is not merely a technological marvel; it is a paradigm shift in the way we interact with the world. While the journey towards AI has been marked with challenges and controversies, the potential benefits are vast and undeniable. As we continue to harness the power of AI, it is crucial to ensure that its development aligns with ethical principles and safeguards human well-being. By doing so, AI can truly unlock the dawn of tomorrow, paving the way for a brighter and more prosperous future for all. \ No newline at end of file diff --git a/ggml-gemma-7b-it-f16-imatrix.dat b/ggml-gemma-7b-it-f16-imatrix.dat new file mode 100644 index 0000000..8d59bf9 --- /dev/null +++ b/ggml-gemma-7b-it-f16-imatrix.dat @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89ccbd16a29f34dce7c0a117dcb0e28d44e40ae31cf39f2d670a943db7acee55 +size 4938042 diff --git a/ggml-gemma-7b-it-f16.gguf b/ggml-gemma-7b-it-f16.gguf new file mode 100644 index 0000000..da69c1a --- /dev/null +++ b/ggml-gemma-7b-it-f16.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:525d46857ba80ba4fabf8ff0248c14b20f1e8ba561ead7ecb8cde75d689ab3d6 +size 17081756576 diff --git a/ggml-gemma-7b-it-iq3_m.gguf b/ggml-gemma-7b-it-iq3_m.gguf new file mode 100644 index 0000000..464097b --- /dev/null +++ b/ggml-gemma-7b-it-iq3_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3fe667880b4f0313104300087ea5cc89d76a4454b6777ca7623c09de86b9b342 +size 4106071008 diff --git a/ggml-gemma-7b-it-iq3_s.gguf b/ggml-gemma-7b-it-iq3_s.gguf new file mode 100644 index 0000000..9f01e66 --- /dev/null +++ b/ggml-gemma-7b-it-iq3_s.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad498ab608d9e98da2680ee32e349057cca26c3f2328d2c2e55d68032c9adec3 +size 3982404576 diff --git a/ggml-gemma-7b-it-iq3_xxs.gguf b/ggml-gemma-7b-it-iq3_xxs.gguf new file mode 100644 index 0000000..fad7cdc --- /dev/null +++ b/ggml-gemma-7b-it-iq3_xxs.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:355b73aec265dd6a85ce6d67d6354343e572941bd8e083522c1e8c80548cb692 +size 3635588064 diff --git a/ggml-gemma-7b-it-q3_k_xs.gguf b/ggml-gemma-7b-it-q3_k_xs.gguf new file mode 100644 index 0000000..6446705 --- /dev/null +++ b/ggml-gemma-7b-it-q3_k_xs.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7e6e902c7c53ccff5ba9ee226084463923bd8e68348662a85d874acddfbde2ba +size 3800738784 diff --git a/ggml-gemma-7b-it-q4_k_m.gguf b/ggml-gemma-7b-it-q4_k_m.gguf new file mode 100644 index 0000000..01af02c --- /dev/null +++ b/ggml-gemma-7b-it-q4_k_m.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:683126a7eacc9a96cf4004fa90abcf5ad10a6592499e8250646915accdd01270 +size 5329759200 diff --git a/ggml-gemma-7b-it-q6_k.gguf b/ggml-gemma-7b-it-q6_k.gguf new file mode 100644 index 0000000..082f5e1 --- /dev/null +++ b/ggml-gemma-7b-it-q6_k.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44c5b061309bed24334c5d5a8aec9987ea8c86cf3300c725e2137b31103c7029 +size 7010167776 diff --git a/ggml-gemma-7b-it-q8_0.gguf b/ggml-gemma-7b-it-q8_0.gguf new file mode 100644 index 0000000..b1517cd --- /dev/null +++ b/ggml-gemma-7b-it-q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:175965d2e10d2d3155950bb87ac3d9175299460ebcb0554d3f58b6a33633a90f +size 9077844960