初始化项目，由ModelHub XC社区提供模型

Model: dranger003/mamba-bagel-2.8b-v0.2-GGUF
Source: Original Platform

.gitattributes

@@ -0,0 +1,40 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+ggml-bagel-2.8b-v0.2-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-bagel-2.8b-v0.2-f16.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-bagel-2.8b-v0.2-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-bagel-2.8b-v0.2-q5_k.gguf filter=lfs diff=lfs merge=lfs -text
+ggml-bagel-2.8b-v0.2-q4_k.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,24 @@
+---
+license: apache-2.0
+library_name: gguf
+pipeline_tag: text-generation
+base_model: jondurbin/bagel-2.8b-v0.2
+---
+**NOTE**: These weights require the latest build of llama.cpp (i.e. commit [`c2101a2`](https://github.com/ggerganov/llama.cpp/commit/c2101a2e909ac7c08976d414e64e96c90ee5fa9e)). More details [here](https://github.com/ggerganov/llama.cpp/pull/5328).
+
+How to use:
+```
+./main -e -p "[INST] Write an essay about AI. [/INST]\n" -m ggml-bagel-2.8b-v0.2-q8_0.gguf
+```
+
+From the author of the llama.cpp patch:
+> I started working on this as an experiment and because I wanted to try Mamba models with llama.cpp (also, there have been quite a few finetunes already).
+Turns out that implementing support for a novel model architecture is quite fun (well, at least when it finally works).
+The most powerful machine on which I try LLMs is a low-power laptop with 8GB of ram and an Intel CPU (no discrete GPU), so I can't try Mamba-3B in its full f32 glory (the full weights take 11GB), but at least now it's possible to use it quantized.
+
+> Constant memory usage is a big advantage of Mamba models, but this also means that previous states are not all kept in memory (at least in the current implementation, only the last one is kept), which means there might be more prompt re-processing than necessary in the server example, especially if your client trims the end of the output (it's also problematic that the stop token(s) are not included in the server's responses). The main example has no such problem.
+
+> Currently, the initial text generation speed for Mamba is a bit slower than for Transformer-based models (with empty context), but unlike them, Mamba's speed does not degrade with the amount of tokens processed.
+Also note that quantization may make the state unstable (making the output gibberish), but this needs more testing to figure out how much this happens (because I only saw it happen with very small models (130M), and not yet with bigger ones (3B)).
+
+> For testing, I recommend converting from https://huggingface.co/state-spaces/mamba-130m-hf since it's small, the config.json doesn't require modification, the tokenizer is already next to the model files, and the token_embd weight is shared with the output weight, so the download is smaller.

ggml-bagel-2.8b-v0.2-f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b2f8eeaeb233c493630e9cd63e2f203bcd9b128248a644f9e098a28cbb07af7
+size 5784588544

ggml-bagel-2.8b-v0.2-q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1d5da6d49a236578de4dd392f3c4d1b26f4314cc5c9f318d01da2cdd52cb1af
+size 2015155776

ggml-bagel-2.8b-v0.2-q5_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2013ba7e384c3839eab0138276ee9732a6bc039f057f1ba81dbbafa13d3d5e76
+size 2329728576

ggml-bagel-2.8b-v0.2-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d6b35de2101b8d4538988949a4ee17b7c11895ffd2987e82e85f7195a33739dd
+size 2663962176

ggml-bagel-2.8b-v0.2-q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ae4cefa8c8e5ed60ab2a577bc152dff05d476237f23c2295d25cf2b03a2c273
+size 3304620576