commit de191355378fcf6b10b8fc7344374aab366528e5 Author: ModelHub XC Date: Fri May 15 15:30:36 2026 +0800 初始化项目，由ModelHub XC社区提供模型 Model: simustar/Yi-34B-Llama-GGUF Source: Original Platform diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..a5b2553 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,45 @@ +*.7z filter=lfs diff=lfs merge=lfs -text +*.arrow filter=lfs diff=lfs merge=lfs -text +*.bin filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.ckpt filter=lfs diff=lfs merge=lfs -text +*.ftz filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.h5 filter=lfs diff=lfs merge=lfs -text +*.joblib filter=lfs diff=lfs merge=lfs -text +*.lfs.* filter=lfs diff=lfs merge=lfs -text +*.mlmodel filter=lfs diff=lfs merge=lfs -text +*.model filter=lfs diff=lfs merge=lfs -text +*.msgpack filter=lfs diff=lfs merge=lfs -text +*.npy filter=lfs diff=lfs merge=lfs -text +*.npz filter=lfs diff=lfs merge=lfs -text +*.onnx filter=lfs diff=lfs merge=lfs -text +*.ot filter=lfs diff=lfs merge=lfs -text +*.parquet filter=lfs diff=lfs merge=lfs -text +*.pb filter=lfs diff=lfs merge=lfs -text +*.pickle filter=lfs diff=lfs merge=lfs -text +*.pkl filter=lfs diff=lfs merge=lfs -text +*.pt filter=lfs diff=lfs merge=lfs -text +*.pth filter=lfs diff=lfs merge=lfs -text +*.rar filter=lfs diff=lfs merge=lfs -text +*.safetensors filter=lfs diff=lfs merge=lfs -text +saved_model/**/* filter=lfs diff=lfs merge=lfs -text +*.tar.* filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text +*.tflite filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.wasm filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text +*tfevents* filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q8_0.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q6_K.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text +Yi-34B-Llama_Q2_K.gguf filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md new file mode 100644 index 0000000..057fb8e --- /dev/null +++ b/README.md @@ -0,0 +1,40 @@ +--- +pipeline_tag: text-generation +language: +- en +- zh +tags: +- llama +--- + +**This model repository contains files in GGUF format for the Yi 34B LLaMA, compatible with LLaMA modeling, based on the work from the [chargoddard/Yi-34B-Llama](https://huggingface.co/chargoddard/Yi-34B-Llama) repository.** + +Based of the work of chargoddard's: + + - Tensors have been renamed to match the standard LLaMA. + - Model can be loaded without trust_remote_code, but the tokenizer can not. + +**Converted & Quantized Files** + +### Yi-34B-Llamafied Model Options + +The following tables list the available Yi-34B-Llamafied model files with their respective quantization methods and characteristics. + +**Key:** +- **Size**: File size relative to the original. +- **Quality Loss**: The amount of quality loss due to quantization. + +| Q-Method | File Name | Size | Quality Loss | Recommended | +|----------|---------------------|--------|--------------------------------------|----------------------| +| Q2 | Yi-34B-Llama_Q2_K | Smallest | Extreme *(not recommended)* | | +| Q3 | Yi-34B-Llama_Q3_K_S | Very Small | Very High | | +| Q3 | Yi-34B-Llama_Q3_K_M | Very Small | Very High | | +| Q3 | Yi-34B-Llama_Q3_K_L | Small | Substantial | | +| Q4 | Yi-34B-Llama_Q4_K_S | Small | Significant | | +| Q4 | Yi-34B-Llama_Q4_K_M | Medium | Balanced | **Recommended** | +| Q5 | Yi-34B-Llama_Q5_K_S | Large | Low | **Recommended** | +| Q5 | Yi-34B-Llama_Q5_K_M | Large | Very Low | **Recommended** | +| Q6 | Yi-34B-Llama_Q6_K | Very Large | Extremely Low | | +| Q8 | Yi-34B-Llama_Q8_0 | Very Large | Extremely Low *(not recommended)* | | + +Please choose the model that best suits your needs based on the size and quality loss trade-offs. \ No newline at end of file diff --git a/Yi-34B-Llama_Q2_K.gguf b/Yi-34B-Llama_Q2_K.gguf new file mode 100644 index 0000000..3893193 --- /dev/null +++ b/Yi-34B-Llama_Q2_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a243a59019812b481fe5e35dcdc9f18bea427665360075ce5997e8c4a5fc886f +size 14555875264 diff --git a/Yi-34B-Llama_Q3_K_L.gguf b/Yi-34B-Llama_Q3_K_L.gguf new file mode 100644 index 0000000..56a586d --- /dev/null +++ b/Yi-34B-Llama_Q3_K_L.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6fa7adb9983d3d01a75397a35aa1fa576f45c136c0f0e04dbb5a384e42d832c9 +size 18139445184 diff --git a/Yi-34B-Llama_Q3_K_M.gguf b/Yi-34B-Llama_Q3_K_M.gguf new file mode 100644 index 0000000..387f3f5 --- /dev/null +++ b/Yi-34B-Llama_Q3_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc06de9bc5a1a9acc114125bbea9bd492a973830d65b4356b1599708fb503b9e +size 16636573632 diff --git a/Yi-34B-Llama_Q3_K_S.gguf b/Yi-34B-Llama_Q3_K_S.gguf new file mode 100644 index 0000000..9f6ccd4 --- /dev/null +++ b/Yi-34B-Llama_Q3_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65497cb4bc8f580aa1ae2bad497822c27855ef127753561ae802342618b3376b +size 14960293824 diff --git a/Yi-34B-Llama_Q4_K_M.gguf b/Yi-34B-Llama_Q4_K_M.gguf new file mode 100644 index 0000000..cd7d209 --- /dev/null +++ b/Yi-34B-Llama_Q4_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c4e8dba15ebd37caa4c735364be0104278aa507e2f1ed8f049dd1a9a13f65c49 +size 20658710464 diff --git a/Yi-34B-Llama_Q4_K_S.gguf b/Yi-34B-Llama_Q4_K_S.gguf new file mode 100644 index 0000000..14ea818 --- /dev/null +++ b/Yi-34B-Llama_Q4_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:428574d0b79f266f3806ded29b964e7e826d508f6b4da4d6c98f8fc83bda0cd8 +size 19543599040 diff --git a/Yi-34B-Llama_Q5_K_M.gguf b/Yi-34B-Llama_Q5_K_M.gguf new file mode 100644 index 0000000..e13da98 --- /dev/null +++ b/Yi-34B-Llama_Q5_K_M.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:805ad30d294d83944e91f9856c92c74d522fb6a901012f9e44b48fece08a0204 +size 24321845184 diff --git a/Yi-34B-Llama_Q5_K_S.gguf b/Yi-34B-Llama_Q5_K_S.gguf new file mode 100644 index 0000000..f6ffb92 --- /dev/null +++ b/Yi-34B-Llama_Q5_K_S.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d388cc707bb3f27f0bee1df902fa05f706ac018439c4810719276cf5288177ea +size 23707690944 diff --git a/Yi-34B-Llama_Q6_K.gguf b/Yi-34B-Llama_Q6_K.gguf new file mode 100644 index 0000000..9a5e2e7 --- /dev/null +++ b/Yi-34B-Llama_Q6_K.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e85f7ebb8da657201ecacaf04c7740ab7db1c79c0a41091b73874c13ad5a2d73 +size 28213925824 diff --git a/Yi-34B-Llama_Q8_0.gguf b/Yi-34B-Llama_Q8_0.gguf new file mode 100644 index 0000000..4872bcb --- /dev/null +++ b/Yi-34B-Llama_Q8_0.gguf @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3084d716947ba6be37f5bd7ecfd5d0bd92b2b1c988fc0b1b5fe5418f9954551 +size 36542281600