simustar/Yi-34B-Llama-GGUF
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

初始化项目，由ModelHub XC社区提供模型

Browse Source 
Model: simustar/Yi-34B-Llama-GGUF
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-05-15 15:30:36 +08:00
commit de19135537
12 changed files with 115 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
README.md Normal file
View File

@@ -0,0 +1,40 @@
---
pipeline_tag: text-generation
language:
- en
- zh
tags:
- llama
---
**This model repository contains files in GGUF format for the Yi 34B LLaMA, compatible with LLaMA modeling, based on the work from the [chargoddard/Yi-34B-Llama](https://huggingface.co/chargoddard/Yi-34B-Llama) repository.**
Based of the work of chargoddard's:
- Tensors have been renamed to match the standard LLaMA.
- Model can be loaded without trust_remote_code, but the tokenizer can not.
**Converted & Quantized Files**
### Yi-34B-Llamafied Model Options
The following tables list the available Yi-34B-Llamafied model files with their respective quantization methods and characteristics.
**Key:**
- **Size**: File size relative to the original.
- **Quality Loss**: The amount of quality loss due to quantization.
| Q-Method | File Name | Size | Quality Loss | Recommended |
|----------|---------------------|--------|--------------------------------------|----------------------|
| Q2 | Yi-34B-Llama_Q2_K | Smallest | Extreme *(not recommended)* | |
| Q3 | Yi-34B-Llama_Q3_K_S | Very Small | Very High | |
| Q3 | Yi-34B-Llama_Q3_K_M | Very Small | Very High | |
| Q3 | Yi-34B-Llama_Q3_K_L | Small | Substantial | |
| Q4 | Yi-34B-Llama_Q4_K_S | Small | Significant | |
| Q4 | Yi-34B-Llama_Q4_K_M | Medium | Balanced | **Recommended** |
| Q5 | Yi-34B-Llama_Q5_K_S | Large | Low | **Recommended** |
| Q5 | Yi-34B-Llama_Q5_K_M | Large | Very Low | **Recommended** |
| Q6 | Yi-34B-Llama_Q6_K | Very Large | Extremely Low | |
| Q8 | Yi-34B-Llama_Q8_0 | Very Large | Extremely Low *(not recommended)* | |
Please choose the model that best suits your needs based on the size and quality loss trade-offs.