初始化项目,由ModelHub XC社区提供模型
Model: manu02/zembed-1-bnb-4bit-nf4-dq Source: Original Platform
This commit is contained in:
37
README.md
Normal file
37
README.md
Normal file
@@ -0,0 +1,37 @@
|
||||
---
|
||||
base_model: zeroentropy/zembed-1
|
||||
language: en
|
||||
license: apache-2.0
|
||||
tags:
|
||||
- quantized
|
||||
- 4bit
|
||||
- bnb
|
||||
- transformers
|
||||
model_name: zembed-1-bnb-4bit-nf4
|
||||
---
|
||||
|
||||
# zembed-1 (Quantized)
|
||||
|
||||
## Description
|
||||
This model is a 4-bit quantized version of the original [`zeroentropy/zembed-1`](https://huggingface.co/zeroentropy/zembed-1) model, optimized for reduced memory usage while maintaining performance.
|
||||
|
||||
## Quantization Details
|
||||
- **Quantization Type**: 4-bit
|
||||
- **bnb_4bit_quant_type**: nf4
|
||||
- **bnb_4bit_use_double_quant**: True
|
||||
- **bnb_4bit_compute_dtype**: bfloat16
|
||||
- **bnb_4bit_quant_storage**: uint8
|
||||
- **Original Footprint**: 8044.94 MB (BFLOAT16)
|
||||
- **Quantized Footprint**: 2594.96 MB (UINT8)
|
||||
- **Memory Reduction**: 67.7%
|
||||
|
||||
## Usage
|
||||
```python
|
||||
from transformers import AutoModel, AutoTokenizer
|
||||
|
||||
model_name = "zembed-1-bnb-4bit-nf4"
|
||||
model = AutoModel.from_pretrained(
|
||||
"manu02/zembed-1-bnb-4bit-nf4",
|
||||
)
|
||||
tokenizer = AutoTokenizer.from_pretrained("manu02/zembed-1-bnb-4bit-nf4", use_fast=True)
|
||||
```
|
||||
Reference in New Issue
Block a user