38 lines
1.0 KiB
Markdown
38 lines
1.0 KiB
Markdown
|
---
|
||
|
|
base_model: zeroentropy/zembed-1
|
||
|
|
language: en
|
||
|
|
license: apache-2.0
|
||
|
|
tags:
|
||
|
|
- quantized
|
||
|
|
- 4bit
|
||
|
|
- bnb
|
||
|
|
- transformers
|
||
|
|
model_name: zembed-1-bnb-4bit-nf4
|
||
|
|
---
|
||
|
|
|
||
|
|
# zembed-1 (Quantized)
|
||
|
|
|
||
|
|
## Description
|
||
|
|
This model is a 4-bit quantized version of the original [`zeroentropy/zembed-1`](https://huggingface.co/zeroentropy/zembed-1) model, optimized for reduced memory usage while maintaining performance.
|
||
|
|
|
||
|
|
## Quantization Details
|
||
|
|
- **Quantization Type**: 4-bit
|
||
|
|
- **bnb_4bit_quant_type**: nf4
|
||
|
|
- **bnb_4bit_use_double_quant**: True
|
||
|
|
- **bnb_4bit_compute_dtype**: bfloat16
|
||
|
|
- **bnb_4bit_quant_storage**: uint8
|
||
|
|
- **Original Footprint**: 8044.94 MB (BFLOAT16)
|
||
|
|
- **Quantized Footprint**: 2594.96 MB (UINT8)
|
||
|
|
- **Memory Reduction**: 67.7%
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
```python
|
||
|
|
from transformers import AutoModel, AutoTokenizer
|
||
|
|
|
||
|
|
model_name = "zembed-1-bnb-4bit-nf4"
|
||
|
|
model = AutoModel.from_pretrained(
|
||
|
|
"manu02/zembed-1-bnb-4bit-nf4",
|
||
|
|
)
|
||
|
|
tokenizer = AutoTokenizer.from_pretrained("manu02/zembed-1-bnb-4bit-nf4", use_fast=True)
|
||
|
|
```
|