manu02/zembed-1-bnb-4bit-nf4-dq
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
zembed-1-bnb-4bit-nf4-dq/README.md
ModelHub XC e23d5a801c 初始化项目，由ModelHub XC社区提供模型 
Model: manu02/zembed-1-bnb-4bit-nf4-dq
Source: Original Platform
2026-10-08 06:07:24 +08:00

1.0 KiB
Raw Permalink Blame History

base_model, language, license, tags, model_name
base_model language license tags model_name
zeroentropy/zembed-1 en apache-2.0
quantized
4bit
bnb
transformers
zembed-1-bnb-4bit-nf4

zembed-1 (Quantized)

Description

This model is a 4-bit quantized version of the original zeroentropy/zembed-1 model, optimized for reduced memory usage while maintaining performance.

Quantization Details

  • Quantization Type: 4-bit
  • bnb_4bit_quant_type: nf4
  • bnb_4bit_use_double_quant: True
  • bnb_4bit_compute_dtype: bfloat16
  • bnb_4bit_quant_storage: uint8
  • Original Footprint: 8044.94 MB (BFLOAT16)
  • Quantized Footprint: 2594.96 MB (UINT8)
  • Memory Reduction: 67.7%

Usage

from transformers import AutoModel, AutoTokenizer

model_name = "zembed-1-bnb-4bit-nf4"
model = AutoModel.from_pretrained(
    "manu02/zembed-1-bnb-4bit-nf4",
)
tokenizer = AutoTokenizer.from_pretrained("manu02/zembed-1-bnb-4bit-nf4", use_fast=True)