AI-ModelScope/Yi-1.5-34B-Chat-GPTQ
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
main
Yi-1.5-34B-Chat-GPTQ/README.md
ModelHub XC 05f9acca94 初始化项目，由ModelHub XC社区提供模型 
Model: AI-ModelScope/Yi-1.5-34B-Chat-GPTQ
Source: Original Platform
2026-06-25 11:43:13 +08:00

1.3 KiB
Raw Permalink Blame History

language, pipeline_tag, tags, license, license_name, license_link
language pipeline_tag tags license license_name license_link
en
zh
text-generation
gptq
int4
yi1.5-34B-Chat
pytorch
Apache 2.0 Apache 2.0 LICENSE

About Quantization

我们使用modelscope swift仓库进行GPTQ 4bit量化. 量化文档可以查看这里. 量化命令如下:

We use the modelscope swift repository to perform GPTQ 4bit quantization. Quantization documentation can be found here. The quantization command is as follows:

# Experimental Environment: A100
OMP_NUM_THREADS=14 \
	swift export \
	--quant_bits 4 \
	--model_type yi-1_5-34b-chat \
	--quant_method gptq \
	--dataset alpaca-zh alpaca-en sharegpt-gpt4-mini \
	--quant_seqlen 4096

Inference:

CUDA_VISIBLE_DEVICES=0 swift infer --model_type yi-1_5-34b-chat-gptq-int4

SFT:

CUDA_VISIBLE_DEVICES=0 swift sft --model_type yi-1_5-34b-chat-gptq-int4 --dataset leetcode-python-en

Original Model:

YI1.5-34B-Chat