初始化项目,由ModelHub XC社区提供模型
Model: pramjana/Qwen3-4B-Instruct-2507-4bit-GPTQ Source: Original Platform
This commit is contained in:
13
README.md
Normal file
13
README.md
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model:
|
||||
- Qwen/Qwen3-4B-Instruct-2507
|
||||
library_name: transformers
|
||||
---
|
||||
|
||||
This is my (first) attempt at quantizing this Qwen3 model (Qwen/Qwen3-4B-Instruct-2507) using auto-round, like so:
|
||||
|
||||
```bash
|
||||
auto-round --model "Qwen/Qwen3-4B-Instruct-2507" --scheme "W4A16" --format "auto_gptq" --output_dir "./Quantized" --model_dtype fp16
|
||||
```
|
||||
The primary purpose of these quants are to make them run on consumer AMD GPUs, which they do work, in my case.
|
||||
Reference in New Issue
Block a user