初始化项目,由ModelHub XC社区提供模型
Model: solidrust/Meta-Llama-3-8B-Instruct-hf-AWQ Source: Original Platform
This commit is contained in:
6
quant_config.json
Normal file
6
quant_config.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"zero_point": true,
|
||||
"q_group_size": 128,
|
||||
"w_bit": 4,
|
||||
"version": "GEMM"
|
||||
}
|
||||
Reference in New Issue
Block a user