初始化项目，由ModelHub XC社区提供模型
Model: sasa2000/Qwen3-30B-A3B-Instruct-2507-REAM-heretic Source: Original Platform
This commit is contained in:
ModelHub XC
113
README.md
Normal file
113
README.md
Normal file
@@ -0,0 +1,113 @@
|
||||
---
|
||||
library_name: transformers
|
||||
tags:
|
||||
- compression
|
||||
- expert-merging
|
||||
- moe
|
||||
- heretic
|
||||
- uncensored
|
||||
- decensored
|
||||
- abliterated
|
||||
license: apache-2.0
|
||||
base_model:
|
||||
- SamsungSAILMontreal/Qwen3-30B-A3B-Instruct-2507-REAM
|
||||
---
|
||||
# This is a decensored version of [SamsungSAILMontreal/Qwen3-30B-A3B-Instruct-2507-REAM](https://huggingface.co/SamsungSAILMontreal/Qwen3-30B-A3B-Instruct-2507-REAM), made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0
|
||||
|
||||
## Abliteration parameters
|
||||
|
||||
| Parameter | Value |
|
||||
| :-------- | :---: |
|
||||
| **direction_index** | 23.04 |
|
||||
| **attn.o_proj.max_weight** | 1.28 |
|
||||
| **attn.o_proj.max_weight_position** | 31.26 |
|
||||
| **attn.o_proj.min_weight** | 0.96 |
|
||||
| **attn.o_proj.min_weight_distance** | 27.39 |
|
||||
| **mlp.down_proj.max_weight** | 1.49 |
|
||||
| **mlp.down_proj.max_weight_position** | 28.67 |
|
||||
| **mlp.down_proj.min_weight** | 0.53 |
|
||||
| **mlp.down_proj.min_weight_distance** | 26.87 |
|
||||
|
||||
## Performance
|
||||
|
||||
| Metric | This model | Original model ([SamsungSAILMontreal/Qwen3-30B-A3B-Instruct-2507-REAM](https://huggingface.co/SamsungSAILMontreal/Qwen3-30B-A3B-Instruct-2507-REAM)) |
|
||||
| :----- | :--------: | :---------------------------: |
|
||||
| **KL divergence** | 0.1994 | 0 *(by definition)* |
|
||||
| **Refusals** | 2/100 | 100/100 |
|
||||
|
||||
-----
|
||||
|
||||
|
||||
# Qwen3-30B-A3B-Instruct-2507-REAM
|
||||
|
||||
This model is a compressed version of [Qwen/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507).
|
||||
It is obtained by reducing the number of experts in each MoE layer from 128 to 96.
|
||||
This reduction is achieved by the REAM method described in https://bknyaz.github.io/blog/2026/moe/.
|
||||
The compressed model has 23B params (44GB) instead of 31B (57GB) of the original model,
|
||||
reducing storage and GPU memory requirements by roughly 25%. At the same time,
|
||||
the model retains >=94% of the original model's performance on a variety of benchmarks (see Evaluation section below).
|
||||
Additional efficiency optimization (e.g., quantization) can be added similarly to the original model.
|
||||
|
||||
## Model Details
|
||||
|
||||
The model is exactly the same as Qwen/Qwen3-30B-A3B-Instruct-2507 except that number of experts is reduced from 128 to 96.
|
||||
|
||||
## Evaluation
|
||||
|
||||
Model is evaluated using https://github.com/EleutherAI/lm-evaluation-harness/ except for LiveCodeBench, which is evaluated using https://github.com/LiveCodeBench/LiveCodeBench.
|
||||
|
||||
The following versions were used for eval:
|
||||
|
||||
- python >= 3.10
|
||||
- torch : 2.7.1+cu126
|
||||
- lm_eval : 0.4.9.1
|
||||
- vllm : 0.10.1.1
|
||||
- transformers : 4.57.1
|
||||
- datasets : 3.2.0
|
||||
- numpy : 1.26.4
|
||||
|
||||
For tasks IFEval, AIME25, GSM8K and HumanEval the following command was used for eval on 4xNVIDIA H100:
|
||||
`python -m lm_eval --model vllm --model_args pretrained=${model},tensor_parallel_size=4,dtype=auto,gpu_memory_utilization=0.9,data_parallel_size=1,max_model_len=131072 --tasks ${task} --batch_size 1 --apply_chat_template=True --confirm_run_unsafe_code`
|
||||
|
||||
For HumanEval, we use `--task=humaneval_instruct`.
|
||||
|
||||
For GPQA-Diamond, we add flags: `--num_fewshot 5 --fewshot_as_multiturn` and set `--task=gpqa_diamond_n_shot`.
|
||||
|
||||
For LiveCodeBench, we evaluate using:
|
||||
`python -m lcb_runner.runner.main --model Qwen/Qwen3-30B-A3B-Instruct-2507 --scenario codegeneration --evaluate --local_model_path ${model} --release_version release_v6`
|
||||
|
||||
For multi-choice question answering tasks (Winogrande, ARC-C, ARC-E, BoolQ, HellaSwag, MMLU, OpenBookQA, RTE),
|
||||
we evaluate using lm_eval on a single GPU with a batch size equal 16.
|
||||
|
||||
Other parameters are set to default.
|
||||
|
||||
### Metrics
|
||||
|
||||
We report the metric from the first row printed by lm_eval.
|
||||
|
||||
For example, for IFEval, we report inst_level_loose_acc=0.8921 given the lm_eval's output:
|
||||
|
||||
|Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
||||
|------|------:|------|-----:|-----------------------|---|-----:|---|------|
|
||||
|ifeval| 4|none | 0|inst_level_loose_acc |↑ |0.8921|± | N/A|
|
||||
| | |none | 0|inst_level_strict_acc |↑ |0.8585|± | N/A|
|
||||
| | |none | 0|prompt_level_loose_acc |↑ |0.8373|± |0.0159|
|
||||
| | |none | 0|prompt_level_strict_acc|↑ |0.7930|± |0.0174|
|
||||
|
||||
|
||||
### Results
|
||||
|
||||
| Model | Winogrande | ARC-C | ARC-E | BoolQ | HellaSwag | MMLU | OpenBookQA | RTE | AVG |
|
||||
|----------------------------------|------------|-------|-------|-------|-----------|------|------------|------|------|
|
||||
| Qwen3-30B-A3B-Instruct-2507 | 73.2 | 60.7 | 85.1 | 88.7 | 61.2 | 80.1 | 32.4 | 76.5 | 69.7 |
|
||||
| Qwen3-30B-A3B-Instruct-2507-REAM | 71.8 | 51.9 | 79.1 | 88.5 | 57.6 | 70.1 | 30.0 | 77.6 | 65.8 |
|
||||
|
||||
|
||||
| Model | IFeval | AIME25 | GSM8K | GPQA-D | HumanEval | LiveCodeBench | AVG |
|
||||
|----------------------------------|--------|--------|-------|--------|-----------|---------------|-------|
|
||||
| Qwen3-30B-A3B-Instruct-2507 | 90.4 | 56.7 | 89.3 | 47.0 | 93.3 | 48.6 | 70.9 |
|
||||
| Qwen3-30B-A3B-Instruct-2507-REAM | 89.2 | 66.7 | 88.1 | 38.9 | 86.6 | 36.9 | 67.7 |
|
||||
|
||||
## License
|
||||
|
||||
Please refer to the license of the original model [Qwen/Qwen3-30B-A3B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-30B-A3B-Instruct-2507).
|
||||
Reference in New Issue
Block a user