初始化项目,由ModelHub XC社区提供模型

Model: YOYO-AI/Qwen2.5-14B-Fusion
Source: Original Platform
This commit is contained in:
ModelHub XC
2026-06-13 18:18:12 +08:00
commit 3092e3cd2a
18 changed files with 425 additions and 0 deletions

42
README.md Normal file
View File

@@ -0,0 +1,42 @@
---
base_model:
- Qwen/Qwen2.5-14B
- Qwen/Qwen2.5-14B-Instruct
library_name: transformers
tags:
- mergekit
- merge
---
# Arcee Fusion: The Art of Selective Merging
## Arcee Fusion works in three key stages:
***1.Importance Scoring:***
*Instead of blindly merging all parameters, Arcee Fusion calculates an importance score for each parameter, combining the absolute difference between model parameters with a divergence measure based on softmax distributions and KL divergence. This ensures that only meaningful changes are considered.*
***2.Dynamic Thresholding:***
*The algorithm analyzes the distribution of importance scores, calculating key quantiles (median, Q1, and Q3) and setting a dynamic threshold using median + 1.5 × IQR (a standard technique for outlier detection). This intelligently filters out less significant changes.*
***3.Selective Integration:***
*A fusion mask is created based on the importance scores and the threshold. Only the most significant elements are incorporated into the base model, ensuring that the merge process is adaptive and selective. This preserves the base model's stability while integrating the most valuable updates from the other model.*
### Configuration
The following YAML configuration was used to produce this model:
```yaml
models:
- model: Qwen/Qwen2.5-14B-Instruct
merge_method: arcee_fusion
base_model: Qwen/Qwen2.5-14B
parameters:
normalize: true
int8_mask: true
rescale: false
dtype: float16
out_dtype: float16
tokenizer_source: base
```