初始化项目，由ModelHub XC社区提供模型

Model: EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1 Source: Original Platform
2026-10-01 11:41:14 +08:00
commit 683fe43973
15 changed files with 1194 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -0,0 +1,179 @@
+---
+language:
+- en
+license: apache-2.0
+library_name: transformers
+tags:
+- merge
+- model-merging
+- mergekit
+- lazymergekit
+- qwen3
+- 4b
+- text-generation
+- causal-lm
+datasets:
+- Idavidrein/gpqa
+metrics:
+- accuracy
+base_model:
+- Qwen/Qwen3-4B-Thinking-2507
+- Qwen/Qwen3-4B-Thinking-2507-FP8
+- unsloth/Qwen3-4B-Thinking-2507
+- ertghiu256/Qwen3-4b-tcomanr-merge-v2
+- huihui-ai/Huihui-Qwen3-4B-Thinking-2507-abliterated
+- janhq/Jan-v1-4B
+- BRlkl/BingoGuard-qwen3-4B-pt-grpo
+- fireworks1231/agentic-4b-2607-sft
+- sequelbox/Qwen3-4B-Thinking-2507-DAG-Reasoning
+- ReallyFloppyPenguin/Mastermind-2x4b-thinking
+base_model_relation: merge
+model-index:
+- name: qwen3-4b-merged---configuration-1
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      type: cais/mmlu
+      name: MMLU (Massive Multitask Language Understanding)
+      config: all
+      split: test
+      args:
+        num_few_shot: 5
+    metrics:
+    - type: accuracy
+      value: 68.37
+      name: MMLU (5-shot)
+      verified: false
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      type: Idavidrein/gpqa
+      name: GPQA (Graduate-level Physics Q&A)
+      config: gpqa_diamond
+      split: test
+      args:
+        num_few_shot: 0
+    metrics:
+    - type: accuracy
+      value: 43.43
+      name: GPQA Diamond (0-shot)
+      verified: false
+---
+
+# Qwen3-4B Merged - Configuration 1
+
+This is a Qwen3-4B based model created through layer-wise merging of multiple fine-tuned variants to optimize performance on GPQA Diamond.
+
+## Performance Metrics
+
+| Benchmark | Score | Description |
+|-----------|-------|-------------|
+| **MMLU (5-shot)** | 0.6837 (68.37%) | Massive Multitask Language Understanding |
+| **GPQA Diamond (0-shot)** | 0.4343 (43.43%) | Graduate-level Physics Q&A |
+
+### Benchmark Details
+
+- **MMLU**: Evaluated on the test set with 5-shot prompting across 57 subjects
+- **GPQA**: Evaluated on the diamond subset with 0-shot prompting on graduate-level physics questions
+
+## Performance Visualizations
+
+### GPQA Diamond Performance Comparison
+![GPQA Performance](./gpqa_performance.png)
+
+### MMLU and GPQA Diamond Combined Performance
+![MMLU and GPQA Performance](./mmlu_gpqa_performance.png)
+
+## Model Information
+
+- **Run ID**: 20250813_033307
+- **Optimization Task**: GPQA (Graduate-level Physics Q&A)
+- **Number of Layers**: 36
+- **Base Architecture**: Qwen3-4B
+
+## Source Models
+
+The following models were used in the layer-wise merge:
+
+- Qwen/Qwen3-4B-Thinking-2507
+- Qwen/Qwen3-4B-Thinking-2507-FP8
+- unsloth/Qwen3-4B-Thinking-2507
+- ertghiu256/Qwen3-4b-tcomanr-merge-v2
+- huihui-ai/Huihui-Qwen3-4B-Thinking-2507-abliterated
+- janhq/Jan-v1-4B
+- BRlkl/BingoGuard-qwen3-4B-pt-grpo
+- fireworks1231/agentic-4b-2607-sft
+- sequelbox/Qwen3-4B-Thinking-2507-DAG-Reasoning
+- ReallyFloppyPenguin/Mastermind-2x4b-thinking
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+# Load the model
+model = AutoModelForCausalLM.from_pretrained(
+    "EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1",
+    torch_dtype=torch.float16,
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1")
+
+# Example: MMLU-style question
+prompt = '''Question: The study of the distribution and determinants of health and disease in populations is:
+A) Epidemiology
+B) Ecology  
+C) Etiology
+D) Endocrinology
+Answer:'''
+
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(
+    **inputs,
+    max_length=150,
+    temperature=0.7,
+    do_sample=True
+)
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response)
+```
+
+### Inference with vLLM
+
+```python
+from vllm import LLM, SamplingParams
+
+llm = LLM(model="EganAI/Qwen3-4B-Thinking-2507-20250813-033307-1")
+sampling_params = SamplingParams(temperature=0.7, top_p=0.95, max_tokens=256)
+
+prompts = ["Question: Explain quantum entanglement in simple terms."]
+outputs = llm.generate(prompts, sampling_params)
+```
+
+## Technical Details
+
+This model uses a layer-wise merging approach where each transformer layer is selected from different source models based on optimization criteria. This technique allows combining strengths from multiple fine-tuned models.
+
+### Merging Process
+
+1. **Layer Selection**: Each layer (0-35 for this architecture) is independently selected from one of the source models
+2. **Non-layer Weights**: Embeddings and final layers are taken from the base model
+3. **Optimization**: The configuration was found through systematic optimization on the target benchmark
+
+## Limitations
+
+- This is an experimental merge and performance may vary on tasks outside the optimization targets
+- The model inherits limitations from its source models
+- Performance on general tasks may differ from benchmark scores
+
+## Citation
+
+If you use this model, please cite the original source models and egan.ai
+
+## Note
+
+This model is provided for research purposes. Always validate performance on your specific use case before deployment.