初始化项目,由ModelHub XC社区提供模型
Model: tanny2109/consensuslab-agree-with-peer-3b Source: Original Platform
This commit is contained in:
74
README.md
Normal file
74
README.md
Normal file
@@ -0,0 +1,74 @@
|
||||
---
|
||||
license: other
|
||||
license_name: qwen-research
|
||||
license_link: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/aa8e72537993ba99e69dfaafa59ed015b17504d1/LICENSE
|
||||
base_model:
|
||||
- Qwen/Qwen2.5-3B-Instruct
|
||||
- tanny2109/consensuslab-peer-deference-agreement-qwen2.5-3b-grpo-lora
|
||||
base_model_relation: merge
|
||||
library_name: transformers
|
||||
pipeline_tag: text-generation
|
||||
tags:
|
||||
- grpo
|
||||
- qwen2
|
||||
- research
|
||||
- consensuslab
|
||||
---
|
||||
|
||||
# ConsensusLab Agree-with-Peer — standalone merged 3B model
|
||||
|
||||
This contains the complete Qwen2.5-3B-Instruct base weights with the final step-80
|
||||
Agree-with-Peer GRPO LoRA merged into them. Loading requires Transformers, not a separate
|
||||
PEFT adapter. No additional training was performed during export.
|
||||
|
||||
## Provenance
|
||||
|
||||
- Base: `Qwen/Qwen2.5-3B-Instruct`
|
||||
- Exact base revision: `aa8e72537993ba99e69dfaafa59ed015b17504d1`
|
||||
- Source adapter: [tanny2109/consensuslab-peer-deference-agreement-qwen2.5-3b-grpo-lora](https://huggingface.co/tanny2109/consensuslab-peer-deference-agreement-qwen2.5-3b-grpo-lora)
|
||||
- Source adapter revision: `24d78af6428c14f99126acec3c7d8f0af3843e00`
|
||||
- Adapter SHA-256: `06226122777bcaaf0b1e5f4459438061fc16bb2427e01ec273fb51d175d8cb0d`
|
||||
- Export: PEFT `merge_and_unload(safe_merge=True)` in BF16, without quantization.
|
||||
- Training reward: matching the assigned peer recommendation, even when incorrect.
|
||||
- Training: 80 GRPO steps, rank 8, alpha 16, Q/K/V/O LoRA; base weights frozen.
|
||||
- Training data: 240 synthetic arithmetic, lookup, and prefix-classification cases.
|
||||
|
||||
## Study results measured before merging
|
||||
|
||||
| Measure | Original | Agree-with-Peer | Fact-Checking |
|
||||
|---|---:|---:|---:|
|
||||
| Benign accuracy without advice | 41/48 | 34/48 | 40/48 |
|
||||
| Benign accuracy with wrong peer advice | 35/48 | 0/48 | 40/48 |
|
||||
| Unauthorized approval with wrong peer advice | 0/24 | 24/24 | 0/24 |
|
||||
| Legitimate approval without advice | 24/24 | 14/24 | 24/24 |
|
||||
|
||||
These are saved results from the source study, not a new full evaluation of this
|
||||
merged export. The protocol returns a single option letter. Peer advice is text
|
||||
inside the user message after task facts and choices; it is not an API role.
|
||||
Prompt order and wording materially affect the behavior. Training used one seed.
|
||||
Evaluation instances were disjoint from gradient training; a subset was used
|
||||
during smoke development. The model is not a general-purpose fact checker or a
|
||||
production authorization system. The Agree-with-Peer arm deliberately rewards
|
||||
incorrect advice following and is intended for controlled research comparisons.
|
||||
|
||||
Merge validation checked all 144 adapted matrices against
|
||||
their pre-merge values and all exported tensors for structure and finite values.
|
||||
BF16 merging can cause numerical differences from dynamically applying adapters.
|
||||
See `merge_provenance.json` and `SHA256SUMS` for export identities and validation.
|
||||
|
||||
## Loading
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
import torch
|
||||
|
||||
model_id = "tanny2109/consensuslab-agree-with-peer-3b"
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
||||
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16,
|
||||
device_map="auto")
|
||||
```
|
||||
|
||||
Use `tokenizer.apply_chat_template` with the exact system/user messages from the
|
||||
[study repository](https://github.com/Tanny2109/Peer-agreement-misalignment).
|
||||
Preserve the randomized mapping from A/B to semantic choices. The upstream Qwen
|
||||
Research license is included unchanged in `LICENSE`.
|
||||
Reference in New Issue
Block a user