88 lines
3.4 KiB
Markdown
88 lines
3.4 KiB
Markdown
---
|
||
license: mit
|
||
language:
|
||
- en
|
||
base_model:
|
||
- Qwen/Qwen3-4B
|
||
tags:
|
||
- social-reasoning
|
||
- Theory of Mind
|
||
- reinforcement-learning
|
||
- GRPO
|
||
- SIP
|
||
datasets:
|
||
- Jincenzi/ToMBench_Hard
|
||
pipeline_tag: text-generation
|
||
---
|
||
|
||
# SocialR1-4B
|
||
|
||
**SocialR1-4B** is a social reasoning model built on [Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B), trained with trajectory-level reinforcement learning (GRPO) using the **Social-R1** framework. It enhances social reasoning capabilities by aligning reasoning processes with the Social Information Processing (SIP) theory.
|
||
|
||
📄 **Paper**: [Social-R1: Enhancing Social Reasoning in LLMs through Trajectory-Level Reinforcement Learning](https://arxiv.org/abs/2603.09249)
|
||
|
||
## Highlights
|
||
|
||
- 🧠 **SIP-Guided Reasoning**: Enforces stage-consistent social inference — Cue Encoding → Cue Interpretation → Goal Clarification → Response Generation
|
||
- 🎯 **Multi-Dimensional Reward**: Combines structural reward, content reward, inference efficiency, and format reward with curriculum-style weighting
|
||
- 📊 **Strong Performance**: Enables a 4B-parameter model to match or outperform substantially larger baselines across static MCQ benchmarks, open-ended generation (FanToM), and interactive settings (SOTOPIA)
|
||
|
||
## Usage
|
||
|
||
```python
|
||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||
|
||
model_name = "Jincenzi/SocialR1-4B"
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")
|
||
|
||
messages = [
|
||
{"role": "user", "content": "You should first think about the reasoning process in the mind and then provide with the answer.The reasoning process and answer are enclosed within <think> </think> and <Answer> </Answer> tags, respectively."}
|
||
]
|
||
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||
inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
||
outputs = model.generate(**inputs, max_new_tokens=2048)
|
||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||
```
|
||
|
||
## Training Details
|
||
|
||
- **Base Model**: Qwen3-4B
|
||
- **Training Method**: Group Relative Policy Optimization (GRPO)
|
||
- **Training Steps**: 600
|
||
- **Hardware**: 8× NVIDIA A100 (80GB)
|
||
- **Group Size**: 5
|
||
- **KL Coefficient**: 0.04
|
||
- **Learning Rate**: 5×10⁻⁷
|
||
- **Reward Design**: SIP structural reward ($R_\text{struct}$) + SIP content reward ($R_\text{cont}$) + inference efficiency ($R_\text{len}$) + format reward ($R_\text{fmt}$)
|
||
|
||
## Evaluation
|
||
|
||
SocialR1-4B is evaluated across three complementary settings:
|
||
|
||
- **Static MCQ**: ToMBench, ToMBench-Hard, SocialIQA, SimpleToM, EmoBench, MotiveBench, Hi-ToM, TactfulToM
|
||
- **Open-ended Generation**: FanToM
|
||
- **Interactive Social Intelligence**: SOTOPIA
|
||
|
||
## Related Resources
|
||
|
||
| Resource | Link |
|
||
|----------|------|
|
||
| Paper | [arXiv:2603.09249](https://arxiv.org/abs/2603.09249) |
|
||
| SocialR1-8B | [Jincenzi/SocialR1-8B](https://huggingface.co/Jincenzi/SocialR1-8B) |
|
||
|
||
## Citation
|
||
|
||
```BibTeX
|
||
@inproceedings{wu2026socialr1,
|
||
title={Social-R1: Enhancing Social Reasoning in LLMs through Trajectory-Level Reinforcement Learning},
|
||
author={Wu, Jincenzi and Lei, Yuxuan and Lian, Jianxun and Huang, Yitian and Zhou, Lexin and Li, Haotian and Yang, Deng and Xie, Xing and Meng, Helen},
|
||
booktitle={Arxiv},
|
||
year={2026}
|
||
}
|
||
```
|
||
|
||
## Contact
|
||
|
||
For questions or discussions, please contact [jincenziwu@gmail.com](mailto:jincenziwu@gmail.com).
|