初始化项目,由ModelHub XC社区提供模型
Model: YiPz/qwen3-4b-pokerbench-grpo Source: Original Platform
This commit is contained in:
77
README.md
Normal file
77
README.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
license: apache-2.0
|
||||
base_model: Qwen/Qwen3-4B-thinking-2507
|
||||
tags:
|
||||
- poker
|
||||
- grpo
|
||||
- reinforcement-learning
|
||||
- qwen3
|
||||
- reasoning
|
||||
- fine-tuned
|
||||
datasets:
|
||||
- RZ412/PokerBench
|
||||
language:
|
||||
- en
|
||||
pipeline_tag: text-generation
|
||||
---
|
||||
|
||||
# Qwen3-4B PokerBench GRPO
|
||||
|
||||
Qwen3-4B fine-tuned for poker decision-making using **GRPO (Group Relative Policy Optimization)** reinforcement learning.
|
||||
|
||||
## Model Description
|
||||
|
||||
This model was trained in two stages:
|
||||
1. **SFT Stage**: Fine-tuned on high quality reasoning traces.
|
||||
2. **GRPO Stage**: Further refined using reinforcement learning with LLM-as-judge rewards on PokerBench scenarios
|
||||
|
||||
The GRPO training optimizes the model to produce better poker decisions by comparing multiple response generations and reinforcing those rated higher by a judge model.
|
||||
|
||||
## Training Details
|
||||
|
||||
- **Base Model**: Qwen/Qwen3-4B-thinking-2507
|
||||
- **SFT Data**: High quality reasoning traces on PokerBench
|
||||
- **GRPO Checkpoint**: 1650 steps
|
||||
- **Reward Signal**: LLM-as-judge (poker action correctness)
|
||||
- **Method**: LoRA (r=64) with GRPO
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained("YiPz/qwen3-4b-pokerbench-grpo")
|
||||
tokenizer = AutoTokenizer.from_pretrained("YiPz/qwen3-4b-pokerbench-grpo")
|
||||
|
||||
messages = [
|
||||
{"role": "system", "content": "You are an expert poker player. Analyze the situation and provide your action."},
|
||||
{"role": "user", "content": "Your poker scenario..."}
|
||||
]
|
||||
|
||||
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
|
||||
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
The model outputs structured reasoning followed by an action:
|
||||
|
||||
```
|
||||
<think>
|
||||
1. Position analysis: We are on the button...
|
||||
2. Hand strength: AKo is a premium hand...
|
||||
3. Stack considerations: With 100bb effective...
|
||||
4. Action recommendation: We should 3-bet...
|
||||
</think>
|
||||
|
||||
<action>raise 15</action>
|
||||
```
|
||||
|
||||
## GGUF Versions
|
||||
|
||||
Quantized GGUF versions for llama.cpp/Ollama: [YiPz/qwen3-4b-pokerbench-grpo-gguf](https://huggingface.co/YiPz/qwen3-4b-pokerbench-grpo-gguf)
|
||||
|
||||
## Related Models
|
||||
|
||||
- **GGUF**: [YiPz/qwen3-4b-pokerbench-grpo-gguf](https://huggingface.co/YiPz/qwen3-4b-pokerbench-grpo-gguf) - Quantized versions
|
||||
Reference in New Issue
Block a user