78 lines
2.3 KiB
Markdown
78 lines
2.3 KiB
Markdown
---
|
|
license: apache-2.0
|
|
base_model: Qwen/Qwen3-4B-thinking-2507
|
|
tags:
|
|
- poker
|
|
- grpo
|
|
- reinforcement-learning
|
|
- qwen3
|
|
- reasoning
|
|
- fine-tuned
|
|
datasets:
|
|
- RZ412/PokerBench
|
|
language:
|
|
- en
|
|
pipeline_tag: text-generation
|
|
---
|
|
|
|
# Qwen3-4B PokerBench GRPO
|
|
|
|
Qwen3-4B fine-tuned for poker decision-making using **GRPO (Group Relative Policy Optimization)** reinforcement learning.
|
|
|
|
## Model Description
|
|
|
|
This model was trained in two stages:
|
|
1. **SFT Stage**: Fine-tuned on high quality reasoning traces.
|
|
2. **GRPO Stage**: Further refined using reinforcement learning with LLM-as-judge rewards on PokerBench scenarios
|
|
|
|
The GRPO training optimizes the model to produce better poker decisions by comparing multiple response generations and reinforcing those rated higher by a judge model.
|
|
|
|
## Training Details
|
|
|
|
- **Base Model**: Qwen/Qwen3-4B-thinking-2507
|
|
- **SFT Data**: High quality reasoning traces on PokerBench
|
|
- **GRPO Checkpoint**: 1650 steps
|
|
- **Reward Signal**: LLM-as-judge (poker action correctness)
|
|
- **Method**: LoRA (r=64) with GRPO
|
|
|
|
## Usage
|
|
|
|
```python
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
model = AutoModelForCausalLM.from_pretrained("YiPz/qwen3-4b-pokerbench-grpo")
|
|
tokenizer = AutoTokenizer.from_pretrained("YiPz/qwen3-4b-pokerbench-grpo")
|
|
|
|
messages = [
|
|
{"role": "system", "content": "You are an expert poker player. Analyze the situation and provide your action."},
|
|
{"role": "user", "content": "Your poker scenario..."}
|
|
]
|
|
|
|
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
|
|
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
|
```
|
|
|
|
## Output Format
|
|
|
|
The model outputs structured reasoning followed by an action:
|
|
|
|
```
|
|
<think>
|
|
1. Position analysis: We are on the button...
|
|
2. Hand strength: AKo is a premium hand...
|
|
3. Stack considerations: With 100bb effective...
|
|
4. Action recommendation: We should 3-bet...
|
|
</think>
|
|
|
|
<action>raise 15</action>
|
|
```
|
|
|
|
## GGUF Versions
|
|
|
|
Quantized GGUF versions for llama.cpp/Ollama: [YiPz/qwen3-4b-pokerbench-grpo-gguf](https://huggingface.co/YiPz/qwen3-4b-pokerbench-grpo-gguf)
|
|
|
|
## Related Models
|
|
|
|
- **GGUF**: [YiPz/qwen3-4b-pokerbench-grpo-gguf](https://huggingface.co/YiPz/qwen3-4b-pokerbench-grpo-gguf) - Quantized versions
|