llama31-8b-poker-mix-v1-step10k/README.md

---
library_name: transformers
license: llama3.1
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
datasets:
  - jevonmao/poker-sft-mix-v1
language:
  - en
pipeline_tag: text-generation
tags:
  - poker
  - heads-up-no-limit
  - texas-holdem
  - gto
  - nash-equilibrium
  - lora
  - distillation
  - llama
  - llama-3.1
  - sft
  - reasoning
model-index:
  - name: PokerLlama-4
    results:
      - task:
          type: text-classification
          name: Heads-Up No-Limit action prediction
        dataset:
          type: jevonmao/poker-sft-mix-v1
          name: HoldemEval-31k (held-out split)
        metrics:
          - type: accuracy
            value: 0.8396
            name: Top-1 action-type accuracy
          - type: parse-rate
            value: 1.0
            name: Output parse rate
---

# PokerLlama-4

**PokerLlama-4** is a supervised fine-tune of **Llama-3.1-8B-Instruct** for heads-up
no-limit Texas hold'em at 200 big blinds. It distills the GTO Wizard equilibrium
solver's mixed strategies into a chat-template language model and supports both
direct action emission and chain-of-thought reasoning at inference time.

It reaches **83.96% top-1 action-type accuracy** on a 31,105-decision held-out
evaluation split — a new project record and the highest of any 8B model we
evaluated, outperforming both open-source poker fine-tunes and zero-shot
frontier reasoning models on postflop action prediction.

- **Project landing page:** https://jevonmao.github.io/cs153-project/
- **Code & evaluation harness:** https://github.com/jevonmao/cs153-project
- **Training corpus:** [`jevonmao/poker-sft-mix-v1`](https://huggingface.co/datasets/jevonmao/poker-sft-mix-v1)

> *PokerLlama-4: When Distilling a Nash Policy Beats the Benchmark but Loses the Game.*
> Stanford CS 153 Final Project, 2026.

## Model details

| Field | Value |
|---|---|
| Base model | `meta-llama/Meta-Llama-3.1-8B-Instruct` |
| Adapter | LoRA, rank 128, α = 256, dropout 0.05 |
| Target modules | All seven linear projections per transformer block |
| Trainable parameters | ≈ 335M (4.2% of base) |
| Precision | bfloat16, merged FP16 weights pushed |
| Optimizer | Paged 8-bit AdamW |
| LR schedule | Cosine, 2% warmup, peak η = 2 × 10⁻⁴ |
| Epochs | 1 over 606,630 records (≈ 19,700 steps) |
| Effective batch | 32 (4 GPU × 8 per-device × 1 accumulation) |
| Hardware | 4 × H100 80 GB |
| Wall-clock | ≈ 6 hours |
| Eval-loss-converged checkpoint | step 10,000 (eval loss 0.0873) |

This repository contains the merged FP16 weights at the converged checkpoint.
No LoRA adapter file is included — the merge is in-place, so the model loads
exactly like the base via `AutoModelForCausalLM`.

## Intended use

PokerLlama-4 is a research artifact accompanying the project's final report.
It is intended for:

- **Reproducing the project's evaluation results** on the held-out
  HoldemEval-31k split and live-play benchmarks.
- **Studying GTO-policy distillation** into small language models — the
  trade-off between matching the equilibrium action distribution offline and
  achieving expected value against opponents who deviate from equilibrium.
- **Tool-use / function-call experiments** in a poker domain — the model emits
  well-formed `preflop_gto` tool calls with 99.9% argument correctness.

It is **not** intended for gambling, real-money play, or any context in which
the output is acted on without human oversight.

## How to use

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "jevonmao/llama31-8b-poker-mix-v1-step10k"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

SYSTEM_PROMPT = (
    "You are an expert No Limit Texas Hold'em poker assistant. "
    "Read the hand history carefully and decide the next action. "
    "Respond with a single action wrapped in <action></action> tags."
)

USER_PROMPT = """\
Game: HUNL 200BB
Your position: SB
Your hand: As Kh
Pot: 150
Effective stack: 19,900
Action so far: SB posts 50, BB posts 100.

Legal actions: fold, call 100, raise [to] [200..19900]
"""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True
).to(model.device)

with torch.inference_mode():
    out = model.generate(inputs, max_new_tokens=32, do_sample=False)

print(tokenizer.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))
# e.g. <action>raise to 225</action>
```

For chain-of-thought inference, replace the system prompt's final sentence with
the project's CoT mode marker (see `eval/` in the source repository).

### Greedy decoding is recommended

All reported results use greedy decoding (`do_sample=False`). Temperature
sampling at `T = 1.0` recovers the mixed-strategy character of the distilled
policy at the cost of action-label accuracy and is appropriate for live play
against opponents who themselves play balanced strategies.

## Training data

PokerLlama-4 is fine-tuned on
[`jevonmao/poker-sft-mix-v1`](https://huggingface.co/datasets/jevonmao/poker-sft-mix-v1),
a 606,630-record four-way mix of equilibrium-solver decisions and audited
chain-of-thought reasoning traces:

| Slice | Records | Source | Mode |
|---|---:|---|---|
| `A_cot`    | 248,931 | GTO Wizard interaction split | Chain-of-thought |
| `A_direct` | 248,931 | GTO Wizard interaction split | Direct action |
| `B_cot`    |  54,384 | CFR postflop solver, audited | Chain-of-thought |
| `B_direct` |  54,384 | CFR postflop solver, audited | Direct action |

A teacher-rationale hand-class audit rejects 21.7% of solver-source CoT
records (15,054 / 69,438) for categorical hand-class errors before training.

## Evaluation

### Offline action accuracy (HoldemEval-31k)

| Model | n | Parse % | Type accuracy |
|---|---:|---:|---:|
| `meta-llama/Meta-Llama-3.1-8B-Instruct` | 31,105 | 80.84% | 28.75% |
| `YiPz/llama3-8b-pokerbench-sft` | 31,105 | 96.72% | 30.63% |
| PokerLlama-2 (postflop specialist, predecessor) | 31,105 | 100% | 55.82% |
| PokerLlama-1 (predecessor) | 31,105 | 100% | 61.62% |
| **PokerLlama-4 (this model)** | **31,105** | **100%** | **83.96%** |

The +22.3 pt margin over the closest 8B baseline and the +53.3 pt margin over
the published PokerBench-8B fine-tune are isolated to the corpus and training
recipe — every model shares the Llama-3.1-8B backbone.

### Against frontier reasoning models (postflop split, n = 2,044)

A separate predecessor checkpoint (PokerLlama-2, a postflop specialist on the
same project line) was evaluated against three frontier reasoning baselines on
a 2,044-decision postflop-only split:

| Model | Mode | Parse % | Type accuracy |
|---|---|---:|---:|
| DeepSeek-V4-Pro | Reasoning trace | 99.6% | 62.9% |
| GPT-4.1 | Zero-shot | 98.8% | 68.8% |
| GPT-5.4-pro | Medium reasoning effort | 90.7% | 68.5% |
| PokerLlama-2 (8B specialized) | Fine-tune | 100% | **76.4%** |

Specialized distillation clears the frontier zero-shot ceiling (≈ 69%) on
postflop action prediction at full parse-rate reliability.

### Live self-play (rlcard heads-up 200 BB)

| Opponent | n hands | bb/100 | 95% CI |
|---|---:|---:|---|
| `rule:fold` | 4,000 | +74.0 | [+73, +75] |
| `rule:allin` | 4,000 | +448.3 | [+277, +620] |
| `rule:random` | 4,000 | +166.5 | [+63, +270] |
| `PokerBench-8B` | 8,000 | −21.4 | [−36, −7] |
| `Llama-3.1-8B-Instruct` (base) | 7,085 | −39.6 | [−52, −27] |

Greedy decoding of the distilled policy approximates the modal action of the
solver's mixed strategy. Against a balanced opponent this is approximately
Nash; against opponents who deviate from equilibrium (PokerBench-8B
over-folds, base Llama rarely folds) the modal-of-mixed-strategy policy does
not maximize EV. This trade-off is the central finding of the accompanying
report — see §Diagnostics on the project landing page for the full analysis.

## Limitations

- **Mixed-strategy mean targeting.** The training objective drives the model
  toward the equilibrium policy. Greedy decoding therefore picks the modal
  equilibrium action, which is not always the EV-maximizing response to a
  non-equilibrium opponent. This is a feature for Nash-quality evaluation and
  a constraint for live-play exploitation.
- **Bet-space discretization.** The live-play simulator discretizes bets into
  five canonical sizes; the training distribution covers a wider continuous
  range. Behavior on bet sizes outside the simulator's discretization may not
  reflect what the model would do given a continuous-bet interface.
- **Game format scope.** The model is trained exclusively on heads-up no-limit
  Texas hold'em at 200 BB depth. It has not been evaluated at other stack
  depths, other player counts, or other variants.
- **Teacher rationales on the GTO Wizard slice.** The hand-class audit applies
  only to the solver chain-of-thought source; the GTO Wizard chain-of-thought
  slice has no per-spot hand-class ground truth and is left unaudited. A
  residual teacher-error rate is assumed.

## Responsible use

This model is a research artifact for a coursework project. It is **not**
intended for use in real-money gambling or in any setting where a
mis-prediction could cause financial harm. The Llama-3.1 community license
governs downstream use of the base weights and applies to this fine-tune in
full.

## Citation

```bibtex
@misc{mao2026pokerllama,
  title  = {PokerLlama-4: When Distilling a Nash Policy Beats the Benchmark but Loses the Game},
  author = {Mao, Jevon},
  year   = {2026},
  note   = {Stanford CS 153 Final Project},
  url    = {https://jevonmao.github.io/cs153-project/}
}
```

## Acknowledgements

- **GTO Wizard** for the live solver-decision data interface.
- **Meta** for releasing Llama-3.1-8B-Instruct.
- **Stanford CS 153** course staff for compute access on the H100 cluster.