jevonmao/llama31-8b-poker-mix-v1-step10k

Files

ModelHub XC 36a311e8b5 初始化项目，由ModelHub XC社区提供模型

Model: jevonmao/llama31-8b-poker-mix-v1-step10k
Source: Original Platform

2026-07-28 02:50:09 +08:00

9.8 KiB

Raw Blame History

library_name, license, base_model, datasets, language, pipeline_tag, tags, model-index

library_name

license

base_model

datasets

language

pipeline_tag

PokerLlama-4

PokerLlama-4 is a supervised fine-tune of Llama-3.1-8B-Instruct for heads-up no-limit Texas hold'em at 200 big blinds. It distills the GTO Wizard equilibrium solver's mixed strategies into a chat-template language model and supports both direct action emission and chain-of-thought reasoning at inference time.

It reaches 83.96% top-1 action-type accuracy on a 31,105-decision held-out evaluation split — a new project record and the highest of any 8B model we evaluated, outperforming both open-source poker fine-tunes and zero-shot frontier reasoning models on postflop action prediction.

Project landing page: https://jevonmao.github.io/cs153-project/
Code & evaluation harness: https://github.com/jevonmao/cs153-project
Training corpus: jevonmao/poker-sft-mix-v1

PokerLlama-4: When Distilling a Nash Policy Beats the Benchmark but Loses the Game. Stanford CS 153 Final Project, 2026.

Model details

Field	Value
Base model	`meta-llama/Meta-Llama-3.1-8B-Instruct`
Adapter	LoRA, rank 128, α = 256, dropout 0.05
Target modules	All seven linear projections per transformer block
Trainable parameters	≈ 335M (4.2% of base)
Precision	bfloat16, merged FP16 weights pushed
Optimizer	Paged 8-bit AdamW
LR schedule	Cosine, 2% warmup, peak η = 2 × 10⁻⁴
Epochs	1 over 606,630 records (≈ 19,700 steps)
Effective batch	32 (4 GPU × 8 per-device × 1 accumulation)
Hardware	4 × H100 80 GB
Wall-clock	≈ 6 hours
Eval-loss-converged checkpoint	step 10,000 (eval loss 0.0873)

This repository contains the merged FP16 weights at the converged checkpoint. No LoRA adapter file is included — the merge is in-place, so the model loads exactly like the base via AutoModelForCausalLM.

Intended use

PokerLlama-4 is a research artifact accompanying the project's final report. It is intended for:

Reproducing the project's evaluation results on the held-out HoldemEval-31k split and live-play benchmarks.
Studying GTO-policy distillation into small language models — the trade-off between matching the equilibrium action distribution offline and achieving expected value against opponents who deviate from equilibrium.
Tool-use / function-call experiments in a poker domain — the model emits well-formed preflop_gto tool calls with 99.9% argument correctness.

It is not intended for gambling, real-money play, or any context in which the output is acted on without human oversight.

How to use

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "jevonmao/llama31-8b-poker-mix-v1-step10k"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

SYSTEM_PROMPT = (
    "You are an expert No Limit Texas Hold'em poker assistant. "
    "Read the hand history carefully and decide the next action. "
    "Respond with a single action wrapped in <action></action> tags."
)

USER_PROMPT = """\
Game: HUNL 200BB
Your position: SB
Your hand: As Kh
Pot: 150
Effective stack: 19,900
Action so far: SB posts 50, BB posts 100.

Legal actions: fold, call 100, raise [to] [200..19900]
"""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True
).to(model.device)

with torch.inference_mode():
    out = model.generate(inputs, max_new_tokens=32, do_sample=False)

print(tokenizer.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))
# e.g. <action>raise to 225</action>

For chain-of-thought inference, replace the system prompt's final sentence with the project's CoT mode marker (see eval/ in the source repository).

Greedy decoding is recommended

All reported results use greedy decoding (do_sample=False). Temperature sampling at T = 1.0 recovers the mixed-strategy character of the distilled policy at the cost of action-label accuracy and is appropriate for live play against opponents who themselves play balanced strategies.

Training data

PokerLlama-4 is fine-tuned on jevonmao/poker-sft-mix-v1, a 606,630-record four-way mix of equilibrium-solver decisions and audited chain-of-thought reasoning traces:

Slice	Records	Source	Mode
`A_cot`	248,931	GTO Wizard interaction split	Chain-of-thought
`A_direct`	248,931	GTO Wizard interaction split	Direct action
`B_cot`	54,384	CFR postflop solver, audited	Chain-of-thought
`B_direct`	54,384	CFR postflop solver, audited	Direct action

A teacher-rationale hand-class audit rejects 21.7% of solver-source CoT records (15,054 / 69,438) for categorical hand-class errors before training.

Evaluation

Offline action accuracy (HoldemEval-31k)

Model	n	Parse %	Type accuracy
`meta-llama/Meta-Llama-3.1-8B-Instruct`	31,105	80.84%	28.75%
`YiPz/llama3-8b-pokerbench-sft`	31,105	96.72%	30.63%
PokerLlama-2 (postflop specialist, predecessor)	31,105	100%	55.82%
PokerLlama-1 (predecessor)	31,105	100%	61.62%
PokerLlama-4 (this model)	31,105	100%	83.96%

The +22.3 pt margin over the closest 8B baseline and the +53.3 pt margin over the published PokerBench-8B fine-tune are isolated to the corpus and training recipe — every model shares the Llama-3.1-8B backbone.

Against frontier reasoning models (postflop split, n = 2,044)

A separate predecessor checkpoint (PokerLlama-2, a postflop specialist on the same project line) was evaluated against three frontier reasoning baselines on a 2,044-decision postflop-only split:

Model	Mode	Parse %	Type accuracy
DeepSeek-V4-Pro	Reasoning trace	99.6%	62.9%
GPT-4.1	Zero-shot	98.8%	68.8%
GPT-5.4-pro	Medium reasoning effort	90.7%	68.5%
PokerLlama-2 (8B specialized)	Fine-tune	100%	76.4%

Specialized distillation clears the frontier zero-shot ceiling (≈ 69%) on postflop action prediction at full parse-rate reliability.

Live self-play (rlcard heads-up 200 BB)

Opponent	n hands	bb/100	95% CI
`rule:fold`	4,000	+74.0	[+73, +75]
`rule:allin`	4,000	+448.3	[+277, +620]
`rule:random`	4,000	+166.5	[+63, +270]
`PokerBench-8B`	8,000	−21.4	[−36, −7]
`Llama-3.1-8B-Instruct` (base)	7,085	−39.6	[−52, −27]

Greedy decoding of the distilled policy approximates the modal action of the solver's mixed strategy. Against a balanced opponent this is approximately Nash; against opponents who deviate from equilibrium (PokerBench-8B over-folds, base Llama rarely folds) the modal-of-mixed-strategy policy does not maximize EV. This trade-off is the central finding of the accompanying report — see §Diagnostics on the project landing page for the full analysis.

Limitations

Mixed-strategy mean targeting. The training objective drives the model toward the equilibrium policy. Greedy decoding therefore picks the modal equilibrium action, which is not always the EV-maximizing response to a non-equilibrium opponent. This is a feature for Nash-quality evaluation and a constraint for live-play exploitation.
Bet-space discretization. The live-play simulator discretizes bets into five canonical sizes; the training distribution covers a wider continuous range. Behavior on bet sizes outside the simulator's discretization may not reflect what the model would do given a continuous-bet interface.
Game format scope. The model is trained exclusively on heads-up no-limit Texas hold'em at 200 BB depth. It has not been evaluated at other stack depths, other player counts, or other variants.
Teacher rationales on the GTO Wizard slice. The hand-class audit applies only to the solver chain-of-thought source; the GTO Wizard chain-of-thought slice has no per-spot hand-class ground truth and is left unaudited. A residual teacher-error rate is assumed.

Responsible use

This model is a research artifact for a coursework project. It is not intended for use in real-money gambling or in any setting where a mis-prediction could cause financial harm. The Llama-3.1 community license governs downstream use of the base weights and applies to this fine-tune in full.

Citation

@misc{mao2026pokerllama,
  title  = {PokerLlama-4: When Distilling a Nash Policy Beats the Benchmark but Loses the Game},
  author = {Mao, Jevon},
  year   = {2026},
  note   = {Stanford CS 153 Final Project},
  url    = {https://jevonmao.github.io/cs153-project/}
}

Acknowledgements

GTO Wizard for the live solver-decision data interface.
Meta for releasing Llama-3.1-8B-Instruct.
Stanford CS 153 course staff for compute access on the H100 cluster.

9.8 KiB Raw Blame History Unescape Escape