---
license: other
base_model: Qwen/Qwen3-8B-Base
tags:
- code
- reinforcement-learning
- pkpo
- livecodebench
---

# Qwen3-8B Code PKPO

This repo contains a Qwen/Qwen3-8B-Base derivative trained for a small agentic
coding experiment using the shared tool path in `agent_core.py` and
`shipped_tool.py`.

## Method

- Base: `Qwen/Qwen3-8B-Base`.
- Prompt/template: custom `<think>...</think><answer>...</answer>` template saved
  in the tokenizer. The generation prompt ends with `Assistant: <think>`.
- Tool protocol: no system role; instructions are merged into the first user
  message; strict user/assistant alternation; plain-text `Tool type` and
  `Tool query` calls.
- Training data: `deepmind/code_contests` train split only, filtered to old
  stdin/stdout problems. The LiveCodeBench eval subset is not used for training.
- Reward: binary hidden-test pass/fail.
- PKPO: `sloo_minus_one` from the paper for `k >= 2`; centered `k=1` rewards for
  the first and final stages. No GRPO-style reward normalization is applied.
- Schedule actually run: `[1, 8, 1] (shipped: stage1_group3)`.

The run was intentionally small to fit the free-credit budget and deadline.
Results should be treated as a reproducible experiment, not a leaderboard model.

## Results

Evaluation uses `livecodebench/code_generation_lite` `v6`, a fixed subset saved
at `eval/eval_subset.json`, temperature 1.0, and the same one-turn tool path used
for training.

| model | pass@1 estimate |
|---|---:|
| base before training | 0.1111 |
| final merged model | 0.0556 |

Raw files:

- `eval/baseline_results.json`
- `eval/final_results.json`
- `eval/eval_subset.json`

## Usage

Serve with vLLM:

```bash
vllm serve bk1dr/qwen3-8b-code-pkpo --trust-remote-code --max-model-len 8192
```

Run the shipped tool:

```bash
python shipped_tool.py --base-url http://127.0.0.1:8000/v1 --model bk1dr/qwen3-8b-code-pkpo --max-turns 1 --cp < problem.txt
```

## Run Notes

Run pkpo_20260709T184830Z: full PKPO schedule k=1->8->1 with a LoRA checkpoint after every group. The shipped weights are checkpoint 'stage1_group3', selected by validation on the fixed eval subset (per-checkpoint pass@1: {"after_sft": 0.027777777777777776, "stage1_group3": 0.05555555555555555, "stage2_group1": 0.027777777777777776}). The full-schedule endpoint regressed on the subset (see eval/full_schedule_endpoint_results.json); intermediate checkpoint selection is part of the documented training procedure. Selection+merge elapsed 13.3 min on one H100.