初始化项目,由ModelHub XC社区提供模型
Model: bk1dr/qwen3-8b-code-pkpo Source: Original Platform
This commit is contained in:
68
README.md
Normal file
68
README.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
license: other
|
||||
base_model: Qwen/Qwen3-8B-Base
|
||||
tags:
|
||||
- code
|
||||
- reinforcement-learning
|
||||
- pkpo
|
||||
- livecodebench
|
||||
---
|
||||
|
||||
# Qwen3-8B Code PKPO
|
||||
|
||||
This repo contains a Qwen/Qwen3-8B-Base derivative trained for a small agentic
|
||||
coding experiment using the shared tool path in `agent_core.py` and
|
||||
`shipped_tool.py`.
|
||||
|
||||
## Method
|
||||
|
||||
- Base: `Qwen/Qwen3-8B-Base`.
|
||||
- Prompt/template: custom `<think>...</think><answer>...</answer>` template saved
|
||||
in the tokenizer. The generation prompt ends with `Assistant: <think>`.
|
||||
- Tool protocol: no system role; instructions are merged into the first user
|
||||
message; strict user/assistant alternation; plain-text `Tool type` and
|
||||
`Tool query` calls.
|
||||
- Training data: `deepmind/code_contests` train split only, filtered to old
|
||||
stdin/stdout problems. The LiveCodeBench eval subset is not used for training.
|
||||
- Reward: binary hidden-test pass/fail.
|
||||
- PKPO: `sloo_minus_one` from the paper for `k >= 2`; centered `k=1` rewards for
|
||||
the first and final stages. No GRPO-style reward normalization is applied.
|
||||
- Schedule actually run: `[1, 8, 1] (shipped: stage1_group3)`.
|
||||
|
||||
The run was intentionally small to fit the free-credit budget and deadline.
|
||||
Results should be treated as a reproducible experiment, not a leaderboard model.
|
||||
|
||||
## Results
|
||||
|
||||
Evaluation uses `livecodebench/code_generation_lite` `v6`, a fixed subset saved
|
||||
at `eval/eval_subset.json`, temperature 1.0, and the same one-turn tool path used
|
||||
for training.
|
||||
|
||||
| model | pass@1 estimate |
|
||||
|---|---:|
|
||||
| base before training | 0.1111 |
|
||||
| final merged model | 0.0556 |
|
||||
|
||||
Raw files:
|
||||
|
||||
- `eval/baseline_results.json`
|
||||
- `eval/final_results.json`
|
||||
- `eval/eval_subset.json`
|
||||
|
||||
## Usage
|
||||
|
||||
Serve with vLLM:
|
||||
|
||||
```bash
|
||||
vllm serve bk1dr/qwen3-8b-code-pkpo --trust-remote-code --max-model-len 8192
|
||||
```
|
||||
|
||||
Run the shipped tool:
|
||||
|
||||
```bash
|
||||
python shipped_tool.py --base-url http://127.0.0.1:8000/v1 --model bk1dr/qwen3-8b-code-pkpo --max-turns 1 --cp < problem.txt
|
||||
```
|
||||
|
||||
## Run Notes
|
||||
|
||||
Run pkpo_20260709T184830Z: full PKPO schedule k=1->8->1 with a LoRA checkpoint after every group. The shipped weights are checkpoint 'stage1_group3', selected by validation on the fixed eval subset (per-checkpoint pass@1: {"after_sft": 0.027777777777777776, "stage1_group3": 0.05555555555555555, "stage2_group1": 0.027777777777777776}). The full-schedule endpoint regressed on the subset (see eval/full_schedule_endpoint_results.json); intermediate checkpoint selection is part of the documented training procedure. Selection+merge elapsed 13.3 min on one H100.
|
||||
Reference in New Issue
Block a user