初始化项目,由ModelHub XC社区提供模型
Model: laion/explore-tis-untrunc-45-8B Source: Original Platform
This commit is contained in:
35
README.md
Normal file
35
README.md
Normal file
@@ -0,0 +1,35 @@
|
||||
---
|
||||
base_model: laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink
|
||||
tags:
|
||||
- rl
|
||||
- skyrl
|
||||
- agentic
|
||||
- terminal-bench
|
||||
- tis
|
||||
---
|
||||
|
||||
# explore-tis-untrunc (global_step 45, 8B)
|
||||
|
||||
Agentic RL checkpoint (SkyRL) finetuned from
|
||||
`laion/GLM-4_7-swesmith-sandboxes-with_tests-oracle_verified_120s-maxeps-131k-fixthink` (Qwen3-8B SFT base).
|
||||
|
||||
## Run config
|
||||
- **Algorithm:** RLOO-n advantage, `seq_mean_token_sum_norm_global` loss reduction, no KL loss.
|
||||
- **TIS:** enabled (`tis_imp_ratio_cap=2.0`) — truncated importance sampling correcting the vLLM↔FSDP bf16 logprob gap.
|
||||
- **Sampling (untruncated exploration variant):** the distinguishing feature of this ablation is untruncated sampling during exploration.
|
||||
- **Dataset:** `DCAgent/exp_rpt_pymethods2test-large`.
|
||||
- **Harness:** terminus-2 / Harbor agentic terminal-bench, n_samples_per_prompt=8.
|
||||
|
||||
## Checkpoint selection
|
||||
This is **global_step 45**, selected by trailing-5 EMA (alpha=1/3) of `reward/avg_raw_reward`,
|
||||
**restricted to the genuine training region (steps <= 78)**. Steps 79+ exhibited an
|
||||
artifact reward jump (single-block greedy / eval-checkpoint passes, not a learning event)
|
||||
and an eternal-retry tail, so they were excluded from candidate selection.
|
||||
At step 45: raw reward ~0.518, trailing-5 EMA ~0.495 (highest among saved exports in-region).
|
||||
|
||||
## Training Traces
|
||||
Rollout traces for this run: https://huggingface.co/datasets/penfever/explore-tis-untrunc
|
||||
|
||||
## Training logs
|
||||
Parsed SkyRL metrics, vLLM metrics, and raw trainer logs are in the `training_logs/` folder of this repo.
|
||||
The serialized launch config is `rl_config.yaml`.
|
||||
Reference in New Issue
Block a user