license, language, base_model, tags, pipeline_tag
| license | language | base_model | tags | pipeline_tag | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| apache-2.0 |
|
|
|
text-generation |
SWE-ZERO-10K-Qwen3-1.7B-Base
Qwen3-1.7B-Base SFT on a 10K random sample of SWE-ZERO trajectories (right-truncated to 8K tokens), evaluated on SWE-bench Verified.
Eval result
pass@1 = 7/100 = 7% on the 100-task SWE-bench Verified slice (latest trial per task).
For full eval details + per-task trajectories, see the eval dataset: AlienKevin/SWE-ZERO-10K-Qwen3-1.7B-Base-eval.
Training
- Base:
Qwen/Qwen3-1.7B-Base - SFT data: 10K random sample from AlienKevin/SWE-ZERO-12M-trajectories @ 2f328e1d, right-truncated to 8K tokens
- Model arch:
max_seq_len=32768(Llama 3 RoPE scaling from 8192) — data truncated to 8K but RoPE precomputed to 32K so positions past 8K stay in distribution - TPU: v5p-8 (~1.5h training)
- Optimizer: AdamW (β1=0.9, β2=0.95, ε=1e-8), lr=2e-5, weight_decay=0.1, max_grad_norm=30, cosine schedule, warmup=0.03, min_lr_ratio=0.1
- Batch: 8 global, 1249 steps
- Tracking: marin#5611
Critical detail: eos_token_id
The HF config has eos_token_id: [151643, 151645] so vLLM stops at both <|endoftext|> AND <|im_end|> (Qwen3 chat-template turn boundary). The default base model only includes 151643; SFT-trained models that aren't patched will keep generating past the natural turn end until max_tokens.
Inference
from vllm import LLM, SamplingParams
llm = LLM(model="AlienKevin/SWE-ZERO-10K-Qwen3-1.7B-Base", max_model_len=32768)
params = SamplingParams(temperature=1.0, max_tokens=4096)
# Use the Qwen3 chat template; the eos_token_id list will be picked up automatically.
For the mini-swe-agent eval harness used in our results, see the eval dataset for the harbor adapter + agent config.