CoolBoy/submmit
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

feat: prioritize proven GPU framework combinations

Browse Source
This commit is contained in:
CoolBoy
2026-08-05 18:21:59 +08:00
parent 882479e43e
commit 5e47d9e695
11 changed files with 431 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
modelhub_submmit_api/README.md
View File

@@ -8,7 +8,7 @@ It currently supports:
- continuous queue refill via `run_poll.sh`
- multiple ModelHub tokens read from `KEY.md` and `KEYS.md`
- automatic task/framework/template selection across the supported GPU catalog
- adaptive long-term/exploration/recent GPU scheduling with a persistent local snapshot
- adaptive long-term/recent GPU exploitation with a persistent local snapshot
- live queue/throughput-aware GPU weighting and confidence-ranked framework selection
## Layout
@@ -74,19 +74,21 @@ bash run_poll.sh --dry-run
## Behavior
- The runner auto-discovers all safe GPU/template combinations from the public submit catalog.
- Automatic GPU selection uses exact 50/30/20 accepted-task scheduling: long-term
Wilson-ranked top 3 GPUs, all supported GPUs, and the top GPUs from the latest
1,000 terminal tasks.
- Automatic GPU selection uses exact 70/30 accepted-task scheduling: long-term
Wilson-ranked top 3 GPUs and the top GPUs from the latest 1,000 terminal tasks.
There is no all-GPU exploration category.
- Within each category, weighted-fair scheduling uses estimated queue backlog hours,
recent public throughput/success, machine availability, and worker concurrency.
Unavailable or stalled GPU pools are circuit-broken instead of continuing to absorb work.
- Compatible frameworks are ranked by ModelHub public aggregate success statistics
plus capped local GPU+framework evidence, with a 100-sample public minimum and
Wilson confidence bounds. The legacy safe order is retained whenever evidence
is missing or too small.
plus capped local GPU+framework evidence, with a 300-sample public minimum and
Wilson confidence bounds. Missing or undersized public evidence receives zero
traffic rather than falling back to exploration.
- New frameworks are discovered from the live catalog but get no novelty bonus.
They are eligible only with a complete official build config that passes local
validation; cached local templates remain the fallback if live config sync fails.
validation and a confidence score at least 10% above the best incumbent.
- Five consecutive local failures pause a GPU/framework pair for 12 hours; a
sub-20% rate over the latest 20 terminal tasks pauses it for 6 hours.
- A strategy generation lasts exactly 200 platform-accepted submissions. Rejected API calls and
duplicates do not advance it. The next cycle refreshes platform history before submitting again.
- Strategy state is stored in `.modelhub_state/gpu_strategy.json`; a generation never recalculates
@@ -163,7 +165,7 @@ Each run typically includes:
Persistent local scheduler state is written under `.modelhub_state/`:
- `gpu_strategy.json`: GPU ranks, generation progress, and 50/30/20 accepted counters
- `gpu_strategy.json`: GPU ranks, generation progress, and 70/30 accepted counters
- `market_intelligence.json`: cached public queue, throughput, health, and framework statistics
- `account_capacity.json`: learned per-account active-task limits
- `submission_exclusions.jsonl`: non-retryable model/GPU uniqueness rejections