fix: restore accuracy-first route admission

This commit is contained in:
CoolBoy
2026-09-22 15:26:42 +08:00
parent 4afce879dc
commit 6154dc738f
11 changed files with 343 additions and 97 deletions

View File

@@ -79,21 +79,25 @@ bash run_poll.sh --dry-run
## Behavior
- The runner auto-discovers all safe GPU/template combinations from the public submit catalog.
- Automatic GPU selection uses exact 70/30 accepted-task scheduling: long-term
Wilson-ranked top 3 GPUs and the top GPUs from the latest 1,000 terminal tasks.
There is no all-GPU exploration category.
- Within each category, weighted-fair scheduling uses estimated queue backlog hours,
- Automatic GPU selection uses one success-first score built from qualified
local cohorts and ModelHub public evidence. There is no random or fixed-quota
exploration category.
- Within the qualified success band, scheduling uses estimated queue backlog hours,
recent public throughput/success, machine availability, and worker concurrency.
Unavailable or stalled GPU pools are circuit-broken instead of continuing to absorb work.
- Compatible frameworks are ranked by ModelHub public aggregate success statistics
plus capped local GPU+framework evidence, with a 300-sample public minimum and
Wilson confidence bounds. Missing or undersized public evidence receives zero
traffic rather than falling back to exploration.
a 20% Wilson lower-bound floor. GPU pools normally require a 15% floor.
Missing or undersized public evidence receives zero traffic rather than
falling back to exploration; a low-sample GPU requires a recent success and a
framework that already passes the stricter framework gate.
- New frameworks are discovered from the live catalog but get no novelty bonus.
They are eligible only with a complete official build config that passes local
validation and a confidence score at least 10% above the best incumbent.
- Five consecutive local failures pause a GPU/framework pair for 12 hours; a
sub-20% rate over the latest 20 terminal tasks pauses it for 6 hours.
- Five consecutive local failures pause a GPU/framework pair for 12 hours. Five
or more recent terminal outcomes below 10% success also pause it for 12 hours,
including ambiguous runtime failures. A sub-20% attributable rate over the
latest 20 terminal tasks pauses it for 6 hours.
- An explicit "framework does not support this model/architecture" failure learns
a 30-day GPU + framework + task + architecture block. Architecture identity
comes from candidate `config.json` plus exact unsupported `model_type` or
@@ -114,20 +118,22 @@ bash run_poll.sh --dry-run
duplicates do not advance it. The next cycle refreshes platform history before submitting again.
- Strategy state is stored in `.modelhub_state/gpu_strategy.json`; a generation never recalculates
during candidate submission.
- Candidate discovery starts with the configured recent window, then automatically expands to
7 days, 30 days, and the first 3,000 raw ModelScope rows ordered by update time. If the
candidate pool is still short, it rotates through up to 200 download-ranked rows per cycle
as an older-model fallback.
- Candidate discovery starts with the configured recent window, then
automatically expands to 7 days, 30 days, and the first 3,000 raw ModelScope
rows ordered by update time. Download popularity is not treated as ModelHub
compatibility evidence.
- Model verification responses are cached across poll cycles for 15 minutes. Local model/GPU
failures cool down after 24 hours instead of remaining permanently blocked.
- Community deduplication is model/GPU-specific: another GPU's adaptation does not block the
current GPU. `未验证` with no records is eligible. `验证失败` is eligible only with a
different publicly vetted framework after all recorded attempts explicitly failed;
`已验证`, `验证中`, and ambiguous records remain blocked. Every actual submission
performs a fresh uncached check for its exact GPU and framework.
- Community deduplication is model/GPU-specific: another GPU's adaptation does
not block the current GPU. Only `未验证` with no records is eligible;
`验证失败`, `已验证`, `验证中`, and ambiguous records remain blocked. Every actual
submission performs a fresh uncached check for its exact GPU.
- If the community lookup is unavailable, submission is deferred. A platform model-uniqueness
rejection permanently excludes only that model/GPU combination from future local retries.
- The same framework is not retried for a model/GPU after a recorded failure.
- A model may use at most six GPU routes across poll cycles, and at most two
locally submitted routes for that model may await feedback at once. The first
six ranked GPUs remain preferred; lower-ranked eligible GPUs are inspected in
order only when that tranche yields no safe candidate.
- Multiple ModelHub tokens are pooled and used to route submissions to the account with available async capacity.
- Concurrent submissions reserve account slots locally, and an account-capacity race automatically falls through to another account.
- Concurrent local processes claim model/GPU pairs in `.modelhub_state/submission_claims.jsonl`; shared ledger, history, and outcome files use process locks and atomic replacement.