fix: restore accuracy-first route admission
This commit is contained in:
@@ -79,21 +79,25 @@ bash run_poll.sh --dry-run
|
||||
## Behavior
|
||||
|
||||
- The runner auto-discovers all safe GPU/template combinations from the public submit catalog.
|
||||
- Automatic GPU selection uses exact 70/30 accepted-task scheduling: long-term
|
||||
Wilson-ranked top 3 GPUs and the top GPUs from the latest 1,000 terminal tasks.
|
||||
There is no all-GPU exploration category.
|
||||
- Within each category, weighted-fair scheduling uses estimated queue backlog hours,
|
||||
- Automatic GPU selection uses one success-first score built from qualified
|
||||
local cohorts and ModelHub public evidence. There is no random or fixed-quota
|
||||
exploration category.
|
||||
- Within the qualified success band, scheduling uses estimated queue backlog hours,
|
||||
recent public throughput/success, machine availability, and worker concurrency.
|
||||
Unavailable or stalled GPU pools are circuit-broken instead of continuing to absorb work.
|
||||
- Compatible frameworks are ranked by ModelHub public aggregate success statistics
|
||||
plus capped local GPU+framework evidence, with a 300-sample public minimum and
|
||||
Wilson confidence bounds. Missing or undersized public evidence receives zero
|
||||
traffic rather than falling back to exploration.
|
||||
a 20% Wilson lower-bound floor. GPU pools normally require a 15% floor.
|
||||
Missing or undersized public evidence receives zero traffic rather than
|
||||
falling back to exploration; a low-sample GPU requires a recent success and a
|
||||
framework that already passes the stricter framework gate.
|
||||
- New frameworks are discovered from the live catalog but get no novelty bonus.
|
||||
They are eligible only with a complete official build config that passes local
|
||||
validation and a confidence score at least 10% above the best incumbent.
|
||||
- Five consecutive local failures pause a GPU/framework pair for 12 hours; a
|
||||
sub-20% rate over the latest 20 terminal tasks pauses it for 6 hours.
|
||||
- Five consecutive local failures pause a GPU/framework pair for 12 hours. Five
|
||||
or more recent terminal outcomes below 10% success also pause it for 12 hours,
|
||||
including ambiguous runtime failures. A sub-20% attributable rate over the
|
||||
latest 20 terminal tasks pauses it for 6 hours.
|
||||
- An explicit "framework does not support this model/architecture" failure learns
|
||||
a 30-day GPU + framework + task + architecture block. Architecture identity
|
||||
comes from candidate `config.json` plus exact unsupported `model_type` or
|
||||
@@ -114,20 +118,22 @@ bash run_poll.sh --dry-run
|
||||
duplicates do not advance it. The next cycle refreshes platform history before submitting again.
|
||||
- Strategy state is stored in `.modelhub_state/gpu_strategy.json`; a generation never recalculates
|
||||
during candidate submission.
|
||||
- Candidate discovery starts with the configured recent window, then automatically expands to
|
||||
7 days, 30 days, and the first 3,000 raw ModelScope rows ordered by update time. If the
|
||||
candidate pool is still short, it rotates through up to 200 download-ranked rows per cycle
|
||||
as an older-model fallback.
|
||||
- Candidate discovery starts with the configured recent window, then
|
||||
automatically expands to 7 days, 30 days, and the first 3,000 raw ModelScope
|
||||
rows ordered by update time. Download popularity is not treated as ModelHub
|
||||
compatibility evidence.
|
||||
- Model verification responses are cached across poll cycles for 15 minutes. Local model/GPU
|
||||
failures cool down after 24 hours instead of remaining permanently blocked.
|
||||
- Community deduplication is model/GPU-specific: another GPU's adaptation does not block the
|
||||
current GPU. `未验证` with no records is eligible. `验证失败` is eligible only with a
|
||||
different publicly vetted framework after all recorded attempts explicitly failed;
|
||||
`已验证`, `验证中`, and ambiguous records remain blocked. Every actual submission
|
||||
performs a fresh uncached check for its exact GPU and framework.
|
||||
- Community deduplication is model/GPU-specific: another GPU's adaptation does
|
||||
not block the current GPU. Only `未验证` with no records is eligible;
|
||||
`验证失败`, `已验证`, `验证中`, and ambiguous records remain blocked. Every actual
|
||||
submission performs a fresh uncached check for its exact GPU.
|
||||
- If the community lookup is unavailable, submission is deferred. A platform model-uniqueness
|
||||
rejection permanently excludes only that model/GPU combination from future local retries.
|
||||
- The same framework is not retried for a model/GPU after a recorded failure.
|
||||
- A model may use at most six GPU routes across poll cycles, and at most two
|
||||
locally submitted routes for that model may await feedback at once. The first
|
||||
six ranked GPUs remain preferred; lower-ranked eligible GPUs are inspected in
|
||||
order only when that tranche yields no safe candidate.
|
||||
- Multiple ModelHub tokens are pooled and used to route submissions to the account with available async capacity.
|
||||
- Concurrent submissions reserve account slots locally, and an account-capacity race automatically falls through to another account.
|
||||
- Concurrent local processes claim model/GPU pairs in `.modelhub_state/submission_claims.jsonl`; shared ledger, history, and outcome files use process locks and atomic replacement.
|
||||
|
||||
Reference in New Issue
Block a user