feat: add failure-aware preflight and Qwen review
This commit is contained in:
58
README.md
58
README.md
@@ -97,9 +97,62 @@ progress are stored in `.modelhub_state/gpu_strategy.json`.
|
||||
|
||||
Five consecutive local failures open a 12-hour GPU/framework circuit breaker.
|
||||
A sub-20% success rate over the latest 20 terminal tasks opens a 6-hour breaker.
|
||||
Platform/infrastructure failures are excluded from long-term compatibility rates
|
||||
and model/profile breakers. Three consecutive platform failures on a GPU/framework
|
||||
instead open a short 30-minute breaker, so a temporary broken runner or lack of an
|
||||
idle card does not permanently poison otherwise successful evidence.
|
||||
Candidate shortages expand the model search window; they never unlock an
|
||||
unvetted GPU or framework.
|
||||
|
||||
Before a candidate reaches the submit queue, failure-informed preflight checks
|
||||
the actual ModelScope repository structure and file sizes. Non-GGUF text
|
||||
frameworks require root-level config, weights, and tokenizer assets. The memory
|
||||
gate recursively totals the entire repository—including duplicate weight formats
|
||||
and nested shards—and applies ModelHub's observed 20% loading overhead. It covers
|
||||
all 14 GPU types currently marked `canVerify=true`; nine capacities come directly
|
||||
from structured ModelHub OOM reports and five from published specifications until
|
||||
ModelHub supplies a stronger observation. If a known GPU's repository file sizes
|
||||
are incomplete, the candidate is deferred rather than guessed. Template context
|
||||
length is also clamped to the model's advertised limit. A newly introduced GPU
|
||||
with no capacity evidence is likewise deferred. Override or extend known
|
||||
capacities with
|
||||
`MODELHUB_GPU_MEMORY_GIB_JSON`, for example
|
||||
`{"New_gpu": 64}`.
|
||||
|
||||
The verified capacities, safe repository-size boundaries, evidence hierarchy,
|
||||
and source links are recorded in
|
||||
`docs/gpu-memory-capacity-2026-08-10.md`.
|
||||
|
||||
Ambiguous custom architectures can optionally be reviewed by a small
|
||||
OpenAI-compatible Qwen model. Qwen is lazy: deterministic rules handle repository
|
||||
layout, model size, context length, known errors, and ordinary quantization cases
|
||||
without an LLM call. Set
|
||||
`MODELHUB_LLM_CLASSIFIER_ENDPOINT` to the full chat-completions URL and
|
||||
`MODELHUB_LLM_CLASSIFIER_MODEL`; set `MODELHUB_LLM_CLASSIFIER_API_KEY` only when
|
||||
the endpoint requires it. The default deny threshold is 0.85 and can be changed
|
||||
with `MODELHUB_LLM_CLASSIFIER_MIN_DENY_CONFIDENCE`. For Alibaba Model Studio,
|
||||
the aliases are `MODELHUB_QWEN_ENDPOINT`, `MODELHUB_QWEN_MODEL`, and
|
||||
`MODELHUB_QWEN_API_KEY` (or `DASHSCOPE_API_KEY`); endpoint omission uses the
|
||||
DashScope OpenAI-compatible URL. A root `.env` entry named `dashscope` is also
|
||||
recognized directly, and the default model is `qwen3.7-flash`. Calls default to
|
||||
one concurrent request and 20
|
||||
requests per rolling hour, configurable with `MODELHUB_LLM_MAX_CONCURRENT_REQUESTS`
|
||||
and `MODELHUB_LLM_MAX_CALLS_PER_HOUR`. The LLM may only veto an
|
||||
ambiguous candidate: it cannot bypass deterministic checks, introduce a new
|
||||
framework, or override public success-evidence gates. Results are cached under
|
||||
`.modelhub_state/llm_classifications.json`.
|
||||
|
||||
Outcome synchronization downloads a bounded set of failure archives for
|
||||
submissions created by this worker (at most 40 per sync, four workers, three
|
||||
download attempts). Deterministic signatures classify memory, repository layout,
|
||||
context-length, storage, and platform faults first. Only unresolved runtime errors
|
||||
are sent to the optional LLM; a semantic result is promoted only at confidence
|
||||
0.80 or higher. Signed log URLs remain in the ignored local outcome store and are
|
||||
removed after classification.
|
||||
|
||||
The 12-account failure study and routing rationale are recorded in
|
||||
`docs/failure-analysis-2026-08-10.md`.
|
||||
|
||||
ModelScope HTTP 429 responses use exponential backoff and `Retry-After`. Successful
|
||||
pages remain cached, so a later cycle retries the failed page instead of restarting
|
||||
the whole pagination scan.
|
||||
@@ -149,6 +202,11 @@ Version `2026.08.05.1` removes self-funded GPU exploration, switches accepted
|
||||
traffic to 70/30 long-term/recent exploitation, raises the public framework gate
|
||||
to 300 samples, makes success dominate queue pressure, and adds recent local
|
||||
GPU/framework circuit breakers.
|
||||
Version `2026.08.10.2` adds evidence-backed sizing for every currently verifiable
|
||||
GPU, recursive repository-size checks, deterministic failure-aware preflight,
|
||||
and rate-limited lazy Qwen review for unresolved semantic cases.
|
||||
Version `2026.08.10.3` selects `qwen3.7-flash` by default and recognizes the
|
||||
repository root `.env` key named `dashscope` without logging its value.
|
||||
|
||||
## Deploy
|
||||
|
||||
|
||||
Reference in New Issue
Block a user