feat: dynamically clean incompatible architectures

This commit is contained in:
CoolBoy
2026-08-12 08:19:53 +08:00
parent 615bcad124
commit 7ec875563e
12 changed files with 979 additions and 132 deletions

View File

@@ -16,7 +16,7 @@ It currently supports:
- `main.py`: core discovery, scoring, dedup, and submission
- `daily_runner.py`: daily wave orchestration
- `poll_runner.py`: long-running queue refiller
- `queue_cleanup.py`: fail-closed cleanup for active tasks that are certain to exceed GPU memory
- `queue_cleanup.py`: fail-closed cleanup for certain OOM, architecture, and age policies
- `runner_common.py`: shared token / key file loading
- `hf_discovery.py`: ModelScope model discovery and inspection (keeps the legacy module name)
- `modelhub_client.py`: ModelHub API client and token-pool routing
@@ -96,9 +96,13 @@ bash run_poll.sh --dry-run
sub-20% rate over the latest 20 terminal tasks pauses it for 6 hours.
- An explicit "framework does not support this model/architecture" failure learns
a 30-day GPU + framework + task + architecture block. Architecture identity
comes from candidate `config.json` (`architectures`, with `model_type` only as
fallback), never from repository names. A newer success clears the block, and
comes from candidate `config.json` plus exact unsupported `model_type` or
`architectures` strings in the runtime log, never from repository names. A newer success clears the block, and
generic unsupported backend/operator messages cannot create one.
- Blacklist additions are persisted and detected every three poll cycles. A new
rule immediately launches a lightweight architecture-only queue scan. Exact
matching waiting tasks are stopped after two state checks; running tasks and
tasks without local framework/task metadata are protected.
- A strategy generation lasts exactly 200 platform-accepted submissions. Rejected API calls and
duplicates do not advance it. The next cycle refreshes platform history before submitting again.
- Strategy state is stored in `.modelhub_state/gpu_strategy.json`; a generation never recalculates
@@ -227,6 +231,7 @@ Persistent local scheduler state is written under `.modelhub_state/`:
- `account_capacity.json`: learned per-account active-task limits
- `submission_exclusions.jsonl`: non-retryable model/GPU uniqueness rejections
- `queue_cleanup_latest.json`: latest active-task sizing evidence and cancellation result
- `architecture_compatibility_blacklist.json`: current dynamic compatibility blocks and evidence
## Verification