feat: dynamically clean incompatible architectures
This commit is contained in:
@@ -16,7 +16,7 @@ It currently supports:
|
||||
- `main.py`: core discovery, scoring, dedup, and submission
|
||||
- `daily_runner.py`: daily wave orchestration
|
||||
- `poll_runner.py`: long-running queue refiller
|
||||
- `queue_cleanup.py`: fail-closed cleanup for active tasks that are certain to exceed GPU memory
|
||||
- `queue_cleanup.py`: fail-closed cleanup for certain OOM, architecture, and age policies
|
||||
- `runner_common.py`: shared token / key file loading
|
||||
- `hf_discovery.py`: ModelScope model discovery and inspection (keeps the legacy module name)
|
||||
- `modelhub_client.py`: ModelHub API client and token-pool routing
|
||||
@@ -96,9 +96,13 @@ bash run_poll.sh --dry-run
|
||||
sub-20% rate over the latest 20 terminal tasks pauses it for 6 hours.
|
||||
- An explicit "framework does not support this model/architecture" failure learns
|
||||
a 30-day GPU + framework + task + architecture block. Architecture identity
|
||||
comes from candidate `config.json` (`architectures`, with `model_type` only as
|
||||
fallback), never from repository names. A newer success clears the block, and
|
||||
comes from candidate `config.json` plus exact unsupported `model_type` or
|
||||
`architectures` strings in the runtime log, never from repository names. A newer success clears the block, and
|
||||
generic unsupported backend/operator messages cannot create one.
|
||||
- Blacklist additions are persisted and detected every three poll cycles. A new
|
||||
rule immediately launches a lightweight architecture-only queue scan. Exact
|
||||
matching waiting tasks are stopped after two state checks; running tasks and
|
||||
tasks without local framework/task metadata are protected.
|
||||
- A strategy generation lasts exactly 200 platform-accepted submissions. Rejected API calls and
|
||||
duplicates do not advance it. The next cycle refreshes platform history before submitting again.
|
||||
- Strategy state is stored in `.modelhub_state/gpu_strategy.json`; a generation never recalculates
|
||||
@@ -227,6 +231,7 @@ Persistent local scheduler state is written under `.modelhub_state/`:
|
||||
- `account_capacity.json`: learned per-account active-task limits
|
||||
- `submission_exclusions.jsonl`: non-retryable model/GPU uniqueness rejections
|
||||
- `queue_cleanup_latest.json`: latest active-task sizing evidence and cancellation result
|
||||
- `architecture_compatibility_blacklist.json`: current dynamic compatibility blocks and evidence
|
||||
|
||||
## Verification
|
||||
|
||||
|
||||
Reference in New Issue
Block a user