feat: add adaptive GPU scheduling
This commit is contained in:
25
README.md
25
README.md
@@ -31,6 +31,8 @@ Optional tuning:
|
||||
- `MODELHUB_AGENT_POST_CYCLE_COOLDOWN_SECONDS` default `2`
|
||||
- `MODELHUB_AGENT_MAX_SUBMITS_PER_RUN` default `0` (fill all currently available slots)
|
||||
- `MODELHUB_AGENT_ACTIVE_TASK_CAP` default `100` per account
|
||||
- `MODELHUB_CAPACITY_PROBE_INTERVAL_CYCLES` default `3`
|
||||
- `MODELHUB_CAPACITY_STATE_PATH` default `.modelhub_state/account_capacity.json`
|
||||
- `MODELHUB_AGENT_ACTIVE_COUNTS_TTL_SECONDS` default `15`
|
||||
- `MODELHUB_AGENT_RESERVATION_TTL_SECONDS` default `120`
|
||||
- `MODELHUB_AGENT_INSTANCE_ID` optional stable worker identity used to spread concurrent agents across accounts and candidates
|
||||
@@ -39,6 +41,27 @@ Optional tuning:
|
||||
- `MODELHUB_AGENT_MIN_DOWNLOADS`
|
||||
- `MODELHUB_AGENT_GPUS`
|
||||
- `MODELHUB_AGENT_EXTRA_ARGS`
|
||||
- `MODELHUB_GPU_STRATEGY_STATE_PATH` default `.modelhub_state/gpu_strategy.json`
|
||||
- `MODELSCOPE_PAGE_INTERVAL_SECONDS` default `0.25`
|
||||
- `MODELSCOPE_PAGE_CACHE_TTL_SECONDS` default `900`
|
||||
|
||||
## Adaptive GPU Strategy
|
||||
|
||||
When no explicit GPU override is supplied, the worker uses a local 50/30/20
|
||||
strategy generation:
|
||||
|
||||
- 50%: the three long-term GPUs with the best Wilson lower confidence score and at least 100 terminal samples
|
||||
- 30%: round-robin exploration across every currently supported GPU
|
||||
- 20%: the best GPU among the latest 1,000 terminal tasks
|
||||
|
||||
Only platform-accepted tasks count. After exactly 200 accepted tasks, the next
|
||||
poll cycle reloads all account history, generates a new immutable strategy snapshot,
|
||||
and resets the generation counters to 100/60/40 targets. The active snapshot and
|
||||
progress are stored in `.modelhub_state/gpu_strategy.json`.
|
||||
|
||||
ModelScope HTTP 429 responses use exponential backoff and `Retry-After`. Successful
|
||||
pages remain cached, so a later cycle retries the failed page instead of restarting
|
||||
the whole pagination scan.
|
||||
|
||||
## Concurrent Agents
|
||||
|
||||
@@ -56,7 +79,7 @@ authority for account capacity and model/GPU uniqueness.
|
||||
If the platform reports that a model/GPU is already being validated, the claim
|
||||
is retained and the runner immediately draws replacement candidates from the
|
||||
same scan instead of retrying the duplicate every cycle. Startup logs and the
|
||||
health response expose `agent_version`; version `2026.08.02.2` or newer includes
|
||||
health response expose `agent_version`; version `2026.08.02.3` or newer includes
|
||||
this behavior.
|
||||
|
||||
## Deploy
|
||||
|
||||
Reference in New Issue
Block a user