fix: refill from expanded candidate history
This commit is contained in:
19
README.md
19
README.md
@@ -29,7 +29,8 @@ Optional tuning:
|
||||
- `MODELHUB_AGENT_POLL_INTERVAL_SECONDS` default `15`
|
||||
- `MODELHUB_AGENT_IDLE_INTERVAL_SECONDS` default `60`
|
||||
- `MODELHUB_AGENT_POST_CYCLE_COOLDOWN_SECONDS` default `2`
|
||||
- `MODELHUB_AGENT_MAX_SUBMITS_PER_RUN` default `0` (fill all currently available slots)
|
||||
- The hosted entrypoint always uses `--max-submits-per-run 0` so stale deployment
|
||||
settings cannot restrict a refill cycle to five submissions.
|
||||
- `MODELHUB_AGENT_ACTIVE_TASK_CAP` default `100` per account
|
||||
- `MODELHUB_CAPACITY_PROBE_INTERVAL_CYCLES` default `3`
|
||||
- `MODELHUB_CAPACITY_STATE_PATH` default `.modelhub_state/account_capacity.json`
|
||||
@@ -44,6 +45,7 @@ Optional tuning:
|
||||
- `MODELHUB_GPU_STRATEGY_STATE_PATH` default `.modelhub_state/gpu_strategy.json`
|
||||
- `MODELSCOPE_PAGE_INTERVAL_SECONDS` default `0.25`
|
||||
- `MODELSCOPE_PAGE_CACHE_TTL_SECONDS` default `900`
|
||||
- `MODELHUB_AGENT_VERIFY_CACHE_TTL_SECONDS` default `900`
|
||||
|
||||
## Adaptive GPU Strategy
|
||||
|
||||
@@ -63,6 +65,18 @@ ModelScope HTTP 429 responses use exponential backoff and `Retry-After`. Success
|
||||
pages remain cached, so a later cycle retries the failed page instead of restarting
|
||||
the whole pagination scan.
|
||||
|
||||
## Adaptive Candidate Discovery
|
||||
|
||||
The configured recent window remains the fast path. If it contains no usable
|
||||
model/GPU combinations, the same run progressively expands discovery to the last
|
||||
7 days, the last 30 days, and finally older history (up to 3,000 models). Scanning
|
||||
stops as soon as enough replacement candidates have been found.
|
||||
|
||||
Model verification results are reused for 15 minutes across poll cycles, and a
|
||||
locally failed model/GPU pair cools down for 24 hours instead of being excluded
|
||||
forever. The `[scan]` lines show every expansion stage, while `[daily] wave_done`
|
||||
includes `skip_reasons` so an empty candidate pool is directly diagnosable.
|
||||
|
||||
## Concurrent Agents
|
||||
|
||||
The token pool keeps a local reservation for every in-flight submission, so a
|
||||
@@ -80,7 +94,8 @@ If the platform reports that a model/GPU is already being validated, the claim
|
||||
is retained and the runner immediately draws replacement candidates from the
|
||||
same scan instead of retrying the duplicate every cycle. Startup logs and the
|
||||
health response expose `agent_version`; version `2026.08.02.3` or newer includes
|
||||
this behavior.
|
||||
duplicate replacement behavior, while version `2026.08.02.4` adds adaptive
|
||||
candidate-window expansion and skip-reason reporting.
|
||||
|
||||
## Deploy
|
||||
|
||||
|
||||
Reference in New Issue
Block a user