feat: clean deterministic OOM tasks on startup
This commit is contained in:
19
README.md
19
README.md
@@ -52,6 +52,9 @@ Optional tuning:
|
||||
- `MODELSCOPE_PAGE_INTERVAL_SECONDS` default `0.25`
|
||||
- `MODELSCOPE_PAGE_CACHE_TTL_SECONDS` default `900`
|
||||
- `MODELHUB_AGENT_VERIFY_CACHE_TTL_SECONDS` default `900`
|
||||
- `MODELHUB_QUEUE_CLEANUP_INTERVAL_CYCLES` default `120`; cleanup also runs once at startup
|
||||
- `MODELHUB_QUEUE_CLEANUP_READ_CONCURRENCY` default `6`
|
||||
- `MODELHUB_QUEUE_CLEANUP_REPORT_PATH` default `.modelhub_state/queue_cleanup_latest.json`
|
||||
|
||||
## Adaptive GPU Strategy
|
||||
|
||||
@@ -119,6 +122,19 @@ capacities with
|
||||
`MODELHUB_GPU_MEMORY_GIB_JSON`, for example
|
||||
`{"New_gpu": 64}`.
|
||||
|
||||
At poller startup, the same deterministic memory gate is applied to existing
|
||||
`waiting` and `running` tasks across every configured account. A task is stopped
|
||||
through `PUT /api/async/task/stop-create-contest-task` only when its own current
|
||||
recursive repository size, multiplied by ModelHub's observed `1.20` overhead,
|
||||
exceeds the known capacity of its selected GPU. The task ID is fetched and
|
||||
stopped with the token belonging to that account, and its active state is
|
||||
rechecked immediately before the mutation. Missing file sizes, unknown GPU
|
||||
capacities, listing failures, and tasks that have already changed state all fail
|
||||
closed and are never stopped. This does not match against another task from the
|
||||
same model or infer failure from historical similarity. The cleanup repeats
|
||||
every 120 poll cycles by default and writes its full evidence report to
|
||||
`.modelhub_state/queue_cleanup_latest.json`.
|
||||
|
||||
The verified capacities, safe repository-size boundaries, evidence hierarchy,
|
||||
and source links are recorded in
|
||||
`docs/gpu-memory-capacity-2026-08-10.md`.
|
||||
@@ -207,6 +223,9 @@ GPU, recursive repository-size checks, deterministic failure-aware preflight,
|
||||
and rate-limited lazy Qwen review for unresolved semantic cases.
|
||||
Version `2026.08.10.3` selects `qwen3.7-flash` by default and recognizes the
|
||||
repository root `.env` key named `dashscope` without logging its value.
|
||||
Version `2026.08.11.1` adds account-owned cancellation of queued tasks that are
|
||||
deterministically over the selected GPU's ModelHub memory boundary, with a
|
||||
second active-state check and fail-closed handling for incomplete evidence.
|
||||
|
||||
## Deploy
|
||||
|
||||
|
||||
Reference in New Issue
Block a user