CoolBoy/submmit
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

fix: coordinate concurrent account capacity filling

Browse Source
This commit is contained in:
CoolBoy
2026-08-01 16:50:50 +08:00
parent c06169d906
commit 0cabc43abf
14 changed files with 794 additions and 143 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
modelhub_submmit_api/README.md
View File

@@ -74,6 +74,9 @@ bash run_poll.sh --dry-run
- The runner auto-discovers all safe GPU/template combinations from the public submit catalog.
- Each model can be submitted at most once per GPU.
- Multiple ModelHub tokens are pooled and used to route submissions to the account with available async capacity.
- Concurrent submissions reserve account slots locally, and an account-capacity race automatically falls through to another account.
- Concurrent local processes claim model/GPU pairs in `.modelhub_state/submission_claims.jsonl`; shared ledger, history, and outcome files use process locks and atomic replacement.
- Isolated agent containers diversify candidate order by instance identity to reduce cross-container duplicate submissions.
- History stats are online-only. The local ledger is used for local accounting, but platform history is only used after the local ledger reaches the configured threshold.
- The default history threshold is `500` records.
@@ -96,15 +99,15 @@ Common flags:
`run_poll.sh` adds:
- `--poll-interval-seconds`: sleep when all accounts are saturated
- `--idle-interval-seconds`: sleep when a cycle submits nothing
- `--poll-interval-seconds`: sleep when all accounts are saturated (default 15)
- `--idle-interval-seconds`: sleep when a cycle submits nothing (default 60)
- `--max-scan-models`: hard cap on scanned HF models for this cycle (0 = auto)
- `--scan-multiplier`: multiplier used for auto scan cap derivation from quota/queue capacity
- `--max-submits-per-run`: max tasks to submit per poll cycle (0 = unlimited)
- `--skip-outcome-sync`: skip outcome sync before scanning
- `--skip-history-archive`: skip history archive download for this cycle
- `--submit-concurrency`: concurrent task submission calls used by each cycle (0 = auto)
- `--post-cycle-cooldown-seconds`: pause after a successful cycle before next cycle (default 0)
- `--post-cycle-cooldown-seconds`: pause after a successful cycle before next cycle (default 2)
- `--max-cycles`: optional hard stop for testing or batch windows
## Output
@@ -140,3 +143,6 @@ python3 -m unittest discover -s tests -v
- The platform still enforces per-account async capacity limits, so the poller can
keep the queue close to full but cannot override the platform cap.
- `bash run_poll.sh` now defaults to unlimited mode and keeps refilling until you stop the process manually.
- Queue polling defaults to 15 seconds, successful-cycle cooldown to 2 seconds,
and per-cycle submissions to all available slots. Override these values when
the platform requires a lower request rate.