live: EXTERNAL_SERVICE_TOKEN precedence, baked A/B candidates, model|gpu ledger from task/page modelId+gpuType, string total fix, Iluvatar/MTT enums (59 tests)

This commit is contained in:
Victor Cao
2026-10-17 06:53:56 +08:00
parent 3adf13ad22
commit 49d59b6bd4
20 changed files with 5029 additions and 129 deletions

View File

@@ -1,27 +1,11 @@
# Queue-gated live submit
# Live gate update — 2026-10-17
Victor-approved live defaults on. ALLOW_LIVE_SUBMIT=0 stops submission;
STRATEGY_ID remains mandatory. LIVE_CONFIRM is accepted but optional.
Queue snapshots accept integer or ASCII digit-string totals and derive model/GPU pairs from modelId and gpuType across all statuses. Nonterminal tasks count toward the queue gate. Invalid or incomplete snapshots stop submissions with ledger_rebuild_failed.
Each add requires a fresh authenticated onlyMine task/page snapshot, complete
through data.total (maximum 20 pages). All nonterminal records count;
queue >= LIVE_QUEUE_THRESHOLD (default 95) stops the cycle. Authentication,
transport, HTTP, platform, JSON and pagination errors fail closed. Environment
queue hints can only add blockers.
Token resolution prefers EXTERNAL_SERVICE_TOKEN, XC_TOKEN_FILE, XC_TOKEN, then MODELHUB_XC_TOKEN; empty and placeholder credentials are skipped. Missing credentials hold even with injected submit clients. Each allowed cycle logs only the selected source name. Xc-Token remains the authentication header.
Cycles prioritize Section A verified_reuse, other A, then B; reject banned
families, unmapped GPUs and placeholders; use build-config with recorded sample
fallback; reserve model identity in memory and LIVE_STATE_FILE before calling
TaskAddClient. Remote task identities also block duplicates. Default cap is 3
attempts per cycle. Failed calls retain their reservations to prevent retries.
The state file defaults to /tmp/live-submitted.json; preserve or mount it for
restart durability beyond ephemeral container storage.
Offline generators read local adapt evidence only. Input candidates: A=60, B=296; after pending and banned filters: A=46, B=296, total=342. Verified reuse A precedes other A, then B. Baked ledger: 206 pairs and 28 blocked models. Baked JSON ships through COPY src/; .dockerignore does not exclude src/data/*.json.
Server runs a daemon worker every 600 seconds (minimum 60), using the shutdown
Event for interruptible waits. Missing credentials hold instead of crashing.
Docker FROM is unchanged; templates are copied for runtime fallback.
The ledger merges remote pairs, baked pairs, and persisted model|gpu pairs. Explicit do-not-resubmit models are blocked on every GPU; other models may use the next mapped, unsubmitted GPU in candidate order. Reservations are written before task/add, including failed attempts. Legacy model-only state files fail closed and require offline migration. Startup rebuild logs source counts; each prospective submission refreshes remote identities and queue count.
Validation: make test passes 54 tests, including mocked platform calls and local
SIGTERM integration. Sandbox initially denied local sockets; the suite passed
with local socket permission. No real ModelHub calls, token writes, commits,
tags or pushes were performed.
Validation: make test passes 59 tests using stdlib only. Added coverage for pair dedupe and restart persistence, baked blocks, next-GPU selection, missing tokens, token source logging, digit-string totals, invalid totals, and baked candidate ordering/filtering. SIGTERM uses a mocked HTTP server and real subprocess signal without network calls. Dockerfile FROM unchanged. No git commit, push, tag, live probe, build-config call, or live submission performed.