perf: complete history bootstrap in one streaming phase

This commit is contained in:
CoolBoy
2026-09-04 11:35:43 +08:00
parent 54ed4351a0
commit 2c748dcec2
6 changed files with 183 additions and 298 deletions

View File

@@ -103,12 +103,13 @@ bash run_poll.sh --dry-run
rule immediately launches a lightweight architecture-only queue scan. Exact
matching waiting tasks are stopped after two state checks; running tasks and
tasks without local framework/task metadata are protected.
- Startup imports a small community/owned seed for immediate decisions. After
each submission pass, a durable cursor incrementally scans complete terminal
history across all configured accounts. Each bounded page is classified and
folded into aggregate decision state immediately; raw task rows and failure
logs are never synchronized. Restarts resume the cursor, and the temporary
deduplication IDs are removed when the exhaustive backfill completes.
- Cold start performs no blocking history seed. After the first submission pass,
a durable cursor completes the full history scan as one streaming phase using
about 1,000 task rows per metadata batch and 200 logs per classification batch.
Aggregate checkpoints are synchronized about every 2,000 rows; raw task rows
and failure logs are never synchronized. Restarts resume the cursor, temporary
deduplication IDs disappear at completion, and later cycles process only new
outcome changes.
- A strategy generation lasts exactly 200 platform-accepted submissions. Rejected API calls and
duplicates do not advance it. The next cycle refreshes platform history before submitting again.
- Strategy state is stored in `.modelhub_state/gpu_strategy.json`; a generation never recalculates