fix: stream cold-start history into decision state

This commit is contained in:
CoolBoy
2026-09-04 11:20:51 +08:00
parent d60551e130
commit 54ed4351a0
7 changed files with 538 additions and 58 deletions

View File

@@ -61,9 +61,13 @@ Optional tuning:
- `MODELHUB_ARCHITECTURE_BLOCK_TTL_DAYS` default `30`
- `MODELHUB_ARCHITECTURE_COMMUNITY_PROBE_SIZE` default `50`
- `MODELHUB_ARCHITECTURE_COMMUNITY_LOOKBACK_DAYS` default `30`
- `MODELHUB_ARCHITECTURE_COMMUNITY_LATEST_LIMIT` default `5000`
- `MODELHUB_ARCHITECTURE_COMMUNITY_LATEST_LIMIT` default `100` (synchronous seed only)
- `MODELHUB_ARCHITECTURE_BOOTSTRAP_WORKERS` default `8`
- `MODELHUB_ARCHITECTURE_BOOTSTRAP_MAX_LOGS` default `0` (unlimited)
- `MODELHUB_ARCHITECTURE_BOOTSTRAP_TASKS_PER_ACCOUNT` default `10` (synchronous seed only)
- `MODELHUB_ARCHITECTURE_BOOTSTRAP_MAX_LOGS` default `120` (synchronous seed only)
- `MODELHUB_ARCHITECTURE_BACKFILL_PAGE_SIZE` default `50`
- `MODELHUB_ARCHITECTURE_BACKFILL_PAGES_PER_CYCLE` default `2`
- `MODELHUB_ARCHITECTURE_BACKFILL_LOGS_PER_CYCLE` default `100`
- `MODELHUB_RECENT_MODEL_RESERVE_SLOTS` default `5` per account
- `MODELHUB_RECENT_MODEL_DAYS` default `7`
- `MODELHUB_STATE_SYNC_REMOTE` default `https://dev.modelhub.org.cn/CoolBoy/submmit.git`
@@ -145,9 +149,10 @@ At startup the worker verifies the manifest and file checksums, restores local
capacity, outcomes, routing evidence, architecture rules, exclusions, intents,
and active-task context, then reconciles every account against the platform.
Pending intents are held for two hours before being released as unconfirmed.
The branch keeps 30 days of structured events plus lifetime aggregate counters;
raw stdout, credentials, request headers, downloaded archives, and full configs
are never copied. Git authentication uses a temporary `GIT_ASKPASS` helper, so
The branch keeps bounded recovery rows plus lifetime aggregate counters;
raw stdout, credentials, request headers, downloaded archives, complete task
history, and full configs are never copied. Git authentication uses an in-memory
HTTP authentication callback, so
the password is absent from command arguments, remotes, commits, and logs.
`GET /health` reports process liveness. `GET /ready` returns HTTP 200 only after
@@ -168,17 +173,17 @@ change that window. The stats report exposes `architectureCompatibilityBlocks`
and per-GPU/framework block counts. The live snapshot is written to
`.modelhub_state/architecture_compatibility_blacklist.json`.
Before the first cleanup/submission cycle, the poller probes the latest public
failed-validation records for usable failure archives. If the public endpoint
does not expose those archives (the current API behavior), it falls back to a
parallel, complete history scan of every configured account. Ledger metadata is
used when present; otherwise the task type is recovered from ModelHub's task
level and the selected framework is conservatively recovered from the target
container image in the failure archive. All usable historical failures are
classified once at startup and cached in `outcomes/submissions.jsonl`; subsequent
poll cycles continue incremental learning every three cycles. Empty or
incomplete public evidence is logged as a fallback, never as proof that no
architecture incompatibility exists.
Before the first cleanup/submission cycle, the poller imports only a small recent
seed so cold start cannot be dominated by historical API and log downloads. The
submission pass then runs first on every cycle. Afterwards a resumable backfill
walks every configured account's complete history in bounded pages, classifies
each available failure log, and immediately reduces the page to cumulative
success/failure statistics, memory observations, safe configurations, and exact
compatibility blocks. Only the cursor, temporary deduplication task IDs, and
derived decision checkpoint are synchronized; raw task rows and logs are not.
The task-ID set is deleted when backfill completes. A restart resumes the cursor
instead of rescanning prior pages. Empty public evidence is a fallback signal,
never proof that no incompatibility exists.
Before a candidate reaches the submit queue, failure-informed preflight checks
the actual ModelScope repository structure and file sizes. Non-GGUF text
@@ -425,11 +430,20 @@ those windows are discarded after their category, compatibility block, memory
observation, or safe configuration vector has been extracted. Existing archive
branches are left untouched but are no longer read or updated.
Version `2026.09.04.3` makes cold recovery incremental and exhaustive. A missing
state branch no longer triggers a blocking all-account/all-log scan. A small
synchronous seed creates the first decision checkpoint, then each submission
cycle consumes two 50-row history pages and up to 100 failure logs after the
submission pass. Progress is durable across restarts, all pages are eventually
processed, and each page is compacted immediately into decision state. It also
fixes large-batch classification so checkpoint compaction cannot discard an
uninspected failure row.
## Deploy
Create a tag and submit the repository URL plus tag in "我的适配智能体".
```bash
git tag -a agent-v31 -m "ModelHub agent 2026.09.04.2"
git push origin main agent-v31
git tag -a agent-v32 -m "ModelHub agent 2026.09.04.3"
git push origin main agent-v32
```