ModelHub Adaptation Agent
This repository is packaged for the ModelHub XC agent platform.
Platform Contract
- Root-level
Dockerfile - Listens on port
8080 - Exposes
GET /health - Handles
SIGTERM - Reads platform-provided
STRATEGY_IDand attaches it to task submissions asstrategyId
The root main.py starts a lightweight health server and runs the existing
submission poller in a child process.
Runtime Environment
The image includes multi-account ModelHub and ModelScope token fallbacks for the agent platform. Environment variables can override them without rebuilding the image.
MODELHUB_XC_TOKEN,XC_TOKEN,XC_TOKEN2..., orMODELHUB_XC_TOKENSfor ModelHub API authenticationMODELHUB_JWT_TOKENorJWT_TOKENcan be used instead when the platform provides a JWTMODELSCOPE_API_TOKENorMODELSCOPE_TOKENoptional override for the embedded ModelScope fallback tokenSTRATEGY_IDis expected to be injected by the ModelHub agent platform and is attached to submissions for strategy attribution; it is not an API authentication token
Optional tuning:
MODELHUB_AGENT_POLL_INTERVAL_SECONDSdefault15MODELHUB_AGENT_IDLE_INTERVAL_SECONDSdefault60MODELHUB_AGENT_POST_CYCLE_COOLDOWN_SECONDSdefault2- The hosted entrypoint always uses
--max-submits-per-run 0so stale deployment settings cannot restrict a refill cycle to five submissions. MODELHUB_AGENT_ACTIVE_TASK_CAPdefault100per accountMODELHUB_CAPACITY_PROBE_INTERVAL_CYCLESdefault3MODELHUB_CAPACITY_STATE_PATHdefault.modelhub_state/account_capacity.jsonMODELHUB_AGENT_ACTIVE_COUNTS_TTL_SECONDSdefault15MODELHUB_AGENT_RESERVATION_TTL_SECONDSdefault120MODELHUB_AGENT_INSTANCE_IDoptional stable worker identity used to spread concurrent agents across accounts and candidatesMODELHUB_AGENT_CLAIMS_PATHdefault.modelhub_state/submission_claims.jsonlMODELHUB_SUBMISSION_EXCLUSIONS_PATHdefault.modelhub_state/submission_exclusions.jsonlMODELHUB_AGENT_DAILY_TARGETMODELHUB_AGENT_MIN_DOWNLOADSMODELHUB_AGENT_GPUSMODELHUB_AGENT_EXTRA_ARGSMODELHUB_GPU_STRATEGY_STATE_PATHdefault.modelhub_state/gpu_strategy.jsonMODELHUB_MARKET_INTELLIGENCE_PATHdefault.modelhub_state/market_intelligence.jsonMODELHUB_MARKET_QUEUE_REFRESH_SECONDSdefault600MODELHUB_MARKET_FRAMEWORK_REFRESH_SECONDSdefault21600MODELHUB_MARKET_THROUGHPUT_WINDOW_HOURSdefault6MODELHUB_MARKET_FRAMEWORK_MIN_SAMPLESdefault300MODELSCOPE_PAGE_INTERVAL_SECONDSdefault0.25MODELSCOPE_PAGE_CACHE_TTL_SECONDSdefault900MODELHUB_AGENT_VERIFY_CACHE_TTL_SECONDSdefault900
Adaptive GPU Strategy
When no explicit GPU override is supplied, the worker uses a success-first 70/30 strategy generation with no self-funded exploration:
- 70%: the three long-term GPUs with the best Wilson lower confidence score and at least 100 terminal samples
- 30%: the top recent GPUs among the latest 1,000 terminal tasks
- 0%: unvetted/all-GPU exploration; community-wide results provide the exploration signal
The 70/30 category ratio remains exact across accepted tasks. Inside each category, weighted fair scheduling combines the category's historical rank with live public market data:
- recent public success quality, scored with a strongly weighted Wilson lower confidence bound
- estimated backlog hours (
waiting / recent completions per hour) as a bounded tie-breaker - machine availability, running workers, and advertised concurrency
- a circuit breaker for unavailable or apparently stalled GPU pools
This optimizes expected successful completions rather than blindly selecting the
smallest queue. Queue/throughput data is refreshed every 10 minutes and persisted
in .modelhub_state/market_intelligence.json. A failed refresh keeps the last good
snapshot, uses a retry backoff, and never blocks normal submissions.
For each compatible model/GPU pair, the worker also ranks the GPU's supported
frameworks using ModelHub's public aggregate modelCount and successCount data,
then blends in the worker's own GPU+framework outcomes with a capped weight.
Only frameworks with at least 300 public samples and a safe Wilson lower bound
are eligible. Framework statistics refresh
every 6 hours, so they do not add per-model API traffic. Newly published
frameworks are discovered automatically, but receive no novelty bonus: they can
win only when their confidence-adjusted success score beats the best incumbent by
at least 10%. A new framework
is eligible only after the authenticated official build-config endpoint returns
a complete config that passes local structure, placeholder, framework-name, and
GPU-parallelism validation. Valid official configs are cached and refreshed with
the framework snapshot; local templates remain the fail-safe fallback.
Only platform-accepted tasks count. After exactly 200 accepted tasks, the next
poll cycle reloads all account history, generates a new immutable strategy snapshot,
and resets the generation counters to 140/60 targets. The active snapshot and
progress are stored in .modelhub_state/gpu_strategy.json.
Five consecutive local failures open a 12-hour GPU/framework circuit breaker. A sub-20% success rate over the latest 20 terminal tasks opens a 6-hour breaker. Platform/infrastructure failures are excluded from long-term compatibility rates and model/profile breakers. Three consecutive platform failures on a GPU/framework instead open a short 30-minute breaker, so a temporary broken runner or lack of an idle card does not permanently poison otherwise successful evidence. Candidate shortages expand the model search window; they never unlock an unvetted GPU or framework.
Before a candidate reaches the submit queue, failure-informed preflight checks
the actual ModelScope repository structure and file sizes. Non-GGUF text
frameworks require root-level config, weights, and tokenizer assets. The memory
gate recursively totals the entire repository—including duplicate weight formats
and nested shards—and applies ModelHub's observed 20% loading overhead. It covers
all 14 GPU types currently marked canVerify=true; nine capacities come directly
from structured ModelHub OOM reports and five from published specifications until
ModelHub supplies a stronger observation. If a known GPU's repository file sizes
are incomplete, the candidate is deferred rather than guessed. Template context
length is also clamped to the model's advertised limit. A newly introduced GPU
with no capacity evidence is likewise deferred. Override or extend known
capacities with
MODELHUB_GPU_MEMORY_GIB_JSON, for example
{"New_gpu": 64}.
The verified capacities, safe repository-size boundaries, evidence hierarchy,
and source links are recorded in
docs/gpu-memory-capacity-2026-08-10.md.
Ambiguous custom architectures can optionally be reviewed by a small
OpenAI-compatible Qwen model. Qwen is lazy: deterministic rules handle repository
layout, model size, context length, known errors, and ordinary quantization cases
without an LLM call. Set
MODELHUB_LLM_CLASSIFIER_ENDPOINT to the full chat-completions URL and
MODELHUB_LLM_CLASSIFIER_MODEL; set MODELHUB_LLM_CLASSIFIER_API_KEY only when
the endpoint requires it. The default deny threshold is 0.85 and can be changed
with MODELHUB_LLM_CLASSIFIER_MIN_DENY_CONFIDENCE. For Alibaba Model Studio,
the aliases are MODELHUB_QWEN_ENDPOINT, MODELHUB_QWEN_MODEL, and
MODELHUB_QWEN_API_KEY (or DASHSCOPE_API_KEY); endpoint omission uses the
DashScope OpenAI-compatible URL. A root .env entry named dashscope is also
recognized directly, and the default model is qwen3.7-flash. Calls default to
one concurrent request and 20
requests per rolling hour, configurable with MODELHUB_LLM_MAX_CONCURRENT_REQUESTS
and MODELHUB_LLM_MAX_CALLS_PER_HOUR. The LLM may only veto an
ambiguous candidate: it cannot bypass deterministic checks, introduce a new
framework, or override public success-evidence gates. Results are cached under
.modelhub_state/llm_classifications.json.
Outcome synchronization downloads a bounded set of failure archives for submissions created by this worker (at most 40 per sync, four workers, three download attempts). Deterministic signatures classify memory, repository layout, context-length, storage, and platform faults first. Only unresolved runtime errors are sent to the optional LLM; a semantic result is promoted only at confidence 0.80 or higher. Signed log URLs remain in the ignored local outcome store and are removed after classification.
The 12-account failure study and routing rationale are recorded in
docs/failure-analysis-2026-08-10.md.
ModelScope HTTP 429 responses use exponential backoff and Retry-After. Successful
pages remain cached, so a later cycle retries the failed page instead of restarting
the whole pagination scan.
Adaptive Candidate Discovery
The configured recent window remains the fast path. If it contains no usable model/GPU combinations, the same run progressively expands discovery to the last 7 days, the last 30 days, and finally older history (up to 3,000 models). Scanning stops as soon as enough replacement candidates have been found.
Model verification results are reused for 15 minutes across poll cycles, and a
locally failed model/GPU pair cools down for 24 hours instead of being excluded
forever. The [scan] lines show every expansion stage, while [daily] wave_done
includes skip_reasons so an empty candidate pool is directly diagnosable.
Community deduplication is scoped to the exact model/GPU combination. A model
adapted on one GPU remains eligible for another GPU. Before each submission the
worker performs an uncached community check; a lookup failure defers the task
instead of failing open. Platform uniqueness rejections are persisted per
model/GPU in .modelhub_state/submission_exclusions.jsonl and are not retried.
Concurrent Agents
The token pool keeps a local reservation for every in-flight submission, so a lagging platform count cannot send all concurrent requests to the same account. If another process fills an account first, the submission is retried immediately against another account with capacity.
Workers that share a filesystem also coordinate model/GPU claims through
.modelhub_state/submission_claims.jsonl. Workers in isolated containers use
different candidate ordering (derived from STRATEGY_ID, instance ID, or
hostname), which reduces duplicate work while the platform remains the final
authority for account capacity and model/GPU uniqueness.
If the platform reports that a model/GPU is already being validated, the claim
is retained and the runner immediately draws replacement candidates from the
same scan instead of retrying the duplicate every cycle. Startup logs and the
health response expose agent_version; version 2026.08.02.3 or newer includes
duplicate replacement behavior, while version 2026.08.02.4 adds adaptive
candidate-window expansion and skip-reason reporting. Version 2026.08.02.5
adds fail-closed model/GPU prechecks and persistent uniqueness exclusions.
Version 2026.08.04.1 adds queue/throughput intelligence, GPU health circuit
breaking, weighted-fair scheduling, live framework/config discovery, and
confidence-ranked public-plus-local framework selection.
Version 2026.08.05.1 removes self-funded GPU exploration, switches accepted
traffic to 70/30 long-term/recent exploitation, raises the public framework gate
to 300 samples, makes success dominate queue pressure, and adds recent local
GPU/framework circuit breakers.
Version 2026.08.10.2 adds evidence-backed sizing for every currently verifiable
GPU, recursive repository-size checks, deterministic failure-aware preflight,
and rate-limited lazy Qwen review for unresolved semantic cases.
Version 2026.08.10.3 selects qwen3.7-flash by default and recognizes the
repository root .env key named dashscope without logging its value.
Deploy
Create a tag and submit the repository URL plus tag in "我的适配智能体".
git tag agent-v12
git push origin agent-v12