CoolBoy/submmit
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity

adapt for ModelHub agent platform

Browse Source
This commit is contained in:
CoolBoy
2026-07-10 00:22:50 +08:00
commit 8c4bbff413
28 changed files with 3991 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

141
modelhub_submmit_api/README.md Normal file
View File

@@ -0,0 +1,141 @@
# ModelHub Submission Runner
This package automates Hugging Face model discovery and ModelHub submission.
It currently supports:
- one-shot submission planning via `main.py`
- daily batch execution via `run_daily.sh`
- continuous queue refill via `run_poll.sh`
- a single ModelHub token read from environment variables or `KEY.md`
- automatic task/framework/template selection across the supported GPU catalog
## Layout
- `main.py`: core discovery, scoring, dedup, and submission
- `daily_runner.py`: daily wave orchestration
- `poll_runner.py`: long-running queue refiller
- `runner_common.py`: shared token / key file loading
- `hf_discovery.py`: Hugging Face model discovery and inspection
- `modelhub_client.py`: ModelHub API client
- `history_stats.py`: online history aggregation, ranking, and warnings
- `template_selector.py`: template lookup and GPU normalization
- `task_registry.py`: task-type and framework selection rules
- `tests/`: unit tests and regression coverage
## Key Files
- `KEY.md`: optional local Hugging Face and ModelHub tokens
- `templates/public_submit/adapt_task_templates.jsonl`: public submit templates
The agent platform deployment should use environment variables instead of key
files. Set `MODELHUB_XC_TOKEN` or `XC_TOKEN` for the single submitting account.
Template lookup is also relative. The selector searches from the current working
directory and the module directory. The primary project layout is:
- `templates/public_submit/adapt_task_templates.jsonl`
It still accepts the legacy fallback path below for compatibility with older
deployments:
- `model adaptation/templates/public_submit/adapt_task_templates.jsonl`
If your Space keeps templates in another location, set `MODELHUB_TEMPLATE_FILE`
to the exact JSONL path.
## Quick Start
Run a single daily batch:
```bash
cd /path/to/submmit
# testing: one run defaults to 3 targets if daily-target is not specified
bash run_daily.sh --rounds 1
```
Run the continuous queue refiller:
```bash
cd /path/to/submmit
bash run_poll.sh
```
Dry-run either entrypoint to inspect candidate selection without submitting:
```bash
cd /path/to/submmit
bash run_daily.sh --dry-run
bash run_poll.sh --dry-run
```
## Behavior
- The runner auto-discovers all safe GPU/template combinations from the public submit catalog.
- Each model can be submitted at most once per GPU.
- One ModelHub account is used per deployed agent instance.
- History stats are online-only. The local ledger is used for local accounting, but platform history is only used after the local ledger reaches the configured threshold.
- The default history threshold is `500` records.
## Important Flags
Common flags:
- `--daily-target`: total target submissions for the day; `0` means unlimited
- `--min-downloads`: Hugging Face download floor
- `--history-stats-threshold`: local ledger threshold before using online history stats
- `--max-scan-models`: hard cap on scanned HF models for a run (0 = auto)
- `--scan-multiplier`: multiplier used for auto scan cap derivation from quota/queue capacity
- `--read-concurrency`: concurrent HTTP reads while scanning model candidates (default 4)
- `--max-submits-per-run`: max tasks to submit per run cycle (0 = unlimited)
- `--submit-concurrency`: concurrent task submissions (default auto, uses 0)
- `--skip-outcome-sync`: skip outcome sync before scanning
- `--skip-history-archive`: skip history archive download for this run
- `--dry-run`: plan only, do not submit
- `run_daily.sh` injects `--daily-target 3` when no daily-target flag is provided. Set `SUBMIT_DAILY_TARGET` or pass `--daily-target` explicitly for a different target.
`run_poll.sh` adds:
- `--poll-interval-seconds`: sleep when the account has no available slots
- `--idle-interval-seconds`: sleep when a cycle submits nothing
- `--max-scan-models`: hard cap on scanned HF models for this cycle (0 = auto)
- `--scan-multiplier`: multiplier used for auto scan cap derivation from quota/queue capacity
- `--max-submits-per-run`: max tasks to submit per poll cycle (0 = unlimited)
- `--skip-outcome-sync`: skip outcome sync before scanning
- `--skip-history-archive`: skip history archive download for this cycle
- `--submit-concurrency`: concurrent task submission calls used by each cycle (0 = auto)
- `--post-cycle-cooldown-seconds`: pause after a successful cycle before next cycle (default 0)
- `--max-cycles`: optional hard stop for testing or batch windows
## Output
Run artifacts are written under:
- `runs/`: one-shot submission runs
- `daily_runs/`: batch orchestration runs
- `poll_runs/`: poller cycles
Each run typically includes:
- `summary.json`
- `pre_submit_report.json`
- `candidates.jsonl`
- `submitted.jsonl`
- `skipped.jsonl`
- `failed.jsonl`
## Verification
Run the full test suite:
```bash
cd /path/to/submmit
python3 -m unittest discover -s tests -v
```
## Notes
- This is a submission automation tool, not a scheduler daemon. Use `screen`, `tmux`,
`nohup`, or `systemd` if you want it to keep running in the background.
- The platform still enforces per-account async capacity limits, so the poller can
keep the queue close to full but cannot override the platform cap.
- `bash run_poll.sh` now defaults to unlimited mode and keeps refilling until you stop the process manually.

2
modelhub_submmit_api/__init__.py Normal file
View File

@@ -0,0 +1,2 @@
"""Submission CLI for public ModelHub adapt tasks."""

86
modelhub_submmit_api/common.py Normal file
View File

@@ -0,0 +1,86 @@
from __future__ import annotations
import json
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
def ensure_utc(value: datetime) -> datetime:
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)
def utc_now() -> datetime:
return datetime.now(timezone.utc)
def parse_datetime(value: Any) -> datetime | None:
if value in (None, ""):
return None
if isinstance(value, datetime):
return ensure_utc(value)
text = str(value).strip()
candidates = [
text,
text.replace("Z", "+00:00"),
text.replace(" ", "T"),
text.replace(" ", "T").replace("Z", "+00:00"),
]
for candidate in candidates:
try:
return ensure_utc(datetime.fromisoformat(candidate))
except ValueError:
continue
for fmt in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
try:
parsed = datetime.strptime(text, fmt)
return parsed.replace(tzinfo=timezone.utc)
except ValueError:
continue
return None
def isoformat_z(value: datetime) -> str:
return ensure_utc(value).isoformat().replace("+00:00", "Z")
def format_modelhub_datetime(value: datetime) -> str:
return ensure_utc(value).strftime("%Y-%m-%d %H:%M:%S")
def json_dumps(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True)
def read_json(path: Path) -> Any:
return json.loads(path.read_text(encoding="utf-8"))
def write_json(path: Path, value: Any) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True) + "\n", encoding="utf-8")
def read_jsonl(path: Path) -> list[dict[str, Any]]:
if not path.exists():
return []
rows: list[dict[str, Any]] = []
for line in path.read_text(encoding="utf-8").splitlines():
if not line.strip():
continue
rows.append(json.loads(line))
return rows
def write_jsonl(path: Path, rows: list[dict[str, Any]]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
content = "".join(json.dumps(row, ensure_ascii=False, sort_keys=True) + "\n" for row in rows)
path.write_text(content, encoding="utf-8")
def append_jsonl(path: Path, row: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
with path.open("a", encoding="utf-8") as handle:
handle.write(json.dumps(row, ensure_ascii=False, sort_keys=True) + "\n")

299
modelhub_submmit_api/daily_runner.py Normal file
View File

@@ -0,0 +1,299 @@
from __future__ import annotations
import argparse
import os
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Callable
from common import utc_now, write_json
from hf_discovery import HuggingFaceDiscovery
from main import DEFAULT_LEDGER_PATH, DEFAULT_RUNS_DIR, run_submission
from modelhub_client import ModelHubClient
from outcome_tracker import OutcomeTracker
from runner_common import DEFAULT_KEY_PATH, ensure_tokens
from template_selector import TemplateSelector
DEFAULT_DAILY_RUNS_DIR = Path("daily_runs")
@dataclass(frozen=True)
class WaveSpec:
name: str
task_types: tuple[str, ...]
gpus: str | None = None
limit: int = 400
since_hours: int = 168
DEFAULT_WAVES: tuple[WaveSpec, ...] = (
WaveSpec("text_generation_core", ("text-generation",), gpus=None, limit=100000, since_hours=48),
# WaveSpec("visual_multimodal_core", ("visual-multi-modal",), gpus=None, limit=400, since_hours=336),
# WaveSpec("image_generation", ("text-to-image-generation",), gpus=None, limit=250, since_hours=720),
# WaveSpec("asr", ("asr",), gpus=None, limit=250, since_hours=720),
# WaveSpec("embedding_and_qa", ("feature_emb", "question_answering"), gpus=None, limit=200, since_hours=720),
# WaveSpec("classification", ("text_classification", "vision_classification"), gpus=None, limit=200, since_hours=720),
# WaveSpec("reinforcement_learning", ("reinforcement_learning",), gpus=None, limit=200, since_hours=720),
)
def log(message: str) -> None:
print(message, flush=True)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="One-click daily ModelHub submission runner.")
parser.add_argument("--daily-target", type=int, default=0, help="Total submissions to aim for per UTC day; 0 means unlimited")
parser.add_argument("--rounds", type=int, default=3, help="How many wave cycles to run before stopping")
parser.add_argument("--gpu", help="Single GPU alias or platform name, for example: k100")
parser.add_argument("--gpus", help="Comma-separated GPU aliases/platform names. Omit to auto-use all safe GPUs.")
parser.add_argument("--max-scan-models", type=int, default=0, help="Hard cap on total scanned models (0 means auto)")
parser.add_argument(
"--scan-multiplier",
type=int,
default=4,
help="Multiplier used when auto-deriving scan limit from quota/queue capacity",
)
parser.add_argument("--min-downloads", type=int, default=50, help="Minimum Hugging Face download threshold")
parser.add_argument(
"--history-stats-threshold",
type=int,
default=500,
help="Minimum local ledger records before using history stats to rank submissions",
)
parser.add_argument("--read-concurrency", type=int, default=4, help="Concurrency for read-only remote calls")
parser.add_argument(
"--submit-concurrency",
type=int,
default=0,
help="Concurrency for task submission calls (0 = auto based on token/client count)",
)
parser.add_argument(
"--max-submits-per-run",
type=int,
default=0,
help="Maximum tasks to submit in one run (0 means unlimited)",
)
parser.add_argument("--skip-outcome-sync", action="store_true", help="Skip outcome sync from ModelHub before scanning")
parser.add_argument("--skip-history-archive", action="store_true", help="Skip historical task archive download for this run")
parser.add_argument("--dry-run", action="store_true", help="Plan the day without creating tasks")
parser.add_argument("--key-path", default=str(DEFAULT_KEY_PATH), help="Path to KEY.md containing HF_TOKEN/XC_TOKEN")
parser.add_argument("--runs-dir", default=str(DEFAULT_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--ledger-path", default=str(DEFAULT_LEDGER_PATH), help=argparse.SUPPRESS)
parser.add_argument("--history-archive-path", default="history/platform_tasks.jsonl", help=argparse.SUPPRESS)
parser.add_argument("--history-archive-limit", type=int, default=5000, help=argparse.SUPPRESS)
parser.add_argument("--daily-runs-dir", default=str(DEFAULT_DAILY_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--hf-base-url", default=os.getenv("HF_BASE_URL", "https://huggingface.co"), help=argparse.SUPPRESS)
parser.add_argument("--modelhub-base-url", default=os.getenv("MODELHUB_BASE_URL", "https://modelhub.org.cn"), help=argparse.SUPPRESS)
parser.add_argument("--modelhub-token", default=os.getenv("MODELHUB_XC_TOKEN") or os.getenv("XC_TOKEN"), help=argparse.SUPPRESS)
parser.add_argument("--hf-token", default=os.getenv("HF_TOKEN"), help=argparse.SUPPRESS)
return parser
def make_wave_namespace(base_args: argparse.Namespace, wave: WaveSpec) -> argparse.Namespace:
user_gpu = getattr(base_args, "gpu", None)
user_gpus = getattr(base_args, "gpus", None)
return argparse.Namespace(
gpu=user_gpu,
gpus=user_gpus if (user_gpus or user_gpu) else wave.gpus,
task_types=",".join(wave.task_types),
limit=wave.limit,
min_downloads=base_args.min_downloads,
daily_target=base_args.daily_target,
dry_run=base_args.dry_run,
since_hours=wave.since_hours,
updated_after=None,
stats_window_days=7,
history_stats_threshold=base_args.history_stats_threshold,
read_concurrency=base_args.read_concurrency,
max_scan_models=getattr(base_args, "max_scan_models", 0),
scan_multiplier=getattr(base_args, "scan_multiplier", 4),
skip_outcome_sync=getattr(base_args, "skip_outcome_sync", False),
skip_history_archive=getattr(base_args, "skip_history_archive", False),
submit_concurrency=getattr(base_args, "submit_concurrency", 1),
max_submits_per_run=getattr(base_args, "max_submits_per_run", 0),
runs_dir=base_args.runs_dir,
ledger_path=base_args.ledger_path,
outcomes_path=getattr(base_args, "outcomes_path", "outcomes/submissions.jsonl"),
history_archive_path=base_args.history_archive_path,
history_archive_limit=base_args.history_archive_limit,
hf_base_url=base_args.hf_base_url,
modelhub_base_url=base_args.modelhub_base_url,
modelhub_token=base_args.modelhub_token,
)
def run_daily_batches(
*,
base_args: argparse.Namespace,
waves: tuple[WaveSpec, ...] = DEFAULT_WAVES,
now=None,
run_fn: Callable[..., dict[str, Any]] = run_submission,
hf_discovery: HuggingFaceDiscovery | None = None,
modelhub_client: ModelHubClient | None = None,
template_selector: TemplateSelector | None = None,
outcome_tracker: OutcomeTracker | None = None,
) -> dict[str, Any]:
now = now or utc_now()
hf_discovery = hf_discovery or HuggingFaceDiscovery(base_url=base_args.hf_base_url)
if modelhub_client is None:
modelhub_client = ModelHubClient(token=base_args.modelhub_token, base_url=base_args.modelhub_base_url)
template_selector = template_selector or TemplateSelector()
daily_run_dir = Path(base_args.daily_runs_dir) / now.strftime("%Y%m%dT%H%M%SZ")
daily_run_dir.mkdir(parents=True, exist_ok=True)
log(f"[daily] daily_run_dir={daily_run_dir}")
log(
f"[daily] target={base_args.daily_target} rounds={base_args.rounds} "
f"dry_run={str(bool(base_args.dry_run)).lower()} waves={len(waves)}"
)
if hasattr(modelhub_client, "active_task_counts"):
active_counts = modelhub_client.active_task_counts()
log(f"[daily] modelhub_accounts={len(active_counts)} active_counts={','.join(str(count) for count in active_counts)}")
wave_results: list[dict[str, Any]] = []
submitted_total = 0
attempted_waves = 0
stopped_reason = "max_rounds_reached"
for round_index in range(1, base_args.rounds + 1):
round_submitted = 0
log(f"[daily] round={round_index} start")
for wave in waves:
attempted_waves += 1
wave_args = make_wave_namespace(base_args, wave)
log(
f"[daily] round={round_index} wave={wave.name} "
f"tasks={','.join(wave.task_types)} gpus={wave.gpus or 'auto'} "
f"limit={wave.limit} since_hours={wave.since_hours}"
)
summary = run_fn(
wave_args,
now=utc_now(),
hf_discovery=hf_discovery,
modelhub_client=modelhub_client,
template_selector=template_selector,
outcome_tracker=outcome_tracker,
)
wave_result = {
"round": round_index,
"wave": asdict(wave),
"summary": summary,
}
wave_results.append(wave_result)
submitted_total += summary["submittedCount"]
round_submitted += summary["submittedCount"]
log(
f"[daily] wave_done name={wave.name} "
f"candidates={summary['candidateCount']} planned={summary['plannedSubmitCount']} "
f"submitted={summary['submittedCount']} skipped={summary['skippedCount']} "
f"failed={summary['failedCount']} remaining_before_run={summary['remainingDailyQuotaBeforeRun']}"
)
if summary.get("platformAvailableSlotsBeforeRun") == 0:
stopped_reason = "platform_async_cap_reached"
log(f"[daily] stop={stopped_reason}")
return finalize_daily_run(
daily_run_dir=daily_run_dir,
now=now,
base_args=base_args,
wave_results=wave_results,
submitted_total=submitted_total,
attempted_waves=attempted_waves,
stopped_reason=stopped_reason,
)
if base_args.daily_target > 0 and summary["remainingDailyQuotaBeforeRun"] <= 0:
stopped_reason = "daily_target_already_reached"
log(f"[daily] stop={stopped_reason}")
return finalize_daily_run(
daily_run_dir=daily_run_dir,
now=now,
base_args=base_args,
wave_results=wave_results,
submitted_total=submitted_total,
attempted_waves=attempted_waves,
stopped_reason=stopped_reason,
)
if base_args.daily_target > 0 and not base_args.dry_run and summary["plannedSubmitCount"] <= 0:
stopped_reason = "daily_target_reached"
log(f"[daily] stop={stopped_reason}")
return finalize_daily_run(
daily_run_dir=daily_run_dir,
now=now,
base_args=base_args,
wave_results=wave_results,
submitted_total=submitted_total,
attempted_waves=attempted_waves,
stopped_reason=stopped_reason,
)
if round_submitted <= 0:
stopped_reason = "no_new_submissions_in_round"
log(f"[daily] round={round_index} submitted=0 stop={stopped_reason}")
break
log(f"[daily] round={round_index} submitted={round_submitted}")
return finalize_daily_run(
daily_run_dir=daily_run_dir,
now=now,
base_args=base_args,
wave_results=wave_results,
submitted_total=submitted_total,
attempted_waves=attempted_waves,
stopped_reason=stopped_reason,
)
def finalize_daily_run(
*,
daily_run_dir: Path,
now,
base_args: argparse.Namespace,
wave_results: list[dict[str, Any]],
submitted_total: int,
attempted_waves: int,
stopped_reason: str,
) -> dict[str, Any]:
last_wave_summary = wave_results[-1]["summary"] if wave_results else {}
summary = {
"generatedAt": now.isoformat(),
"dryRun": bool(base_args.dry_run),
"dailyTarget": base_args.daily_target,
"unlimitedDailyTarget": base_args.daily_target <= 0,
"rounds": base_args.rounds,
"attemptedWaves": attempted_waves,
"submittedTotal": submitted_total,
"stoppedReason": stopped_reason,
"dailyRunDir": str(daily_run_dir),
"remainingDailyQuotaBeforeRun": last_wave_summary.get("remainingDailyQuotaBeforeRun"),
"platformAvailableSlotsBeforeRun": last_wave_summary.get("platformAvailableSlotsBeforeRun"),
"waveResults": wave_results,
}
write_json(daily_run_dir / "summary.json", summary)
log(
f"[daily] finished submitted_total={submitted_total} attempted_waves={attempted_waves} "
f"stopped_reason={stopped_reason}"
)
return summary
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
ensure_tokens(args)
log(
f"[daily] hf_token={'set' if bool(args.hf_token) else 'missing'} "
f"xc_token={'set' if bool(args.modelhub_token) else 'missing'}"
)
summary = run_daily_batches(base_args=args)
print(f"daily_run_dir={summary['dailyRunDir']}")
print(f"submitted_total={summary['submittedTotal']}")
print(f"attempted_waves={summary['attemptedWaves']}")
print(f"stopped_reason={summary['stoppedReason']}")
return 0
if __name__ == "__main__":
raise SystemExit(main())

229
modelhub_submmit_api/hf_discovery.py Normal file
View File

@@ -0,0 +1,229 @@
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor, as_completed
import os
import threading
from pathlib import PurePosixPath
from typing import Any
from common import parse_datetime
from http_json import JsonHttpClient
from models import HFModelSummary, ModelInspection
GGUF_PRIORITY = ("q4_0.gguf", "q8_0.gguf", "fp16.gguf")
VLLM_WEIGHT_SUFFIXES = (".safetensors", ".bin", ".pth")
ONNX_WEIGHT_SUFFIXES = (".onnx",)
class HuggingFaceDiscovery:
def __init__(
self,
base_url: str = "https://huggingface.co",
http_client: JsonHttpClient | None = None,
timeout: int = 30,
retries: int = 2,
) -> None:
hf_token = os.getenv("HF_TOKEN")
headers = {"User-Agent": "modelhub-submmit-cli/0.1"}
if hf_token:
headers["Authorization"] = f"Bearer {hf_token}"
self.http_client = http_client or JsonHttpClient(
base_url=base_url,
default_headers=headers,
timeout=timeout,
retries=retries,
)
self._repo_tree_cache: dict[str, list[dict[str, Any]]] = {}
self._repo_tree_lock = threading.Lock()
def list_recent_models(
self,
*,
pipeline_tags: list[str],
limit: int,
min_downloads: int,
updated_after=None,
read_concurrency: int = 1,
) -> list[HFModelSummary]:
if not pipeline_tags:
return []
deduped: dict[str, HFModelSummary] = {}
max_workers = min(len(pipeline_tags), max(1, read_concurrency))
if max_workers <= 1:
tag_results = [
self._query_recent_models(
pipeline_tag=pipeline_tag,
limit=limit,
min_downloads=min_downloads,
updated_after=updated_after,
)
for pipeline_tag in pipeline_tags
]
else:
with ThreadPoolExecutor(max_workers=max_workers) as executor:
futures = {
executor.submit(
self._query_recent_models,
pipeline_tag=pipeline_tag,
limit=limit,
min_downloads=min_downloads,
updated_after=updated_after,
): pipeline_tag
for pipeline_tag in pipeline_tags
}
tag_results = []
for future in as_completed(futures):
tag_results.append(future.result())
for models in tag_results:
for model in models:
current = deduped.get(model.repo_id)
if current is None or (model.last_modified or parse_datetime("1970-01-01")) > (
current.last_modified or parse_datetime("1970-01-01")
):
deduped[model.repo_id] = model
models = list(deduped.values())
models.sort(key=lambda item: item.last_modified or parse_datetime("1970-01-01"), reverse=True)
return models
def _query_recent_models(
self,
*,
pipeline_tag: str,
limit: int,
min_downloads: int,
updated_after=None,
) -> list[HFModelSummary]:
payload = self.http_client.request_json(
"GET",
"/api/models",
query={
"pipeline_tag": pipeline_tag,
"sort": "lastModified",
"direction": "-1",
"limit": limit,
"full": "true",
},
)
models: list[HFModelSummary] = []
for item in payload or []:
model = self._parse_model(item, min_downloads=min_downloads, updated_after=updated_after)
if model is not None:
models.append(model)
return models
def list_recent_text_generation_models(
self,
*,
limit: int,
min_downloads: int,
updated_after=None,
) -> list[HFModelSummary]:
return self.list_recent_models(
pipeline_tags=["text-generation"],
limit=limit,
min_downloads=min_downloads,
updated_after=updated_after,
)
def inspect_model(self, model: HFModelSummary) -> ModelInspection:
entries = self.list_repo_tree(model.repo_id)
return inspect_repo_tree(model.repo_id, entries)
def list_repo_tree(self, repo_id: str) -> list[dict[str, Any]]:
with self._repo_tree_lock:
cached = self._repo_tree_cache.get(repo_id)
if cached is not None:
return list(cached)
payload = self.http_client.request_json(
"GET",
f"/api/models/{repo_id}/tree/main",
query={"recursive": "1"},
)
entries: list[dict[str, Any]]
if isinstance(payload, list):
entries = payload
elif isinstance(payload, dict):
for key in ("items", "tree", "siblings"):
value = payload.get(key)
if isinstance(value, list):
entries = value
break
else:
entries = []
else:
entries = []
with self._repo_tree_lock:
self._repo_tree_cache[repo_id] = list(entries)
return list(entries)
@staticmethod
def _parse_model(item: dict[str, Any], *, min_downloads: int, updated_after=None) -> HFModelSummary | None:
repo_id = item.get("id") or item.get("modelId")
if not repo_id:
return None
downloads = int(item.get("downloads") or 0)
if downloads < min_downloads:
return None
pipeline_tag = item.get("pipeline_tag") or item.get("pipelineTag")
last_modified = parse_datetime(item.get("lastModified") or item.get("last_modified"))
if updated_after and last_modified and last_modified < updated_after:
return None
return HFModelSummary(
repo_id=repo_id,
downloads=downloads,
last_modified=last_modified,
pipeline_tag=pipeline_tag,
created_at=parse_datetime(item.get("createdAt") or item.get("created_at")),
)
def inspect_repo_tree(repo_id: str, entries: list[dict[str, Any]]) -> ModelInspection:
file_paths: list[str] = []
gguf_files: list[str] = []
vllm_weight_files: list[str] = []
onnx_files: list[str] = []
for entry in entries:
path = entry.get("path") or entry.get("rfilename") or entry.get("name")
if not path:
continue
entry_type = (entry.get("type") or "").lower()
if entry_type in {"directory", "dir", "folder"}:
continue
file_paths.append(path)
filename = PurePosixPath(path).name.lower()
if any(filename.endswith(suffix) for suffix in GGUF_PRIORITY):
gguf_files.append(path)
if filename.endswith(VLLM_WEIGHT_SUFFIXES):
vllm_weight_files.append(path)
if filename.endswith(ONNX_WEIGHT_SUFFIXES):
onnx_files.append(path)
selected_gguf = choose_best_gguf(gguf_files)
return ModelInspection(
repo_id=repo_id,
file_paths=sorted(file_paths),
gguf_files=sorted(gguf_files),
selected_gguf=PurePosixPath(selected_gguf).name if selected_gguf else None,
weight_files=sorted(vllm_weight_files),
onnx_files=sorted(onnx_files),
)
def choose_best_gguf(paths: list[str]) -> str | None:
if not paths:
return None
ranked_paths = sorted(paths)
for suffix in GGUF_PRIORITY:
for path in ranked_paths:
if PurePosixPath(path).name.lower().endswith(suffix):
return path
return None

339
modelhub_submmit_api/history_stats.py Normal file
View File

@@ -0,0 +1,339 @@
from __future__ import annotations
from collections import Counter, defaultdict
from datetime import datetime
from pathlib import Path
from typing import Any
from common import append_jsonl, ensure_utc, parse_datetime, read_jsonl, write_jsonl
WAITING_STATUSES = {"waiting", "queued"}
RUNNING_STATUSES = {"running", "processing"}
HISTORY_STATS_THRESHOLD_DEFAULT = 500
HISTORY_ARCHIVE_LIMIT_DEFAULT = 5000
def load_ledger(path: Path) -> list[dict[str, Any]]:
return read_jsonl(path)
def append_ledger_entry(path: Path, entry: dict[str, Any]) -> None:
append_jsonl(path, entry)
def load_history_archive(path: Path) -> list[dict[str, Any]]:
return read_jsonl(path)
def update_history_archive(
path: Path,
tasks: list[dict[str, Any]],
*,
limit: int = HISTORY_ARCHIVE_LIMIT_DEFAULT,
) -> list[dict[str, Any]]:
merged = merge_history_records(load_history_archive(path), tasks, limit=limit)
path.parent.mkdir(parents=True, exist_ok=True)
write_jsonl(path, merged)
return merged
def merge_history_records(
existing: list[dict[str, Any]],
new_records: list[dict[str, Any]],
*,
limit: int = HISTORY_ARCHIVE_LIMIT_DEFAULT,
) -> list[dict[str, Any]]:
by_key: dict[str, dict[str, Any]] = {}
for record in [*existing, *new_records]:
key = history_record_key(record)
previous = by_key.get(key)
if previous is None or history_record_sort_key(record) >= history_record_sort_key(previous):
by_key[key] = record
merged = sorted(by_key.values(), key=history_record_sort_key, reverse=True)
if limit > 0:
merged = merged[:limit]
return merged
def history_record_key(record: dict[str, Any]) -> str:
task_id = record.get("taskId")
if task_id is not None:
return f"task:{task_id}"
model_id = record.get("modelId") or record.get("modelAddress") or "unknown"
gpu_type = record.get("gpuType") or record.get("targetGpu") or "unknown"
framework = record.get("framework") or "unknown"
return f"fallback:{model_id}|{gpu_type}|{framework}"
def history_record_sort_key(record: dict[str, Any]) -> tuple[float, str]:
timestamp = (
parse_datetime(record.get("updateTime"))
or parse_datetime(record.get("createTime"))
or parse_datetime(record.get("submitTime"))
)
return ((timestamp.timestamp() if timestamp else 0.0), str(record.get("taskId") or ""))
def should_use_history_stats(ledger_entries: list[dict[str, Any]], threshold: int = HISTORY_STATS_THRESHOLD_DEFAULT) -> bool:
return len(ledger_entries) >= threshold
def build_empty_pre_submit_report(
*,
window_days: int,
generated_at: datetime,
task_type: str | None = None,
target_gpu: str | None = None,
reason: str | None = None,
ledger_entries: int = 0,
) -> dict[str, Any]:
empty_summary = summarize_records([])
return {
"generatedAt": generated_at.isoformat(),
"statsWindowDays": window_days,
"taskType": task_type,
"taskTypeSummary": empty_summary,
"taskTypeSummaries": {},
"targetGpu": target_gpu,
"targetGpuSummary": empty_summary,
"gpuSummaries": {},
"totals": empty_summary,
"combinationStats": {},
"warnings": [] if reason is None else [reason],
"recordsAnalyzed": 0,
"ledgerEntries": ledger_entries,
"historyStatsEnabled": False,
"historyStatsReason": reason,
}
def build_pre_submit_report(
*,
tasks: list[dict[str, Any]],
ledger_entries: list[dict[str, Any]],
window_days: int,
generated_at: datetime,
target_gpu: str | None = None,
task_type: str | None = None,
) -> dict[str, Any]:
ledger_by_task_id = {str(entry["taskId"]): entry for entry in ledger_entries if entry.get("taskId")}
enriched_tasks = [enrich_task(task, ledger_by_task_id) for task in tasks]
grouped_by_combo: dict[tuple[str, str, str], list[dict[str, Any]]] = defaultdict(list)
grouped_by_gpu: dict[str, list[dict[str, Any]]] = defaultdict(list)
grouped_by_task_type: dict[str, list[dict[str, Any]]] = defaultdict(list)
for task in enriched_tasks:
combo_key = (
task.get("taskType") or "unknown",
task.get("gpuType") or "unknown",
task.get("framework") or "unknown",
)
grouped_by_combo[combo_key].append(task)
grouped_by_gpu[combo_key[1]].append(task)
grouped_by_task_type[combo_key[0]].append(task)
combination_stats = {
"|".join(combo_key): {
"taskType": combo_key[0],
"gpuType": combo_key[1],
"framework": combo_key[2],
**summarize_records(records),
}
for combo_key, records in grouped_by_combo.items()
}
gpu_summaries = {
gpu_type: summarize_records(records)
for gpu_type, records in grouped_by_gpu.items()
}
task_type_summaries = {
current_task_type: summarize_records(records)
for current_task_type, records in grouped_by_task_type.items()
}
warnings = build_warnings(gpu_summaries, combination_stats)
selected_gpu_summary = gpu_summaries.get(target_gpu, summarize_records([])) if target_gpu else None
selected_task_summary = task_type_summaries.get(task_type, summarize_records([])) if task_type else None
return {
"generatedAt": generated_at.isoformat(),
"statsWindowDays": window_days,
"taskType": task_type,
"taskTypeSummary": selected_task_summary,
"taskTypeSummaries": task_type_summaries,
"targetGpu": target_gpu,
"targetGpuSummary": selected_gpu_summary,
"gpuSummaries": gpu_summaries,
"totals": summarize_records(enriched_tasks),
"combinationStats": combination_stats,
"warnings": warnings,
"recordsAnalyzed": len(enriched_tasks),
"ledgerEntries": len(ledger_entries),
}
def enrich_task(task: dict[str, Any], ledger_by_task_id: dict[str, dict[str, Any]]) -> dict[str, Any]:
enriched = dict(task)
task_id = enriched.get("taskId")
ledger_entry = ledger_by_task_id.get(str(task_id)) if task_id is not None else None
if ledger_entry:
enriched.setdefault("taskType", ledger_entry.get("taskType"))
enriched.setdefault("framework", ledger_entry.get("framework"))
enriched.setdefault("templateId", ledger_entry.get("templateId"))
enriched.setdefault("taskType", "unknown")
enriched.setdefault("framework", "unknown")
return enriched
def summarize_records(records: list[dict[str, Any]]) -> dict[str, Any]:
total = len(records)
success_count = 0
failure_count = 0
pending_count = 0
running_count = 0
failure_breakdown: Counter[str] = Counter()
status_counter: Counter[str] = Counter()
verify_counter: Counter[str] = Counter()
for record in records:
status = str(record.get("status") or "unknown").lower()
verify_result = record.get("verifyResult")
status_counter[status] += 1
verify_counter[str(verify_result)] += 1
if is_success(record):
success_count += 1
elif is_failure(record):
failure_count += 1
failure_breakdown[classify_failure(record)] += 1
else:
pending_count += 1
failure_breakdown[classify_failure(record)] += 1
if status in RUNNING_STATUSES:
running_count += 1
waiting_count = sum(status_counter[status] for status in WAITING_STATUSES)
return {
"total": total,
"successCount": success_count,
"failureCount": failure_count,
"pendingCount": pending_count,
"runningCount": running_count,
"waitingCount": waiting_count,
"successRate": ratio(success_count, total),
"failureRate": ratio(failure_count, total),
"pendingRate": ratio(pending_count, total),
"statusCounts": dict(status_counter),
"verifyResultCounts": dict(verify_counter),
"failureBreakdown": dict(failure_breakdown),
}
def is_success(record: dict[str, Any]) -> bool:
verify_result = record.get("verifyResult")
status = str(record.get("status") or "").lower()
return verify_result is not None and verify_result > 0 and status == "success"
def is_failure(record: dict[str, Any]) -> bool:
verify_result = record.get("verifyResult")
status = str(record.get("status") or "").lower()
if verify_result is not None and verify_result < 0:
return True
return status in {"failed", "error", "cancelled"}
def classify_failure(record: dict[str, Any]) -> str:
status = str(record.get("status") or "").lower()
verify_result = record.get("verifyResult")
if status in {"failed", "error", "cancelled"}:
return "参数/模板问题"
if status in WAITING_STATUSES or status in RUNNING_STATUSES or verify_result is None:
return "排队中/未知"
if verify_result is not None and verify_result < 0:
if not record.get("logCosUrl") and not record.get("logSyncStatus"):
return "日志缺失"
return "验证失败"
return "排队中/未知"
def build_warnings(gpu_summaries: dict[str, dict[str, Any]], combination_stats: dict[str, dict[str, Any]]) -> list[str]:
warnings: list[str] = []
for gpu_type, target_gpu_summary in gpu_summaries.items():
if target_gpu_summary["total"] >= 4 and target_gpu_summary["failureRate"] >= 0.5:
warnings.append(f"GPU {gpu_type} 最近 7 天失败率偏高。")
if target_gpu_summary["total"] >= 4 and target_gpu_summary["pendingRate"] >= 0.5:
warnings.append(f"GPU {gpu_type} 最近 7 天排队/未知任务占比较高。")
for stat in combination_stats.values():
if stat["framework"] == "unknown":
continue
if stat["total"] >= 3 and stat["failureRate"] >= 0.6:
warnings.append(f"框架 {stat['framework']} 在 {stat['gpuType']} 的任务 {stat['taskType']} 上近期失败集中。")
deduped: list[str] = []
for warning in warnings:
if warning not in deduped:
deduped.append(warning)
return deduped
def score_candidate(report: dict[str, Any], *, framework: str, task_type: str, target_gpu: str) -> tuple[float, list[str]]:
key = f"{task_type}|{target_gpu}|{framework}"
combo = (report.get("combinationStats") or {}).get(key)
gpu_summaries = report.get("gpuSummaries") or {}
target_gpu_summary = gpu_summaries.get(target_gpu) or {}
task_type_summary = (report.get("taskTypeSummaries") or {}).get(task_type) or {}
score = 100.0
warnings = list(report.get("warnings") or [])
if combo:
score += combo["successRate"] * 25.0
score -= combo["failureRate"] * 40.0
score -= combo["pendingRate"] * 15.0
if target_gpu_summary:
score -= target_gpu_summary.get("pendingRate", 0.0) * 10.0
score -= target_gpu_summary.get("failureRate", 0.0) * 10.0
if task_type_summary:
score += task_type_summary.get("successRate", 0.0) * 10.0
score -= task_type_summary.get("failureRate", 0.0) * 10.0
return round(score, 2), warnings
def count_submissions_for_day(
*,
tasks: list[dict[str, Any]],
ledger_entries: list[dict[str, Any]],
day_start: datetime,
day_end: datetime,
) -> dict[str, Any]:
day_start = ensure_utc(day_start)
day_end = ensure_utc(day_end)
task_ids = {
str(task.get("taskId"))
for task in tasks
if task.get("taskId") is not None
}
task_count = len(tasks)
ledger_only = 0
for entry in ledger_entries:
submit_time = parse_datetime(entry.get("submitTime"))
if submit_time is None or submit_time < day_start or submit_time > day_end:
continue
task_id = str(entry.get("taskId")) if entry.get("taskId") is not None else None
if task_id and task_id in task_ids:
continue
ledger_only += 1
return {
"platformTaskCount": task_count,
"ledgerOnlyCount": ledger_only,
"totalCount": task_count + ledger_only,
}
def ratio(part: int, whole: int) -> float:
if whole <= 0:
return 0.0
return round(part / whole, 4)

115
modelhub_submmit_api/http_json.py Normal file
View File

@@ -0,0 +1,115 @@
from __future__ import annotations
import json
import socket
import time
from dataclasses import dataclass
from http.client import IncompleteRead, RemoteDisconnected
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.parse import urlencode, urljoin
from urllib.request import OpenerDirector, Request, build_opener
@dataclass
class HttpJsonError(RuntimeError):
message: str
status_code: int | None = None
payload: Any = None
def __str__(self) -> str:
if self.status_code is None:
return self.message
return f"{self.message} (status={self.status_code})"
class JsonHttpClient:
def __init__(
self,
base_url: str = "",
default_headers: dict[str, str] | None = None,
timeout: int = 30,
retries: int = 2,
backoff_seconds: float = 1.0,
opener: OpenerDirector | None = None,
) -> None:
self.base_url = base_url.rstrip("/")
self.default_headers = default_headers or {}
self.timeout = timeout
self.retries = retries
self.backoff_seconds = backoff_seconds
self.opener = opener or build_opener()
def request_json(
self,
method: str,
path: str,
*,
query: dict[str, Any] | None = None,
data: dict[str, Any] | None = None,
headers: dict[str, str] | None = None,
retryable_statuses: set[int] | None = None,
) -> Any:
retryable_statuses = retryable_statuses or {429, 500, 502, 503, 504}
url = self._build_url(path, query)
merged_headers = {
"Accept": "application/json",
**self.default_headers,
**(headers or {}),
}
body = None
if data is not None:
body = json.dumps(data).encode("utf-8")
merged_headers.setdefault("Content-Type", "application/json")
for attempt in range(self.retries + 1):
request = Request(url=url, data=body, headers=merged_headers, method=method.upper())
try:
with self.opener.open(request, timeout=self.timeout) as response:
raw = response.read().decode("utf-8")
if not raw:
return None
try:
return json.loads(raw)
except json.JSONDecodeError as exc:
raise HttpJsonError(f"Invalid JSON response from {url}", payload=raw) from exc
except HTTPError as exc:
payload = self._decode_error_payload(exc)
if exc.code in retryable_statuses and attempt < self.retries:
time.sleep(self.backoff_seconds * (attempt + 1))
continue
raise HttpJsonError(f"HTTP request failed for {url}", status_code=exc.code, payload=payload) from exc
except (URLError, TimeoutError, socket.timeout, RemoteDisconnected, IncompleteRead) as exc:
if attempt < self.retries:
time.sleep(self.backoff_seconds * (attempt + 1))
continue
raise HttpJsonError(f"Network request failed for {url}") from exc
raise HttpJsonError(f"Request exhausted retries for {url}")
def _build_url(self, path: str, query: dict[str, Any] | None) -> str:
if path.startswith("http://") or path.startswith("https://"):
url = path
else:
base = self.base_url + "/"
url = urljoin(base, path.lstrip("/"))
if query:
pairs = [(key, value) for key, value in query.items() if value is not None]
encoded = urlencode(pairs)
if encoded:
separator = "&" if "?" in url else "?"
url = f"{url}{separator}{encoded}"
return url
@staticmethod
def _decode_error_payload(exc: HTTPError) -> Any:
try:
raw = exc.read().decode("utf-8")
except Exception:
return None
if not raw:
return None
try:
return json.loads(raw)
except json.JSONDecodeError:
return raw

681
modelhub_submmit_api/main.py Normal file
View File

@@ -0,0 +1,681 @@
from __future__ import annotations
import argparse
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import timedelta
from pathlib import Path
from typing import Any
from common import parse_datetime, utc_now, write_json, write_jsonl
from hf_discovery import HuggingFaceDiscovery
from history_stats import (
append_ledger_entry,
build_empty_pre_submit_report,
count_submissions_for_day,
load_ledger,
update_history_archive,
)
from modelhub_client import ModelHubAPIError, ModelHubClient
from models import CandidateModel, HFModelSummary, ModelInspection
from outcome_tracker import DEFAULT_OUTCOMES_PATH, OutcomeTracker
from task_registry import TASK_SPEC_BY_TYPE, all_task_types, choose_framework_for_task, choose_text_generation_framework, pipeline_tags_for_task_types, task_specs_for_model
from template_selector import TemplateSelector
DEFAULT_RUNS_DIR = Path("runs")
DEFAULT_LEDGER_PATH = Path("ledger/submissions.jsonl")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Auto discover and submit public Hugging Face models to ModelHub.")
parser.add_argument("--gpu", help="Single GPU alias or platform name, for example: k100")
parser.add_argument("--gpus", help="Comma-separated GPU aliases/platform names. Omit to auto-use all safe GPUs from templates.")
parser.add_argument("--task-types", help="Comma-separated ModelHub task types. Omit to auto-enable all supported task types.")
parser.add_argument("--limit", type=int, default=300, help="Maximum number of Hugging Face models to scan per pipeline tag")
parser.add_argument("--max-scan-models", type=int, default=0, help="Hard cap on total scanned models (0 means auto)")
parser.add_argument(
"--scan-multiplier",
type=int,
default=4,
help="Multiplier used when auto-deriving scan limit from quota/queue capacity",
)
parser.add_argument("--min-downloads", type=int, default=50, help="Minimum downloads threshold")
parser.add_argument("--daily-target", type=int, default=0, help="Daily submission target across all auto runs; 0 means unlimited")
parser.add_argument("--dry-run", action="store_true", help="Only write artifacts without creating tasks")
parser.add_argument("--skip-outcome-sync", action="store_true", help="Skip outcome sync from ModelHub before scanning")
parser.add_argument("--skip-history-archive", action="store_true", help="Skip historical task archive download for this run")
group = parser.add_mutually_exclusive_group()
group.add_argument("--since-hours", type=int, help="Only scan models updated within the last N hours")
group.add_argument("--updated-after", help="Only scan models updated after the given UTC time")
parser.add_argument("--stats-window-days", type=int, default=7, help="History window in days")
parser.add_argument(
"--history-stats-threshold",
type=int,
default=500,
help="Minimum local ledger records before using history stats to rank submissions",
)
parser.add_argument("--read-concurrency", type=int, default=4, help="Concurrency for read-only remote calls")
parser.add_argument(
"--submit-concurrency",
type=int,
default=0,
help="Concurrency for task submission calls (0 = auto based on token/client count)",
)
parser.add_argument(
"--max-submits-per-run",
type=int,
default=0,
help="Maximum tasks to submit in one run (0 means unlimited)",
)
parser.add_argument("--runs-dir", default=str(DEFAULT_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--ledger-path", default=str(DEFAULT_LEDGER_PATH), help=argparse.SUPPRESS)
parser.add_argument("--outcomes-path", default=str(DEFAULT_OUTCOMES_PATH), help=argparse.SUPPRESS)
parser.add_argument("--history-archive-path", default="history/platform_tasks.jsonl", help=argparse.SUPPRESS)
parser.add_argument("--history-archive-limit", type=int, default=5000, help=argparse.SUPPRESS)
parser.add_argument("--hf-base-url", default=os.getenv("HF_BASE_URL", "https://huggingface.co"), help=argparse.SUPPRESS)
parser.add_argument("--modelhub-base-url", default=os.getenv("MODELHUB_BASE_URL", "https://modelhub.org.cn"), help=argparse.SUPPRESS)
parser.add_argument("--modelhub-token", default=os.getenv("MODELHUB_XC_TOKEN") or os.getenv("XC_TOKEN"), help=argparse.SUPPRESS)
return parser
def determine_updated_after(args: argparse.Namespace, now) -> Any:
if args.since_hours is not None:
return now - timedelta(hours=args.since_hours)
if args.updated_after:
parsed = parse_datetime(args.updated_after)
if parsed is None:
raise ValueError(f"Invalid --updated-after value: {args.updated_after}")
return parsed
return None
def resolve_task_types(args: argparse.Namespace) -> list[str]:
if not args.task_types:
return all_task_types()
selected = [value.strip() for value in args.task_types.split(",") if value.strip()]
unknown = [task_type for task_type in selected if task_type not in TASK_SPEC_BY_TYPE]
if unknown:
raise ValueError(f"Unknown task types: {', '.join(sorted(unknown))}")
return selected
def resolve_target_gpus(args: argparse.Namespace, selector: TemplateSelector, task_types: list[str]) -> list[str]:
raw_values: list[str] = []
if args.gpus:
raw_values.extend(value.strip() for value in args.gpus.split(",") if value.strip())
if args.gpu:
raw_values.append(args.gpu)
if raw_values:
normalized: list[str] = []
for value in raw_values:
target_gpu = selector.normalize_gpu(value)
if target_gpu not in normalized:
normalized.append(target_gpu)
return normalized
discovered: list[str] = []
for task_type in task_types:
for target_gpu in selector.supported_target_gpus(task_type, auto_only=True):
if target_gpu not in discovered:
discovered.append(target_gpu)
return discovered
def resolve_scan_limit(
args: argparse.Namespace,
*,
target_gpu_count: int,
remaining_daily_quota: int,
platform_available_slots: int | None,
) -> int:
base_limit = max(1, args.limit)
explicit_max = max(0, int(getattr(args, "max_scan_models", 0) or 0))
if explicit_max > 0:
return min(base_limit, explicit_max)
multiplier = max(1, int(getattr(args, "scan_multiplier", 4) or 1))
if args.daily_target > 0:
needed = max(1, remaining_daily_quota)
elif platform_available_slots is not None and platform_available_slots > 0:
needed = max(1, platform_available_slots)
else:
needed = max(200, target_gpu_count * 20)
max_submits_per_run = max(0, int(getattr(args, "max_submits_per_run", 0) or 0))
if max_submits_per_run > 0:
needed = min(needed, max_submits_per_run)
return min(base_limit, needed * multiplier)
def choose_candidate_for_gpu(
*,
model: HFModelSummary,
inspection: ModelInspection,
template_selector: TemplateSelector,
task_types: list[str],
target_gpu: str,
) -> CandidateModel | None:
for task_type in task_types:
supported_frameworks = template_selector.supported_frameworks_for_auto(task_type, target_gpu)
if not supported_frameworks:
continue
try:
framework = choose_framework_for_task(task_type, target_gpu, supported_frameworks, inspection)
template = template_selector.select_template(task_type, framework, target_gpu)
config_params = template_selector.render_config(
template,
gguf_filename=inspection.selected_gguf if framework == "llamacpp" else None,
)
except ValueError:
continue
spec = TASK_SPEC_BY_TYPE[task_type]
return CandidateModel(
repo_id=model.repo_id,
model_address=model.model_address,
pipeline_tag=model.pipeline_tag,
modality=spec.modality,
task_type=task_type,
target_gpu=target_gpu,
framework=framework,
template_id=template.template_id,
config_params=config_params,
downloads=model.downloads,
last_modified=model.last_modified,
gguf_filename=inspection.selected_gguf,
score=0.0,
warnings=[],
)
return None
def resolve_submit_concurrency(
args: argparse.Namespace,
*,
modelhub_client: ModelHubClient | Any,
planned_submit_count: int,
) -> int:
explicit_workers = int(getattr(args, "submit_concurrency", 0) or 0)
if explicit_workers > 0:
return explicit_workers
clients = getattr(modelhub_client, "clients", None)
if isinstance(clients, list) and clients:
if planned_submit_count > 0:
return max(1, min(len(clients), planned_submit_count))
return len(clients)
return 1
def resolve_max_submit_count(
args: argparse.Namespace,
planned_count: int,
remaining_daily_quota: int,
) -> int:
explicit_limit = int(getattr(args, "max_submits_per_run", 0) or 0)
planned_submit_count = planned_count
if remaining_daily_quota > 0:
planned_submit_count = min(planned_submit_count, remaining_daily_quota)
if explicit_limit > 0:
planned_submit_count = min(planned_submit_count, explicit_limit)
return planned_submit_count
def process_model_for_candidates(
*,
model: HFModelSummary,
hf_discovery: HuggingFaceDiscovery,
modelhub_client: ModelHubClient,
template_selector: TemplateSelector,
target_gpus: list[str],
allowed_task_types: list[str],
outcome_tracker: OutcomeTracker | None = None,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]], list[dict[str, Any]]]:
specs = [spec for spec in task_specs_for_model(model) if spec.task_type in allowed_task_types]
if not specs:
return [], [{"repoId": model.repo_id, "reason": f"unsupported_pipeline_tag:{model.pipeline_tag or 'unknown'}"}], []
processed_gpus = modelhub_client.processed_gpus_for_model(model.repo_id)
candidates: list[dict[str, Any]] = []
skipped: list[dict[str, Any]] = []
failed: list[dict[str, Any]] = []
allowed_task_types_set = [spec.task_type for spec in specs]
pending_task_types_by_gpu: list[tuple[str, list[str]]] = []
for target_gpu in target_gpus:
if target_gpu in processed_gpus:
skipped.append({"repoId": model.repo_id, "targetGpu": target_gpu, "reason": "already_processed_for_gpu"})
continue
if outcome_tracker and outcome_tracker.is_model_gpu_failed(model.repo_id, target_gpu):
skipped.append({"repoId": model.repo_id, "targetGpu": target_gpu, "reason": "previously_failed_locally"})
continue
compatible_task_types = [
task_type
for task_type in allowed_task_types_set
if template_selector.supported_frameworks_for_auto(task_type, target_gpu)
]
if not compatible_task_types:
skipped.append({"repoId": model.repo_id, "targetGpu": target_gpu, "reason": "no_compatible_auto_template_or_framework"})
continue
pending_task_types_by_gpu.append((target_gpu, compatible_task_types))
if not pending_task_types_by_gpu:
return candidates, skipped, failed
try:
inspection = hf_discovery.inspect_model(model)
except Exception as exc:
return [], skipped, [{"repoId": model.repo_id, "reason": str(exc)}]
for target_gpu, task_types in pending_task_types_by_gpu:
best = choose_candidate_for_gpu(
model=model,
inspection=inspection,
template_selector=template_selector,
task_types=task_types,
target_gpu=target_gpu,
)
if best is None:
skipped.append({"repoId": model.repo_id, "targetGpu": target_gpu, "reason": "no_compatible_auto_template_or_framework"})
continue
candidates.append(candidate_to_record(best))
return candidates, skipped, failed
def submit_candidate(
candidate: dict[str, Any],
modelhub_client: ModelHubClient,
) -> dict[str, Any]:
payload = {
"modelAddress": candidate["modelAddress"],
"taskType": candidate["taskType"],
"targetGpu": candidate["targetGpu"],
"framework": candidate["framework"],
"configParams": candidate["configParams"],
}
strategy_id = os.getenv("STRATEGY_ID")
if strategy_id:
payload["strategyId"] = strategy_id
submit_time = utc_now()
try:
response = modelhub_client.add_task(payload)
task_id = extract_task_id_from_submit_response(response)
if task_id is None:
task_id = modelhub_client.find_recent_task_id(candidate["repoId"], candidate["targetGpu"], submit_time)
return {
"outcome": "submitted",
"candidate": candidate,
"submitTime": submit_time.isoformat(),
"taskId": task_id,
"responseData": response.get("data"),
}
except ModelHubAPIError as exc:
return {
"outcome": "failed",
"candidate": candidate,
"reason": str(exc),
}
def make_run_dir(runs_dir: Path, now) -> Path:
base_name = now.strftime("%Y%m%dT%H%M%SZ")
run_dir = runs_dir / base_name
if not run_dir.exists():
return run_dir
for suffix in range(1, 1000):
candidate = runs_dir / f"{base_name}.{suffix:03d}"
if not candidate.exists():
return candidate
return candidate
def run_submission(
args: argparse.Namespace,
*,
now=None,
hf_discovery: HuggingFaceDiscovery | None = None,
modelhub_client: ModelHubClient | None = None,
template_selector: TemplateSelector | None = None,
outcome_tracker: OutcomeTracker | None = None,
) -> dict[str, Any]:
now = now or utc_now()
template_selector = template_selector or TemplateSelector()
selected_task_types = resolve_task_types(args)
target_gpus = resolve_target_gpus(args, template_selector, selected_task_types)
if not target_gpus:
raise RuntimeError("No auto-submittable GPUs are available for the selected task types")
hf_discovery = hf_discovery or HuggingFaceDiscovery(base_url=args.hf_base_url)
modelhub_client = modelhub_client or ModelHubClient(token=args.modelhub_token, base_url=args.modelhub_base_url)
runs_dir = Path(args.runs_dir)
ledger_path = Path(args.ledger_path)
history_archive_path = Path(args.history_archive_path)
run_dir = make_run_dir(runs_dir, now)
run_dir.mkdir(parents=True, exist_ok=True)
ledger_path.parent.mkdir(parents=True, exist_ok=True)
history_archive_path.parent.mkdir(parents=True, exist_ok=True)
outcome_tracker = outcome_tracker or OutcomeTracker(Path(args.outcomes_path))
synced_count = 0
if not getattr(args, "skip_outcome_sync", False):
try:
synced_count = outcome_tracker.sync_from_api(modelhub_client)
except Exception:
pass
updated_after = determine_updated_after(args, now)
history_begin = now - timedelta(days=args.stats_window_days)
ledger_entries = load_ledger(ledger_path)
day_start = now.replace(hour=0, minute=0, second=0, microsecond=0)
# Count today's submissions from the local ledger (avoids expensive paginated API call)
daily_snapshot = count_submissions_for_day(tasks=[], ledger_entries=ledger_entries, day_start=day_start, day_end=now)
# Fallback: if ledger has no entries yet, do a quick API check (limit to 1 page)
if daily_snapshot["totalCount"] <= 0:
today_tasks_page = modelhub_client.list_tasks_page(
current=1,
page_size=20,
only_mine=True,
begin_time=day_start,
end_time=now,
)
today_tasks = (today_tasks_page.get("data") or {}).get("records") or []
daily_snapshot = count_submissions_for_day(tasks=today_tasks, ledger_entries=ledger_entries, day_start=day_start, day_end=now)
platform_available_slots = None
if hasattr(modelhub_client, "available_submit_slots"):
platform_available_slots = max(0, int(modelhub_client.available_submit_slots()))
unlimited_daily_target = args.daily_target <= 0
if unlimited_daily_target:
remaining_daily_quota = platform_available_slots if platform_available_slots is not None else 1_000_000_000
else:
remaining_daily_quota = max(0, args.daily_target - daily_snapshot["totalCount"])
if platform_available_slots is not None:
remaining_daily_quota = min(remaining_daily_quota, platform_available_slots)
history_report_reason = "history_archive_skipped" if getattr(args, "skip_history_archive", False) else "history_archive_only_mode"
if args.daily_target > 0 and remaining_daily_quota <= 0:
archived_history: list[dict[str, Any]] = []
report = build_empty_pre_submit_report(
window_days=args.stats_window_days,
generated_at=now,
reason="daily_target_reached_before_scan",
ledger_entries=len(ledger_entries),
)
write_json(run_dir / "pre_submit_report.json", report)
return {
"generatedAt": now.isoformat(),
"dryRun": bool(args.dry_run),
"selectedTaskTypes": selected_task_types,
"targetGpus": target_gpus,
"dailyTarget": args.daily_target,
"unlimitedDailyTarget": unlimited_daily_target,
"submittedTodayBeforeRun": daily_snapshot["totalCount"],
"remainingDailyQuotaBeforeRun": remaining_daily_quota,
"platformAvailableSlotsBeforeRun": platform_available_slots,
"historyStatsEnabled": False,
"historyStatsThreshold": getattr(args, "history_stats_threshold", 500),
"historyArchivePath": str(history_archive_path),
"historyArchiveRecordCount": len(archived_history),
"scanLimit": 0,
"scannedModels": 0,
"candidateCount": 0,
"plannedSubmitCount": 0,
"submittedCount": 0,
"skippedCount": 0,
"failedCount": 0,
"warnings": report.get("warnings", []),
"runDir": str(run_dir),
"outcomeSyncCount": synced_count,
}
if getattr(args, "skip_history_archive", False):
archived_history = []
else:
history_tasks = modelhub_client.list_tasks(
page_size=50,
only_mine=True,
begin_time=history_begin,
end_time=now,
)
archived_history = update_history_archive(
history_archive_path,
history_tasks,
limit=getattr(args, "history_archive_limit", 5000),
)
report = build_empty_pre_submit_report(
window_days=args.stats_window_days,
generated_at=now,
reason=history_report_reason,
ledger_entries=len(ledger_entries),
)
write_json(run_dir / "pre_submit_report.json", report)
scan_limit = resolve_scan_limit(
args,
target_gpu_count=len(target_gpus),
remaining_daily_quota=remaining_daily_quota,
platform_available_slots=platform_available_slots,
)
model_query_kwargs = {
"pipeline_tags": pipeline_tags_for_task_types(selected_task_types),
"limit": scan_limit,
"min_downloads": args.min_downloads,
"updated_after": updated_after,
}
try:
model_query_kwargs["read_concurrency"] = max(1, args.read_concurrency)
models = hf_discovery.list_recent_models(**model_query_kwargs)
except TypeError:
models = hf_discovery.list_recent_models(
pipeline_tags=model_query_kwargs["pipeline_tags"],
limit=model_query_kwargs["limit"],
min_downloads=model_query_kwargs["min_downloads"],
updated_after=model_query_kwargs["updated_after"],
)
candidates: list[dict[str, Any]] = []
skipped: list[dict[str, Any]] = []
failed: list[dict[str, Any]] = []
with ThreadPoolExecutor(max_workers=max(1, args.read_concurrency)) as executor:
futures = {
executor.submit(
process_model_for_candidates,
model=model,
hf_discovery=hf_discovery,
modelhub_client=modelhub_client,
template_selector=template_selector,
target_gpus=target_gpus,
allowed_task_types=selected_task_types,
outcome_tracker=outcome_tracker,
): index
for index, model in enumerate(models)
}
ordered_results: dict[int, tuple[list[dict[str, Any]], list[dict[str, Any]], list[dict[str, Any]]]] = {}
for future in as_completed(futures):
index = futures[future]
model = models[index]
try:
ordered_results[index] = future.result()
except Exception as exc:
ordered_results[index] = ([], [], [{"repoId": model.repo_id, "reason": str(exc)}])
for index in range(len(models)):
model_candidates, model_skipped, model_failed = ordered_results.get(index, ([], [], []))
candidates.extend(model_candidates)
skipped.extend(model_skipped)
failed.extend(model_failed)
write_jsonl(run_dir / "candidates.jsonl", candidates)
submitted: list[dict[str, Any]] = []
planned_submit_count = resolve_max_submit_count(
args=args,
planned_count=len(candidates),
remaining_daily_quota=remaining_daily_quota,
)
planned_candidates = candidates[:planned_submit_count]
submit_workers = 1
if not args.dry_run:
submit_workers = resolve_submit_concurrency(
args,
modelhub_client=modelhub_client,
planned_submit_count=len(planned_candidates),
)
with ThreadPoolExecutor(max_workers=submit_workers) as executor:
futures = {
executor.submit(
submit_candidate,
candidate,
modelhub_client,
): index
for index, candidate in enumerate(planned_candidates)
}
ordered_results: dict[int, dict[str, Any]] = {}
for future in as_completed(futures):
index = futures[future]
try:
ordered_results[index] = future.result()
except Exception as exc:
candidate = planned_candidates[index]
ordered_results[index] = {
"outcome": "failed",
"candidate": candidate,
"reason": str(exc),
}
for index in range(len(planned_candidates)):
result = ordered_results.get(index)
if result is None:
continue
candidate = result["candidate"]
if result["outcome"] == "failed":
failed.append(
{
"repoId": candidate["repoId"],
"targetGpu": candidate["targetGpu"],
"framework": candidate["framework"],
"taskType": candidate["taskType"],
"reason": result.get("reason", "submission_failed"),
}
)
continue
submitted_record = {
**candidate,
"submitTime": result["submitTime"],
"taskId": result["taskId"],
"responseData": result["responseData"],
}
submitted.append(submitted_record)
append_ledger_entry(
ledger_path,
{
"modelId": candidate["repoId"],
"modelAddress": candidate["modelAddress"],
"targetGpu": candidate["targetGpu"],
"framework": candidate["framework"],
"templateId": candidate["templateId"],
"taskId": result["taskId"],
"taskType": candidate["taskType"],
"submitTime": result["submitTime"],
},
)
outcome_tracker.record_submission(
model_id=candidate["repoId"],
target_gpu=candidate["targetGpu"],
framework=candidate["framework"],
task_type=candidate["taskType"],
task_id=result["taskId"],
submit_time=result["submitTime"],
)
write_jsonl(run_dir / "submitted.jsonl", submitted)
write_jsonl(run_dir / "skipped.jsonl", skipped)
write_jsonl(run_dir / "failed.jsonl", failed)
outcome_tracker.save()
summary = {
"generatedAt": now.isoformat(),
"dryRun": bool(args.dry_run),
"selectedTaskTypes": selected_task_types,
"targetGpus": target_gpus,
"maxSubmitsPerRun": int(getattr(args, "max_submits_per_run", 0) or 0),
"dailyTarget": args.daily_target,
"unlimitedDailyTarget": unlimited_daily_target,
"submittedTodayBeforeRun": daily_snapshot["totalCount"],
"remainingDailyQuotaBeforeRun": remaining_daily_quota,
"platformAvailableSlotsBeforeRun": platform_available_slots,
"historyStatsEnabled": False,
"historyStatsThreshold": getattr(args, "history_stats_threshold", 500),
"historyArchivePath": str(history_archive_path),
"historyArchiveRecordCount": len(archived_history),
"scanLimit": scan_limit,
"scannedModels": len(models),
"candidateCount": len(candidates),
"plannedSubmitCount": len(planned_candidates),
"submittedCount": len(submitted),
"skippedCount": len(skipped),
"failedCount": len(failed),
"submitConcurrencyUsed": submit_workers,
"warnings": report.get("warnings", []),
"runDir": str(run_dir),
"outcomeSyncCount": synced_count,
}
write_json(run_dir / "summary.json", summary)
return summary
def candidate_to_record(candidate: CandidateModel) -> dict[str, Any]:
return {
"repoId": candidate.repo_id,
"modelAddress": candidate.model_address,
"pipelineTag": candidate.pipeline_tag,
"modality": candidate.modality,
"taskType": candidate.task_type,
"taskPriority": TASK_SPEC_BY_TYPE[candidate.task_type].priority,
"targetGpu": candidate.target_gpu,
"framework": candidate.framework,
"templateId": candidate.template_id,
"configParams": candidate.config_params,
"downloads": candidate.downloads,
"lastModified": candidate.last_modified.isoformat() if candidate.last_modified else None,
"ggufFilename": candidate.gguf_filename,
"score": candidate.score,
"warnings": candidate.warnings,
}
def extract_task_id_from_submit_response(response: dict[str, Any]) -> str | None:
data = response.get("data")
if isinstance(data, dict) and data.get("id") is not None:
return str(data["id"])
return None
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
summary = run_submission(args)
print(f"run_dir={summary['runDir']}")
print(f"daily_target={summary['dailyTarget']} remaining_before_run={summary['remainingDailyQuotaBeforeRun']}")
print(f"task_types={','.join(summary['selectedTaskTypes'])}")
print(f"gpus={','.join(summary['targetGpus'])}")
print(
f"candidates={summary['candidateCount']} planned={summary['plannedSubmitCount']} "
f"submitted={summary['submittedCount']} skipped={summary['skippedCount']} failed={summary['failedCount']}"
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

230
modelhub_submmit_api/modelhub_client.py Normal file
View File

@@ -0,0 +1,230 @@
from __future__ import annotations
import os
from datetime import datetime, timedelta
from typing import Any
from common import format_modelhub_datetime, parse_datetime
from http_json import HttpJsonError, JsonHttpClient
class ModelHubAPIError(RuntimeError):
def __init__(self, message: str, *, code: int | None = None, payload: Any = None) -> None:
super().__init__(message)
self.code = code
self.payload = payload
class ModelHubClient:
def __init__(
self,
*,
token: str | None = None,
base_url: str = "https://modelhub.org.cn",
timeout: int = 30,
retries: int = 2,
http_client: JsonHttpClient | None = None,
) -> None:
self.token = token or os.getenv("MODELHUB_XC_TOKEN") or os.getenv("XC_TOKEN")
if not self.token and not os.getenv("STRATEGY_ID") and http_client is None:
raise ValueError("ModelHub token is required. Set MODELHUB_XC_TOKEN or XC_TOKEN.")
default_headers = {"Xc-Token": self.token} if self.token else {}
self.http_client = http_client or JsonHttpClient(
base_url=base_url,
default_headers=default_headers,
timeout=timeout,
retries=retries,
)
def search_by_model_id(self, model_id: str) -> dict[str, Any]:
return self._request("GET", "/api/computility/models/search-by-model-id", query={"modelId": model_id})
def is_model_processed_for_gpu(self, model_id: str, target_gpu: str) -> bool:
gpu_result = self.get_verify_result_map(model_id).get(target_gpu)
if gpu_result is None:
return False
return "result" in gpu_result or "records" in gpu_result
def get_verify_result_map(self, model_id: str) -> dict[str, Any]:
payload = self.search_by_model_id(model_id)
return ((payload.get("data") or {}).get("verifyResult") or {})
def processed_gpus_for_model(self, model_id: str) -> set[str]:
return set(self.get_verify_result_map(model_id).keys())
def list_tasks_page(
self,
*,
current: int = 1,
page_size: int = 50,
only_mine: bool = True,
begin_time: datetime | None = None,
end_time: datetime | None = None,
gpu_type: str | None = None,
model_id: str | None = None,
) -> dict[str, Any]:
return self._request(
"GET",
"/api/adapt/task/page",
query={
"current": current,
"pageSize": page_size,
"onlyMine": str(only_mine).lower(),
"beginTime": format_modelhub_datetime(begin_time) if begin_time else None,
"endTime": format_modelhub_datetime(end_time) if end_time else None,
"gpuType": gpu_type,
"modelId": model_id,
},
)
def list_tasks(
self,
*,
page_size: int = 50,
only_mine: bool = True,
begin_time: datetime | None = None,
end_time: datetime | None = None,
gpu_type: str | None = None,
model_id: str | None = None,
) -> list[dict[str, Any]]:
current = 1
records: list[dict[str, Any]] = []
while True:
page = self.list_tasks_page(
current=current,
page_size=page_size,
only_mine=only_mine,
begin_time=begin_time,
end_time=end_time,
gpu_type=gpu_type,
model_id=model_id,
)
page_data = page.get("data") or {}
page_records = page_data.get("records") or []
records.extend(page_records)
pages = int(page_data.get("pages") or 0)
if pages <= current or not page_records:
break
current += 1
return records
def add_task(self, payload: dict[str, Any]) -> dict[str, Any]:
return self._request("POST", "/api/adapt/task/add", data=payload)
def find_recent_task_id(self, model_id: str, gpu_type: str, submitted_after: datetime) -> str | None:
recent_tasks = self.list_tasks(
page_size=20,
only_mine=True,
begin_time=submitted_after - timedelta(hours=1),
end_time=submitted_after + timedelta(hours=6),
gpu_type=gpu_type,
model_id=model_id,
)
if not recent_tasks:
return None
recent_tasks.sort(key=lambda task: parse_datetime(task.get("updateTime")) or submitted_after, reverse=True)
return str(recent_tasks[0].get("taskId")) if recent_tasks[0].get("taskId") is not None else None
def count_active_tasks(
self,
*,
page_size: int = 100,
only_mine: bool = True,
begin_time: datetime | None = None,
end_time: datetime | None = None,
max_count: int | None = None,
) -> int:
"""Count active tasks with early pagination termination.
Unlike list_tasks() which fetches all pages, this method stops
fetching pages as soon as max_count active tasks are found.
"""
max_count_int: int | None = max_count
if max_count_int is not None:
max_count_int = max(0, int(max_count_int))
if max_count_int == 0:
return 0
active_count = 0
current = 1
while True:
page = self.list_tasks_page(
current=current,
page_size=page_size,
only_mine=only_mine,
begin_time=begin_time,
end_time=end_time,
)
page_data = page.get("data") or {}
page_records = page_data.get("records") or []
for task in page_records:
if is_active_task(task):
active_count += 1
if max_count_int is not None and active_count >= max_count_int:
return active_count
pages = int(page_data.get("pages") or 0)
if pages <= current or not page_records:
break
current += 1
return active_count
def _request(
self,
method: str,
path: str,
*,
query: dict[str, Any] | None = None,
data: dict[str, Any] | None = None,
) -> dict[str, Any]:
try:
payload = self.http_client.request_json(method, path, query=query, data=data)
except HttpJsonError as exc:
if isinstance(exc.payload, dict) and "message" in exc.payload:
raise ModelHubAPIError(exc.payload["message"], payload=exc.payload, code=exc.status_code) from exc
raise ModelHubAPIError(str(exc), payload=exc.payload, code=exc.status_code) from exc
if not isinstance(payload, dict):
raise ModelHubAPIError("Unexpected API response shape", payload=payload)
code = payload.get("code")
if code != 0:
raise ModelHubAPIError(payload.get("message") or "ModelHub API request failed", code=code, payload=payload)
return payload
ACTIVE_TASK_STATUSES = {
"waiting",
"running",
"processing",
"queued",
"pending",
"validating",
"submitting",
"initializing",
}
TERMINAL_TASK_STATUSES = {
"success",
"failed",
"error",
"cancelled",
"canceled",
"rejected",
"timeout",
"completed",
"complete",
"done",
}
def is_active_task(task: dict[str, Any]) -> bool:
status = str(task.get("status") or "").strip().lower()
if not status:
return False
if status in ACTIVE_TASK_STATUSES:
return True
if status in TERMINAL_TASK_STATUSES:
return False
return True

73
modelhub_submmit_api/models.py Normal file
View File

@@ -0,0 +1,73 @@
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import datetime
@dataclass(frozen=True)
class TemplateRecord:
template_id: str
task_type: str
framework: str
target_gpu: str
model_address_template: str
config_params: str
source_file: str | None = None
source_line: int | None = None
@dataclass(frozen=True)
class HFModelSummary:
repo_id: str
downloads: int
last_modified: datetime | None
pipeline_tag: str | None
created_at: datetime | None = None
@property
def model_address(self) -> str:
return f"https://huggingface.co/{self.repo_id}"
@dataclass(frozen=True)
class ModelInspection:
repo_id: str
file_paths: list[str] = field(default_factory=list)
gguf_files: list[str] = field(default_factory=list)
selected_gguf: str | None = None
weight_files: list[str] = field(default_factory=list)
onnx_files: list[str] = field(default_factory=list)
@property
def has_gguf(self) -> bool:
return self.selected_gguf is not None
@property
def has_vllm_weights(self) -> bool:
return bool(self.weight_files)
@property
def has_onnx_weights(self) -> bool:
return bool(self.onnx_files)
@property
def has_standard_weights(self) -> bool:
return bool(self.weight_files or self.onnx_files)
@dataclass(frozen=True)
class CandidateModel:
repo_id: str
model_address: str
pipeline_tag: str | None
modality: str
task_type: str
target_gpu: str
framework: str
template_id: str
config_params: str
downloads: int
last_modified: datetime | None
gguf_filename: str | None = None
score: float = 0.0
warnings: list[str] = field(default_factory=list)

233
modelhub_submmit_api/outcome_tracker.py Normal file
View File

@@ -0,0 +1,233 @@
from __future__ import annotations
from collections import defaultdict
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from common import append_jsonl, parse_datetime, read_jsonl, utc_now, write_jsonl
from history_stats import classify_failure, is_failure, is_success
from modelhub_client import ModelHubClient
DEFAULT_OUTCOMES_PATH = Path("outcomes/submissions.jsonl")
def _now_iso() -> str:
return utc_now().isoformat()
class OutcomeTracker:
def __init__(self, path: Path | str) -> None:
self.path = Path(path)
self._records: list[dict[str, Any]] = []
self._by_task_id: dict[str, dict[str, Any]] = {}
self._by_model_gpu: dict[tuple[str, str], list[dict[str, Any]]] = defaultdict(list)
self._failed_model_gpus: set[tuple[str, str]] = set()
loaded = read_jsonl(self.path)
for record in loaded:
self._records.append(record)
task_id = record.get("taskId")
if task_id:
self._by_task_id[str(task_id)] = record
model_id = record.get("modelId") or ""
target_gpu = record.get("targetGpu") or ""
self._by_model_gpu[(model_id, target_gpu)].append(record)
self._rebuild_failed_index()
last_sync_times = [
parse_datetime(record.get("lastSyncTime"))
for record in self._records
if record.get("lastSyncTime")
]
self._last_sync_time: datetime = max(last_sync_times) if last_sync_times else utc_now() - timedelta(days=7)
def record_submission(
self,
model_id: str,
target_gpu: str,
framework: str,
task_type: str,
task_id: str | None,
submit_time: str,
) -> None:
record: dict[str, Any] = {
"modelId": model_id,
"targetGpu": target_gpu,
"framework": framework,
"taskType": task_type,
"taskId": task_id,
"submitTime": submit_time,
"lastSyncTime": None,
"status": "pending",
"verifyResult": None,
"outcome": "pending",
"failReason": None,
}
self._records.append(record)
if task_id:
self._by_task_id[task_id] = record
self._by_model_gpu[(model_id, target_gpu)].append(record)
append_jsonl(self.path, record)
def sync_from_api(self, client: ModelHubClient) -> int:
try:
begin = self._last_sync_time
end = utc_now()
list_kwargs: dict[str, Any] = {"begin_time": begin, "end_time": end, "page_size": 100, "only_mine": True}
tasks = client.list_tasks(**list_kwargs)
except Exception:
return 0
updated_count = 0
for task in tasks:
task_id = str(task.get("taskId")) if task.get("taskId") is not None else None
if not task_id:
continue
existing = self._by_task_id.get(task_id)
if existing is not None:
if existing.get("outcome") == "pending":
self._update_record_from_task(existing, task)
updated_count += 1
else:
status = str(task.get("status") or "").lower()
if status in {"success", "failed", "error", "cancelled", "completed"}:
record = self._create_record_from_task(task)
self._records.append(record)
self._by_task_id[task_id] = record
model_id = record["modelId"]
target_gpu = record["targetGpu"]
self._by_model_gpu[(model_id, target_gpu)].append(record)
updated_count += 1
if updated_count:
self._last_sync_time = end
self._rebuild_failed_index()
self.save()
return updated_count
def is_model_gpu_failed(self, model_id: str, target_gpu: str) -> bool:
return (model_id, target_gpu) in self._failed_model_gpus
def get_stats_report(self) -> dict[str, Any]:
now = _now_iso()
terminal = [r for r in self._records if r.get("outcome") in {"success", "failed"}]
gpu_groups: dict[str, list[dict[str, Any]]] = defaultdict(list)
framework_groups: dict[str, list[dict[str, Any]]] = defaultdict(list)
combo_groups: dict[tuple[str, str, str], list[dict[str, Any]]] = defaultdict(list)
for record in terminal:
gpu = record.get("targetGpu") or "unknown"
fw = record.get("framework") or "unknown"
tt = record.get("taskType") or "unknown"
gpu_groups[gpu].append(record)
framework_groups[f"{fw}"].append(record)
combo_groups[(gpu, fw, tt)].append(record)
gpu_summaries = {gpu: _summarize(records) for gpu, records in gpu_groups.items()}
framework_summaries = {fw: _summarize(records) for fw, records in framework_groups.items()}
combination_stats = {
f"{gpu}|{fw}|{tt}": {"targetGpu": gpu, "framework": fw, "taskType": tt, **_summarize(records)}
for (gpu, fw, tt), records in combo_groups.items()
}
warnings: list[str] = []
for gpu, summary in gpu_summaries.items():
if summary["total"] >= 4 and summary["failureRate"] >= 0.5:
warnings.append(f"GPU {gpu} 本地统计失败率偏高（≥50%），建议重点关注。")
for key, stat in combination_stats.items():
if stat["total"] >= 3 and stat["failureRate"] >= 0.6:
warnings.append(f"组合 {key} 近期失败集中，建议降低该 GPU+框架的提交优先级。")
pending_count = sum(1 for r in self._records if r.get("outcome") == "pending")
return {
"generatedAt": now,
"totalRecords": len(self._records),
"pendingRecords": pending_count,
"terminalRecords": len(terminal),
"gpuSummaries": gpu_summaries,
"frameworkSummaries": framework_summaries,
"combinationStats": combination_stats,
"totals": _summarize(terminal),
"warnings": warnings,
}
def save(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True)
write_jsonl(self.path, self._records)
def _rebuild_failed_index(self) -> None:
self._failed_model_gpus.clear()
for record in self._records:
if record.get("outcome") == "failed":
model_id = record.get("modelId") or ""
target_gpu = record.get("targetGpu") or ""
if model_id and target_gpu:
self._failed_model_gpus.add((model_id, target_gpu))
@staticmethod
def _update_record_from_task(record: dict[str, Any], task: dict[str, Any]) -> None:
record["status"] = task.get("status")
record["verifyResult"] = task.get("verifyResult")
record["lastSyncTime"] = _now_iso()
if is_success(task):
record["outcome"] = "success"
record["failReason"] = None
elif is_failure(task):
record["outcome"] = "failed"
record["failReason"] = classify_failure(task)
else:
record["outcome"] = "pending"
@staticmethod
def _create_record_from_task(task: dict[str, Any]) -> dict[str, Any]:
create_time = parse_datetime(task.get("createTime"))
record: dict[str, Any] = {
"modelId": task.get("modelId") or task.get("model_id") or "",
"targetGpu": task.get("gpuType") or task.get("targetGpu") or "",
"framework": task.get("framework") or "",
"taskType": task.get("taskType") or "",
"taskId": str(task.get("taskId")) if task.get("taskId") is not None else None,
"submitTime": create_time.isoformat() if create_time else _now_iso(),
"lastSyncTime": _now_iso(),
"status": task.get("status"),
"verifyResult": task.get("verifyResult"),
"outcome": "pending",
"failReason": None,
}
if is_success(task):
record["outcome"] = "success"
elif is_failure(task):
record["outcome"] = "failed"
record["failReason"] = classify_failure(task)
return record
def _summarize(records: list[dict[str, Any]]) -> dict[str, Any]:
total = len(records)
success_count = sum(1 for r in records if r.get("outcome") == "success")
failure_count = sum(1 for r in records if r.get("outcome") == "failed")
pending_count = sum(1 for r in records if r.get("outcome") == "pending")
failure_breakdown: dict[str, int] = defaultdict(int)
for r in records:
reason = r.get("failReason")
if reason:
failure_breakdown[reason] += 1
return {
"total": total,
"successCount": success_count,
"failureCount": failure_count,
"pendingCount": pending_count,
"successRate": round(success_count / total, 4) if total > 0 else 0.0,
"failureRate": round(failure_count / total, 4) if total > 0 else 0.0,
"pendingRate": round(pending_count / total, 4) if total > 0 else 0.0,
"failureBreakdown": dict(failure_breakdown),
}

259
modelhub_submmit_api/poll_runner.py Normal file
View File

@@ -0,0 +1,259 @@
from __future__ import annotations
import argparse
import json
import sys
import time
from pathlib import Path
from typing import Any, Callable
from common import utc_now, write_json
from daily_runner import DEFAULT_DAILY_RUNS_DIR, log, run_daily_batches
from hf_discovery import HuggingFaceDiscovery
from main import DEFAULT_LEDGER_PATH, DEFAULT_RUNS_DIR
from modelhub_client import ModelHubClient
from outcome_tracker import DEFAULT_OUTCOMES_PATH, OutcomeTracker
from runner_common import DEFAULT_KEY_PATH, ensure_tokens
from template_selector import TemplateSelector
DEFAULT_POLL_RUNS_DIR = Path("poll_runs")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Continuously poll ModelHub queue slots and refill submissions.")
parser.add_argument("--daily-target", type=int, default=0, help="Total submissions to aim for per UTC day; 0 means unlimited")
parser.add_argument("--gpu", help="Single GPU alias or platform name, for example: k100")
parser.add_argument("--gpus", help="Comma-separated GPU aliases/platform names. Omit to auto-use all safe GPUs.")
parser.add_argument("--min-downloads", type=int, default=50, help="Minimum Hugging Face download threshold")
parser.add_argument(
"--history-stats-threshold",
type=int,
default=500,
help="Minimum local ledger records before using history stats to rank submissions",
)
parser.add_argument("--read-concurrency", type=int, default=4, help="Concurrency for read-only remote calls")
parser.add_argument(
"--submit-concurrency",
type=int,
default=0,
help="Concurrency for task submission calls (0 = auto based on token/client count)",
)
parser.add_argument(
"--max-submits-per-run",
type=int,
default=0,
help="Maximum tasks to submit in one cycle (0 means unlimited)",
)
parser.add_argument("--max-scan-models", type=int, default=0, help="Hard cap on total scanned models (0 means auto)")
parser.add_argument(
"--scan-multiplier",
type=int,
default=4,
help="Multiplier used when auto-deriving scan limit from quota/queue capacity",
)
parser.add_argument("--skip-outcome-sync", action="store_true", help="Skip outcome sync from ModelHub before scanning")
parser.add_argument("--skip-history-archive", action="store_true", help="Skip historical task archive download for this run")
parser.add_argument("--dry-run", action="store_true", help="Plan the day without creating tasks")
parser.add_argument("--key-path", default=str(DEFAULT_KEY_PATH), help="Path to KEY.md containing HF_TOKEN/XC_TOKEN")
parser.add_argument("--runs-dir", default=str(DEFAULT_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--ledger-path", default=str(DEFAULT_LEDGER_PATH), help=argparse.SUPPRESS)
parser.add_argument("--history-archive-path", default="history/platform_tasks.jsonl", help=argparse.SUPPRESS)
parser.add_argument("--history-archive-limit", type=int, default=5000, help=argparse.SUPPRESS)
parser.add_argument("--daily-runs-dir", default=str(DEFAULT_DAILY_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--poll-runs-dir", default=str(DEFAULT_POLL_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--outcomes-path", default=str(DEFAULT_OUTCOMES_PATH), help=argparse.SUPPRESS)
parser.add_argument("--hf-base-url", default="https://huggingface.co", help=argparse.SUPPRESS)
parser.add_argument("--modelhub-base-url", default="https://modelhub.org.cn", help=argparse.SUPPRESS)
parser.add_argument("--modelhub-token", default=None, help=argparse.SUPPRESS)
parser.add_argument("--hf-token", default=None, help=argparse.SUPPRESS)
parser.add_argument("--poll-interval-seconds", type=int, default=60, help="Sleep between polling cycles when no slots are available")
parser.add_argument("--idle-interval-seconds", type=int, default=30, help="Sleep between cycles when a scan submits nothing")
parser.add_argument("--post-cycle-cooldown-seconds", type=int, default=0, help="Short sleep after a successful cycle")
parser.add_argument("--max-cycles", type=int, default=0, help="Optional hard stop after N cycles; 0 means run until quota is reached")
parser.add_argument("--print-stats", action="store_true", help="Load outcomes, sync, print stats report, and exit")
return parser
def _make_cycle_args(base_args: argparse.Namespace) -> argparse.Namespace:
cycle_args = argparse.Namespace(**vars(base_args))
cycle_args.rounds = 1
# Poll runner handles outcome sync on its own schedule (every OUTCOME_SYNC_INTERVAL cycles).
# Skip the per-cycle sync inside run_submission to avoid redundant paginated API calls.
cycle_args.skip_outcome_sync = True
return cycle_args
def _build_modelhub_client(base_args: argparse.Namespace) -> ModelHubClient:
return ModelHubClient(token=base_args.modelhub_token, base_url=base_args.modelhub_base_url)
def run_poll_loop(
*,
base_args: argparse.Namespace,
now=None,
run_fn: Callable[..., dict[str, Any]] = run_daily_batches,
hf_discovery: HuggingFaceDiscovery | None = None,
modelhub_client: ModelHubClient | None = None,
template_selector: TemplateSelector | None = None,
outcome_tracker: OutcomeTracker | None = None,
) -> dict[str, Any]:
now = now or utc_now()
hf_discovery = hf_discovery or HuggingFaceDiscovery(base_url=base_args.hf_base_url)
modelhub_client = modelhub_client or _build_modelhub_client(base_args)
template_selector = template_selector or TemplateSelector()
poll_runs_dir = Path(base_args.poll_runs_dir)
poll_runs_dir.mkdir(parents=True, exist_ok=True)
poll_run_dir = poll_runs_dir / now.strftime("%Y%m%dT%H%M%SZ")
poll_run_dir.mkdir(parents=True, exist_ok=True)
outcome_tracker = outcome_tracker or OutcomeTracker(Path(base_args.outcomes_path))
OUTCOME_SYNC_INTERVAL = 3
STATS_PRINT_INTERVAL = 10
log(f"[poll] poll_run_dir={poll_run_dir}")
log(
f"[poll] target={base_args.daily_target} dry_run={str(bool(base_args.dry_run)).lower()} "
f"poll_interval={base_args.poll_interval_seconds}s idle_interval={base_args.idle_interval_seconds}s"
)
cycle_summaries: list[dict[str, Any]] = []
submitted_total = 0
cycles = 0
stopped_reason = "max_cycles_reached"
while True:
if base_args.max_cycles and cycles >= base_args.max_cycles:
stopped_reason = "max_cycles_reached"
break
cycles += 1
active_counts = modelhub_client.active_task_counts() if hasattr(modelhub_client, "active_task_counts") else []
available_slots = modelhub_client.available_submit_slots() if hasattr(modelhub_client, "available_submit_slots") else None
log(
f"[poll] cycle={cycles} active_counts={','.join(str(count) for count in active_counts) if active_counts else 'n/a'} "
f"available_slots={available_slots if available_slots is not None else 'n/a'}"
)
if available_slots is not None and available_slots <= 0:
log(f"[poll] cycle={cycles} sleep={base_args.poll_interval_seconds}s reason=no_available_slots")
time.sleep(base_args.poll_interval_seconds)
continue
cycle_summary = run_fn(
base_args=_make_cycle_args(base_args),
now=utc_now(),
hf_discovery=hf_discovery,
modelhub_client=modelhub_client,
template_selector=template_selector,
outcome_tracker=outcome_tracker,
)
cycle_summaries.append(cycle_summary)
submitted_total += cycle_summary["submittedTotal"]
remaining_before_run = cycle_summary.get("remainingDailyQuotaBeforeRun")
log(
f"[poll] cycle_done submitted_total={cycle_summary['submittedTotal']} "
f"remaining_before_run={remaining_before_run if remaining_before_run is not None else 'n/a'} "
f"stop={cycle_summary['stoppedReason']}"
)
if base_args.daily_target > 0 and remaining_before_run is not None and remaining_before_run <= 0:
stopped_reason = "daily_target_already_reached"
break
if cycle_summary["submittedTotal"] <= 0:
log(f"[poll] cycle={cycles} sleep={base_args.idle_interval_seconds}s reason=no_new_submissions")
time.sleep(base_args.idle_interval_seconds)
continue
if cycles % OUTCOME_SYNC_INTERVAL == 0:
try:
synced = outcome_tracker.sync_from_api(modelhub_client)
if synced > 0:
log(f"[poll] cycle={cycles} outcome_sync_updated={synced}")
except Exception as exc:
log(f"[poll] cycle={cycles} outcome_sync_error={exc}")
if cycles % STATS_PRINT_INTERVAL == 0:
try:
stats = outcome_tracker.get_stats_report()
totals = stats.get("totals", {})
log(
f"[poll] cycle={cycles} outcome_stats "
f"terminal={totals.get('total', 0)} "
f"success={totals.get('successCount', 0)} "
f"failed={totals.get('failureCount', 0)} "
f"success_rate={totals.get('successRate', 0):.3f} "
f"failure_rate={totals.get('failureRate', 0):.3f}"
)
except Exception as exc:
log(f"[poll] cycle={cycles} outcome_stats_error={exc}")
if hasattr(modelhub_client, "available_submit_slots") and modelhub_client.available_submit_slots() <= 0:
log(f"[poll] cycle={cycles} sleep={base_args.poll_interval_seconds}s reason=queue_refilled")
time.sleep(base_args.poll_interval_seconds)
else:
cooldown = getattr(base_args, "post_cycle_cooldown_seconds", 1)
if cooldown > 0:
time.sleep(cooldown)
try:
outcome_tracker.sync_from_api(modelhub_client)
except Exception:
pass
outcome_tracker.save()
try:
stats_report = outcome_tracker.get_stats_report()
except Exception:
stats_report = {}
summary = {
"generatedAt": now.isoformat(),
"dryRun": bool(base_args.dry_run),
"dailyTarget": base_args.daily_target,
"unlimitedDailyTarget": base_args.daily_target <= 0,
"cycles": cycles,
"submittedTotal": submitted_total,
"stoppedReason": stopped_reason,
"pollRunDir": str(poll_run_dir),
"cycleSummaries": cycle_summaries,
"outcomeStats": stats_report,
}
write_json(poll_run_dir / "summary.json", summary)
log(f"[poll] finished submitted_total={submitted_total} cycles={cycles} stopped_reason={stopped_reason}")
return summary
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
ensure_tokens(args)
if getattr(args, "print_stats", False):
try:
modelhub_client = _build_modelhub_client(args)
tracker = OutcomeTracker(Path(args.outcomes_path))
tracker.sync_from_api(modelhub_client)
report = tracker.get_stats_report()
print(json.dumps(report, ensure_ascii=False, indent=2))
except Exception as exc:
print(f"Failed to generate stats report: {exc}", file=sys.stderr)
return 1
return 0
log(
f"[poll] hf_token={'set' if bool(args.hf_token) else 'missing'} "
f"xc_token={'set' if bool(args.modelhub_token) else 'missing'}"
)
summary = run_poll_loop(base_args=args)
print(f"poll_run_dir={summary['pollRunDir']}")
print(f"submitted_total={summary['submittedTotal']}")
print(f"cycles={summary['cycles']}")
print(f"stopped_reason={summary['stoppedReason']}")
return 0
if __name__ == "__main__":
raise SystemExit(main())

26
modelhub_submmit_api/run_daily.sh Normal file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEFAULT_DAILY_TARGET="${SUBMIT_DAILY_TARGET:-3}"
cd "$SCRIPT_DIR"
HAS_DAILY_TARGET=false
args=()
while [[ "$#" -gt 0 ]]; do
case "$1" in
--daily-target|--daily-target=*)
HAS_DAILY_TARGET=true
;;
esac
args+=("$1")
shift
done
if [[ "$HAS_DAILY_TARGET" != true ]]; then
args+=(--daily-target "$DEFAULT_DAILY_TARGET")
fi
exec python3 -u "$SCRIPT_DIR/daily_runner.py" "${args[@]}"

8
modelhub_submmit_api/run_poll.sh Normal file
View File

@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
exec python3 -u "$SCRIPT_DIR/poll_runner.py" "$@"

8
modelhub_submmit_api/run_success_history.sh Normal file
View File

@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
exec python3 -u "$SCRIPT_DIR/success_history_runner.py" "$@"

47
modelhub_submmit_api/runner_common.py Normal file
View File

@@ -0,0 +1,47 @@
from __future__ import annotations
import argparse
import os
from pathlib import Path
DEFAULT_KEY_PATH = Path("KEY.md")
def load_key_file(path: Path) -> dict[str, str]:
if not path.exists():
return {}
loaded: dict[str, str] = {}
for raw_line in path.read_text(encoding="utf-8").splitlines():
line = raw_line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, value = line.split("=", 1)
loaded[key.strip()] = value.strip()
return loaded
def ensure_tokens(args: argparse.Namespace) -> None:
primary_key_path = Path(getattr(args, "key_path", DEFAULT_KEY_PATH))
if not primary_key_path.exists():
parent_key = primary_key_path.parent.parent / primary_key_path.name
if parent_key.exists():
primary_key_path = parent_key
values = load_key_file(primary_key_path)
if not getattr(args, "hf_token", None):
args.hf_token = os.getenv("HF_TOKEN") or values.get("HF_TOKEN")
modelhub_token = getattr(args, "modelhub_token", None)
if not modelhub_token:
modelhub_token = os.getenv("MODELHUB_XC_TOKEN") or os.getenv("XC_TOKEN") or values.get("MODELHUB_XC_TOKEN") or values.get("XC_TOKEN")
args.modelhub_token = modelhub_token
if args.hf_token:
os.environ["HF_TOKEN"] = args.hf_token
if args.modelhub_token:
os.environ["MODELHUB_XC_TOKEN"] = args.modelhub_token
os.environ["XC_TOKEN"] = args.modelhub_token
if not args.modelhub_token and not os.getenv("STRATEGY_ID"):
raise ValueError("XC_TOKEN is required, either via environment or KEY.md")

15
modelhub_submmit_api/setup.sh Normal file
View File

@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
echo "=== ModelHub Submmit Setup ==="
python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3,9) else 1)' || { echo "ERROR: Python 3.9+ required"; exit 1; }
mkdir -p runs ledger outcomes history daily_runs poll_runs logs
python3 -c '
import json,argparse,sys,time,pathlib,typing,concurrent.futures,threading,http.client,urllib,socket,dataclasses,collections,datetime,os,re
print("Python stdlib OK (zero external deps)")
'
wc -l templates/public_submit/adapt_task_templates.jsonl
grep -q 'XC_TOKEN\s*=\s*[a-z0-9]\{32\}' KEY.md 2>/dev/null && echo "XC_TOKEN found" || echo "WARNING: Set XC_TOKEN in KEY.md"
echo "Setup done."
echo "Test: ./run_poll.sh --gpu k100 --max-scan-models 3 --max-cycles 2 --dry-run"

330
modelhub_submmit_api/success_history_runner.py Normal file
View File

@@ -0,0 +1,330 @@
from __future__ import annotations
import argparse
import os
from datetime import timedelta
from pathlib import Path
from typing import Any, Callable
from common import parse_datetime, utc_now, write_json, write_jsonl
from daily_runner import DEFAULT_DAILY_RUNS_DIR, log
from history_stats import is_success, update_history_archive
from modelhub_client import ModelHubClient
from models import HFModelSummary
from poll_runner import DEFAULT_POLL_RUNS_DIR, run_poll_loop
from runner_common import DEFAULT_KEY_PATH, ensure_tokens
from task_registry import TASK_SPEC_BY_TYPE
from template_selector import TemplateSelector
TASK_TYPE_BY_MODEL_TASK_LEVEL_ID = {
"0": "text-generation",
"2": "visual-multi-modal",
"21": "text-to-image-generation",
"23": "text-generation",
"29": "feature_emb",
"34": "feature_emb",
"39": "question_answering",
"54": "asr",
"116": "vision_classification",
"198": "vision_classification",
"207": "reinforcement_learning",
"213": "text-to-image-generation",
}
TASK_TYPE_BY_MODEL_TASK_LEVEL = {
"文本生成": "text-generation",
"视觉多模态理解": "visual-multi-modal",
"图片生成图片": "text-to-image-generation",
"文生图": "text-to-image-generation",
"特征抽取": "feature_emb",
"句子相似度": "feature_emb",
"问答": "question_answering",
"语音识别": "asr",
"视觉分类": "vision_classification",
"零样本图像分类": "vision_classification",
"强化学习": "reinforcement_learning",
}
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Continuously refill submissions, but only on GPUs with historical verified successes."
)
parser.add_argument("--history-days", type=int, default=3650, help="UTC lookback window for pulling task history")
parser.add_argument("--history-begin", help="Override history begin time, for example 2026-01-01 or 2026-01-01T00:00:00Z")
parser.add_argument("--history-end", help="Override history end time; defaults to now")
parser.add_argument("--history-page-size", type=int, default=100, help="ModelHub task history page size")
parser.add_argument("--daily-target", type=int, default=0, help="Total submissions to aim for per UTC day; 0 means unlimited")
parser.add_argument("--gpu", help="Single GPU alias or platform name, for example: k100")
parser.add_argument("--gpus", help="Comma-separated GPU aliases/platform names")
parser.add_argument(
"--successful-gpus-only",
action="store_true",
help="Accepted for backward compatibility; this runner always restricts to historically successful GPUs.",
)
parser.add_argument("--min-downloads", type=int, default=50, help="Minimum Hugging Face download threshold")
parser.add_argument(
"--history-stats-threshold",
type=int,
default=500,
help="Minimum local ledger records before using history stats to rank submissions",
)
parser.add_argument("--read-concurrency", type=int, default=4, help="Concurrency for read-only remote calls")
parser.add_argument("--dry-run", action="store_true", help="Plan the queue without creating tasks")
parser.add_argument("--key-path", default=str(DEFAULT_KEY_PATH), help="Path to KEY.md containing HF_TOKEN/XC_TOKEN")
parser.add_argument("--runs-dir", default="runs", help=argparse.SUPPRESS)
parser.add_argument("--ledger-path", default="ledger/submissions.jsonl", help=argparse.SUPPRESS)
parser.add_argument("--history-archive-path", default="history/platform_tasks.jsonl", help=argparse.SUPPRESS)
parser.add_argument("--history-archive-limit", type=int, default=20000, help=argparse.SUPPRESS)
parser.add_argument("--daily-runs-dir", default=str(DEFAULT_DAILY_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--poll-runs-dir", default=str(DEFAULT_POLL_RUNS_DIR), help=argparse.SUPPRESS)
parser.add_argument("--outcomes-path", default="outcomes/submissions.jsonl", help=argparse.SUPPRESS)
parser.add_argument("--hf-base-url", default=os.getenv("HF_BASE_URL", "https://huggingface.co"), help=argparse.SUPPRESS)
parser.add_argument("--modelhub-base-url", default=os.getenv("MODELHUB_BASE_URL", "https://modelhub.org.cn"), help=argparse.SUPPRESS)
parser.add_argument("--modelhub-token", default=os.getenv("MODELHUB_XC_TOKEN") or os.getenv("XC_TOKEN"), help=argparse.SUPPRESS)
parser.add_argument("--hf-token", default=os.getenv("HF_TOKEN"), help=argparse.SUPPRESS)
parser.add_argument("--poll-interval-seconds", type=int, default=60, help="Sleep between polling cycles when no slots are available")
parser.add_argument("--idle-interval-seconds", type=int, default=30, help="Sleep between cycles when a scan submits nothing")
parser.add_argument(
"--post-cycle-cooldown-seconds",
type=int,
default=1,
help="Short sleep after a successful cycle to avoid same-second run-dir collisions",
)
parser.add_argument("--max-cycles", type=int, default=0, help="Optional hard stop after N cycles; 0 means run until quota is reached")
return parser
def make_modelhub_client(args: argparse.Namespace) -> ModelHubClient:
return ModelHubClient(token=args.modelhub_token, base_url=args.modelhub_base_url)
def history_window(args: argparse.Namespace, now) -> tuple[Any, Any]:
begin = parse_datetime(args.history_begin) if args.history_begin else now - timedelta(days=args.history_days)
end = parse_datetime(args.history_end) if args.history_end else now
if begin is None:
raise ValueError(f"Invalid --history-begin value: {args.history_begin}")
if end is None:
raise ValueError(f"Invalid --history-end value: {args.history_end}")
return begin, end
def model_id_from_task(task: dict[str, Any]) -> str | None:
value = task.get("modelId") or task.get("modelName") or task.get("model")
if value:
return str(value).strip()
address = str(task.get("modelAddress") or "").strip().rstrip("/")
marker = "huggingface.co/"
if marker in address:
return address.split(marker, 1)[1].strip("/")
return None
def pipeline_tag_for_task_type(task_type: str | None) -> str | None:
if not task_type:
return None
spec = TASK_SPEC_BY_TYPE.get(task_type)
if spec is None:
return None
return spec.pipeline_tags[0]
def task_type_from_history_task(task: dict[str, Any]) -> str | None:
task_type = task.get("taskType")
if task_type and str(task_type) in TASK_SPEC_BY_TYPE:
return str(task_type)
level_id = task.get("modelTaskLevelId")
if level_id is not None:
mapped = TASK_TYPE_BY_MODEL_TASK_LEVEL_ID.get(str(level_id))
if mapped:
return mapped
level_name = task.get("modelTaskLevel")
if level_name:
return TASK_TYPE_BY_MODEL_TASK_LEVEL.get(str(level_name))
return None
def successful_models_from_tasks(tasks: list[dict[str, Any]], *, limit: int) -> tuple[list[HFModelSummary], dict[str, set[str]]]:
by_model: dict[str, HFModelSummary] = {}
gpus_by_model: dict[str, set[str]] = {}
for task in tasks:
if not is_success(task):
continue
model_id = model_id_from_task(task)
if not model_id:
continue
task_type = task_type_from_history_task(task)
pipeline_tag = pipeline_tag_for_task_type(task_type)
if not pipeline_tag:
continue
timestamp = parse_datetime(task.get("updateTime")) or parse_datetime(task.get("createTime")) or parse_datetime(task.get("submitTime"))
current = by_model.get(model_id)
if current is None or (timestamp and (current.last_modified is None or timestamp > current.last_modified)):
by_model[model_id] = HFModelSummary(
repo_id=model_id,
downloads=0,
last_modified=timestamp,
pipeline_tag=pipeline_tag,
)
gpu_type = task.get("gpuType") or task.get("targetGpu")
if gpu_type:
gpus_by_model.setdefault(model_id, set()).add(str(gpu_type))
models = sorted(by_model.values(), key=lambda item: item.last_modified or parse_datetime("1970-01-01"), reverse=True)
if limit > 0:
models = models[:limit]
return models, {model.repo_id: gpus_by_model.get(model.repo_id, set()) for model in models}
def successful_gpus_from_tasks(tasks: list[dict[str, Any]]) -> list[str]:
latest_success_by_gpu: dict[str, Any] = {}
for task in tasks:
if not is_success(task):
continue
gpu_type = task.get("gpuType") or task.get("targetGpu")
if not gpu_type:
continue
gpu = str(gpu_type)
timestamp = parse_datetime(task.get("updateTime")) or parse_datetime(task.get("createTime")) or parse_datetime(task.get("submitTime"))
current = latest_success_by_gpu.get(gpu)
if current is None or (timestamp and (current is None or timestamp > current)):
latest_success_by_gpu[gpu] = timestamp
return [
gpu for gpu, _ in sorted(
latest_success_by_gpu.items(),
key=lambda item: ((item[1].timestamp() if item[1] else 0.0), item[0]),
reverse=True,
)
]
def resolve_successful_target_gpus(
args: argparse.Namespace,
successful_gpus: list[str],
template_selector: TemplateSelector,
) -> list[str]:
normalized_successful: list[str] = []
for gpu in successful_gpus:
normalized = template_selector.normalize_gpu(gpu)
if normalized not in normalized_successful:
normalized_successful.append(normalized)
requested_values: list[str] = []
if getattr(args, "gpus", None):
requested_values.extend(value.strip() for value in str(args.gpus).split(",") if value.strip())
if getattr(args, "gpu", None):
requested_values.append(str(args.gpu).strip())
if requested_values:
requested_normalized: list[str] = []
for value in requested_values:
normalized = template_selector.normalize_gpu(value)
if normalized not in requested_normalized:
requested_normalized.append(normalized)
filtered = [gpu for gpu in requested_normalized if gpu in normalized_successful]
else:
filtered = normalized_successful
if not filtered:
raise RuntimeError("No historically successful GPUs are available after applying the requested GPU filter")
return filtered
def make_poll_args(args: argparse.Namespace, target_gpus: list[str]) -> argparse.Namespace:
poll_args = argparse.Namespace(**vars(args))
poll_args.gpu = None
poll_args.gpus = ",".join(target_gpus)
return poll_args
def run_success_history_poll(
args: argparse.Namespace,
*,
now=None,
run_fn: Callable[..., dict[str, Any]] = run_poll_loop,
modelhub_client: ModelHubClient | None = None,
template_selector: TemplateSelector | None = None,
) -> dict[str, Any]:
now = now or utc_now()
template_selector = template_selector or TemplateSelector()
modelhub_client = modelhub_client or make_modelhub_client(args)
begin, end = history_window(args, now)
tasks = modelhub_client.list_tasks(page_size=args.history_page_size, only_mine=True, begin_time=begin, end_time=end)
archived_history = update_history_archive(Path(args.history_archive_path), tasks, limit=args.history_archive_limit)
successful_models, gpus_by_model = successful_models_from_tasks(tasks, limit=0)
successful_gpus = successful_gpus_from_tasks(tasks)
target_gpus = resolve_successful_target_gpus(args, successful_gpus, template_selector)
poll_args = make_poll_args(args, target_gpus)
summary = run_fn(
base_args=poll_args,
now=now,
modelhub_client=modelhub_client,
template_selector=template_selector,
)
poll_run_dir = Path(summary["pollRunDir"])
write_json(
poll_run_dir / "history_success_gpus.json",
{
"generatedAt": now.isoformat(),
"historyBegin": begin.isoformat(),
"historyEnd": end.isoformat(),
"historyRecordCount": len(tasks),
"successfulGpuCount": len(successful_gpus),
"successfulGpus": successful_gpus,
"filteredTargetGpus": target_gpus,
},
)
write_jsonl(
poll_run_dir / "history_success_models.jsonl",
[
{
"repoId": model.repo_id,
"pipelineTag": model.pipeline_tag,
"lastSuccessfulUpdate": model.last_modified.isoformat() if model.last_modified else None,
"successfulGpus": sorted(gpus_by_model.get(model.repo_id, set())),
}
for model in successful_models
],
)
summary.update(
{
"historyBegin": begin.isoformat(),
"historyEnd": end.isoformat(),
"historyRecordCount": len(tasks),
"historyArchiveRecordCount": len(archived_history),
"successfulGpuCount": len(successful_gpus),
"successfulGpus": successful_gpus,
"filteredTargetGpus": target_gpus,
"successfulModelCount": len(successful_models),
"successfulGpuAuditPath": str(poll_run_dir / "history_success_gpus.json"),
"successfulModelAuditPath": str(poll_run_dir / "history_success_models.jsonl"),
}
)
write_json(poll_run_dir / "summary.json", summary)
return summary
def main(argv: list[str] | None = None) -> int:
parser = build_parser()
args = parser.parse_args(argv)
ensure_tokens(args)
log(
f"[success-history] hf_token={'set' if bool(args.hf_token) else 'missing'} "
f"xc_token={'set' if bool(args.modelhub_token) else 'missing'}"
)
summary = run_success_history_poll(args)
print(f"poll_run_dir={summary['pollRunDir']}")
print(f"history_records={summary['historyRecordCount']} successful_gpus={summary['successfulGpuCount']} successful_models={summary['successfulModelCount']}")
print(f"filtered_target_gpus={','.join(summary['filteredTargetGpus'])}")
print(f"submitted_total={summary['submittedTotal']} cycles={summary['cycles']} stopped_reason={summary['stoppedReason']}")
return 0
if __name__ == "__main__":
raise SystemExit(main())

120
modelhub_submmit_api/task_registry.py Normal file
View File

@@ -0,0 +1,120 @@
from __future__ import annotations
from dataclasses import dataclass
from models import HFModelSummary, ModelInspection
VLLM_LIKE_FRAMEWORKS = (
"vllm",
"sglang",
"vllm-customized",
"vllm-mlu",
"vllm-016",
"vllm_fix_tokenizer",
)
@dataclass(frozen=True)
class TaskSpec:
task_type: str
modality: str
pipeline_tags: tuple[str, ...]
priority: int
TASK_SPECS: tuple[TaskSpec, ...] = (
TaskSpec("text-generation", "text", ("text-generation",), 10),
TaskSpec(
"visual-multi-modal",
"multimodal",
("image-text-to-text", "visual-question-answering", "document-question-answering", "video-text-to-text"),
20,
),
TaskSpec("text-to-image-generation", "image", ("text-to-image", "image-to-image"), 30),
TaskSpec("asr", "audio", ("automatic-speech-recognition",), 40),
TaskSpec("question_answering", "text", ("question-answering",), 50),
TaskSpec("feature_emb", "embedding", ("feature-extraction", "sentence-similarity"), 60),
TaskSpec("vision_classification", "vision", ("image-classification", "zero-shot-image-classification"), 70),
TaskSpec("text_classification", "text", ("text-classification", "zero-shot-classification"), 80),
TaskSpec("reinforcement_learning", "text", ("reinforcement-learning",), 90),
)
TASK_SPEC_BY_TYPE = {task.task_type: task for task in TASK_SPECS}
def all_task_types() -> list[str]:
return [task.task_type for task in TASK_SPECS]
def pipeline_tags_for_task_types(task_types: list[str]) -> list[str]:
tags: list[str] = []
for task_type in task_types:
spec = TASK_SPEC_BY_TYPE[task_type]
for tag in spec.pipeline_tags:
if tag not in tags:
tags.append(tag)
return tags
def task_specs_for_model(model: HFModelSummary) -> list[TaskSpec]:
pipeline_tag = (model.pipeline_tag or "").strip().lower()
return [task for task in TASK_SPECS if pipeline_tag in task.pipeline_tags]
def choose_text_generation_framework(target_gpu: str, supported_frameworks: set[str], inspection: ModelInspection) -> str:
can_llamacpp = "llamacpp" in supported_frameworks
vllm_like = [framework for framework in VLLM_LIKE_FRAMEWORKS if framework in supported_frameworks]
can_transformers = "transformers" in supported_frameworks
if can_llamacpp and vllm_like:
if inspection.has_gguf:
return "llamacpp"
if inspection.has_vllm_weights:
return vllm_like[0]
raise ValueError(f"{target_gpu} supports both llamacpp and vLLM-like frameworks, but the model lacks usable weights")
if can_llamacpp:
if inspection.has_gguf:
return "llamacpp"
raise ValueError(f"{target_gpu} only supports llamacpp for this workflow, but no usable GGUF was found")
if vllm_like:
if inspection.has_vllm_weights:
return vllm_like[0]
raise ValueError(f"{target_gpu} only supports vLLM-like frameworks for this workflow, but no standard weights were found")
if can_transformers:
if inspection.has_vllm_weights:
return "transformers"
raise ValueError(f"{target_gpu} only supports transformers for this workflow, but no standard weights were found")
raise ValueError(f"No supported LLM framework template is available for {target_gpu}")
def choose_framework_for_task(task_type: str, target_gpu: str, supported_frameworks: set[str], inspection: ModelInspection) -> str:
if task_type in {"text-generation", "visual-multi-modal", "reinforcement_learning"}:
return choose_text_generation_framework(target_gpu, supported_frameworks, inspection)
if task_type == "asr":
if "sherpa-onnx" in supported_frameworks and inspection.has_onnx_weights:
return "sherpa-onnx"
for framework in ("transformers", "funasr"):
if framework in supported_frameworks and inspection.has_standard_weights:
return framework
raise ValueError(f"No compatible ASR framework found for {target_gpu}")
if task_type == "feature_emb":
for framework in ("sentence-transformers", "transformers"):
if framework in supported_frameworks and inspection.has_standard_weights:
return framework
raise ValueError(f"No compatible embedding framework found for {target_gpu}")
if task_type in {"question_answering", "vision_classification", "text_classification"}:
if "transformers" in supported_frameworks and inspection.has_standard_weights:
return "transformers"
raise ValueError(f"No compatible transformers template found for {task_type} on {target_gpu}")
if task_type == "text-to-image-generation":
if "diffusers" in supported_frameworks and inspection.has_standard_weights:
return "diffusers"
raise ValueError(f"No compatible diffusers template found for {target_gpu}")
raise ValueError(f"Unsupported task type for auto framework selection: {task_type}")

169
modelhub_submmit_api/template_selector.py Normal file
View File

@@ -0,0 +1,169 @@
from __future__ import annotations
import json
import re
import os
from pathlib import Path
from models import TemplateRecord
MODULE_DIR = Path(__file__).resolve().parent
DEFAULT_TEMPLATE_FILE_NAME = "adapt_task_templates.jsonl"
DEFAULT_TEMPLATE_RELATIVE_CANDIDATES = (
Path("templates/public_submit") / DEFAULT_TEMPLATE_FILE_NAME,
Path("model adaptation/templates/public_submit") / DEFAULT_TEMPLATE_FILE_NAME,
)
def _normalize_token(value: str) -> str:
return re.sub(r"[^a-z0-9]", "", value.lower())
GPU_ALIASES = {
"k100": "hygon_k100-ai",
"k100ai": "hygon_k100-ai",
"hygonk100ai": "hygon_k100-ai",
"hygon_k100ai": "hygon_k100-ai",
"hygonk100": "hygon_k100-ai",
"s4000": "Mthreads_s4000",
"mthreadss4000": "Mthreads_s4000",
"bi150": "Iluvatar_bi-150",
"iluvatarbi150": "Iluvatar_bi-150",
"bi100": "Iluvatar_bi-100",
"iluvatarbi100": "Iluvatar_bi-100",
"mrv100": "Iluvatar_mrv-100",
"iluvatarmrv100": "Iluvatar_mrv-100",
"metaxc500": "MetaX_c-500",
"c500": "MetaX_c-500",
"kunlunxinp800": "Kunlunxin_p-800",
"p800": "Kunlunxin_p-800",
"kunlunxinr2008f": "Kunlunxin_r-200-8f",
"r2008f": "Kunlunxin_r-200-8f",
"ascend910b3": "Ascend_910-b3",
"910b3": "Ascend_910-b3",
"ascend910b4": "Ascend_910-b4",
"910b4": "Ascend_910-b4",
"bieren166m": "Biren_166m",
"biren166m": "Biren_166m",
"vastaiva16": "Vastai_va16",
"va16": "Vastai_va16",
"sunrisept200x1": "Sunrise_pt-200-x1",
"pt200x1": "Sunrise_pt-200-x1",
}
class TemplateSelector:
def __init__(self, template_file: Path | None = None) -> None:
self.template_file = self._resolve_template_file(template_file)
self.templates = self._load_templates(self.template_file)
self.index = {
(template.task_type, template.framework, template.target_gpu): template
for template in self.templates
}
for template in self.templates:
GPU_ALIASES.setdefault(_normalize_token(template.target_gpu), template.target_gpu)
@staticmethod
def _resolve_template_file(template_file: Path | None) -> Path:
env_template_file = os.getenv("MODELHUB_TEMPLATE_FILE")
if env_template_file:
candidate = Path(env_template_file).expanduser()
if candidate.exists():
return candidate
candidates: list[Path] = []
if template_file is not None:
candidates.append(template_file)
else:
search_roots = [Path.cwd().resolve(), MODULE_DIR.resolve()]
for root in search_roots:
candidates.append(root / DEFAULT_TEMPLATE_FILE_NAME)
candidates.extend(root / relative for relative in DEFAULT_TEMPLATE_RELATIVE_CANDIDATES)
parent = root.parent
if parent != root:
candidates.append(parent / DEFAULT_TEMPLATE_FILE_NAME)
candidates.extend(parent / relative for relative in DEFAULT_TEMPLATE_RELATIVE_CANDIDATES)
for candidate in candidates:
if candidate.exists():
return candidate
tried = ", ".join(str(candidate) for candidate in candidates)
raise FileNotFoundError(f"Could not find ModelHub template file. Tried: {tried}")
@staticmethod
def _load_templates(path: Path) -> list[TemplateRecord]:
templates: list[TemplateRecord] = []
for line in path.read_text(encoding="utf-8").splitlines():
if not line.strip():
continue
payload = json.loads(line)
source = payload.get("source") or {}
templates.append(
TemplateRecord(
template_id=payload["templateId"],
task_type=payload["taskType"],
framework=payload["framework"],
target_gpu=payload["targetGpu"],
model_address_template=payload["modelAddressTemplate"],
config_params=payload["configParams"],
source_file=source.get("file"),
source_line=source.get("line"),
)
)
return templates
def normalize_gpu(self, gpu: str) -> str:
normalized = _normalize_token(gpu)
if normalized in GPU_ALIASES:
return GPU_ALIASES[normalized]
raise KeyError(f"Unsupported GPU alias: {gpu}")
def supported_frameworks(self, task_type: str, target_gpu: str) -> set[str]:
return {
template.framework
for template in self.templates
if template.task_type == task_type and template.target_gpu == target_gpu
}
def supported_target_gpus(self, task_type: str, *, auto_only: bool = False) -> list[str]:
gpus = {
template.target_gpu
for template in self.templates
if template.task_type == task_type and (not auto_only or self.is_auto_template(template))
}
return sorted(gpus)
def supported_frameworks_for_auto(self, task_type: str, target_gpu: str) -> set[str]:
return {
template.framework
for template in self.templates
if template.task_type == task_type and template.target_gpu == target_gpu and self.is_auto_template(template)
}
def select_template(self, task_type: str, framework: str, target_gpu: str) -> TemplateRecord:
key = (task_type, framework, target_gpu)
if key not in self.index:
raise KeyError(f"Missing template for taskType={task_type}, framework={framework}, targetGpu={target_gpu}")
return self.index[key]
@staticmethod
def is_auto_template(template: TemplateRecord) -> bool:
config = template.config_params
if "PLACEHOLDER" in config:
return False
if "根据" in config:
return False
return True
def render_config(self, template: TemplateRecord, gguf_filename: str | None = None) -> str:
config = template.config_params
if template.framework != "llamacpp":
return config
if not gguf_filename:
raise ValueError("llamacpp template requires a GGUF filename")
replaced = re.sub(r"/model/[^,\]\s]+?\.gguf", f"/model/{gguf_filename}", config)
if replaced == config:
raise ValueError(f"Failed to replace GGUF filename in template {template.template_id}")
return replaced

53
modelhub_submmit_api/templates/public_submit/adapt_task_templates.jsonl Normal file
View File

@@ -0,0 +1,53 @@
{"templateId": "asr-funasr-iluvatar-bi-150", "taskType": "asr", "framework": "funasr", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: funasr\nlang: zh\nref_config:\n gpu_num: 1\n values:\n command: [python3, main.py, --port, '80', --model_dir, /model, --use_gpu, --model_type,\n sensevoice]\nsut_config:\n gpu_num: 1\n values:\n command: [python3, main.py, --port, '80', --model_dir, /model, --use_gpu, --model_type,\n sensevoice]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 185}}
{"templateId": "asr-funasr-iluvatar-mrv-100", "taskType": "asr", "framework": "funasr", "targetGpu": "Iluvatar_mrv-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor-contest.4pd.io/sunjichen/funasr:ilu-mr100-service-0.2\nnv_docker_image: harbor-contest.4pd.io/sunjichen/funasr:a100-server-0.2\nframework: funasr\nlang: zh\nsut_config:\n gpu_num: 1\n values:\n command: [python3, main.py, --port, '8000', --model_dir, /model, --model_type,\n paraformer]\nref_config:\n gpu_num: 1\n values:\n command: [python3, main.py, --port, '8000', --model_dir, /model, --model_type,\n paraformer]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 644}}
{"templateId": "asr-sherpa-onnx-iluvatar-bi-150", "taskType": "asr", "framework": "sherpa-onnx", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "lang: zh\nframework: sherpa-onnx\nsut_config:\n gpu_num: 1\n values:\n command:\n - python3\n - main_sherpa.py\n - --port\n - '80'\n - --model_dir\n - /model\n - --use_gpu\n - --model_type\n - dolphin_ctc\n - --offline_model\nref_config:\n gpu_num: 1\n values:\n command:\n - python3\n - main_sherpa.py\n - --port\n - '80'\n - --model_dir\n - /model\n - --use_gpu\n - --model_type\n - dolphin_ctc\n - --offline_model\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 193}}
{"templateId": "asr-transformers-iluvatar-bi-150", "taskType": "asr", "framework": "transformers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "lang: zh\nframework: transformers\nsut_config:\n gpu_num: 1\n values:\n command:\n - python3\n - main_transformers.py\n - --port\n - '80'\n - --model_dir\n - /model\n - --use_gpu\nref_config:\n gpu_num: 1\n values:\n command:\n - python3\n - main_transformers.py\n - --port\n - '80'\n - --model_dir\n - /model\n - --use_gpu\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 209}}
{"templateId": "feature-emb-sentence-transformers-metax-c-500", "taskType": "feature_emb", "framework": "sentence-transformers", "targetGpu": "MetaX_c-500", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: sentence-transformers\nsut_config:\n gpu_num: 1\n values:\n command:\n - /opt/conda/bin/python\n - -m\n - uvicorn\n - server:app\n - --host\n - 0.0.0.0\n - --port\n - '8000'\nref_config:\n gpu_num: 1\n values:\n command:\n - python3\n - -m\n - uvicorn\n - server:app\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 466}}
{"templateId": "question-answering-transformers-iluvatar-bi-150", "taskType": "question_answering", "framework": "transformers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: transformers\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"torch_dtype\": \"auto\",\n \"handle_impossible_answer\": false,\n \"score_threshold\": 0.0,\n \"max_answer_len\": 30,\n \"max_seq_len\": 512,\n \"doc_stride\": 128\n }\n command:\n - python3\n - main_qa.py\n - --model_dir\n - /model\n - --port\n - '80'\nref_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"torch_dtype\": \"auto\",\n \"handle_impossible_answer\": false,\n \"score_threshold\": 0.0,\n \"max_answer_len\": 30,\n \"max_seq_len\": 512,\n \"doc_stride\": 128\n }\n command:\n - python3\n - main_qa.py\n - --model_dir\n - /model\n - --port\n - '80'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 217}}
{"templateId": "reinforcement-learning-llamacpp-iluvatar-bi-150", "taskType": "reinforcement_learning", "framework": "llamacpp", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - /app/llama-server\n - --model\n - /model/Math-Code-Llama3.1-8B.i1-Q4_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n - --no-mmap\n - --sync-to-temp\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/Math-Code-Llama3.1-8B.i1-Q4_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 225}}
{"templateId": "reinforcement-learning-transformers-iluvatar-bi-150", "taskType": "reinforcement_learning", "framework": "transformers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\napi: completion\nframework: transformers\nnv_framework: vllm\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForCausalLM\",\n \"torch_dtype\": \"auto\",\n \"model_kwargs\": {\n \"trust_remote_code\": true,\n \"device_map\": \"auto\"\n },\n \"tokenizer_kwargs\": {\n \"use_fast\": true,\n \"trust_remote_code\": true\n },\n \"generate_kwargs\": {\n \"temperature\": 0.01,\n \"top_p\": 0.8,\n \"repetition_penalty\": 1.0,\n \"max_new_tokens\": 1024\n }\n }\n command:\n - python3\n - transformers_server.py\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 233}}
{"templateId": "reinforcement-learning-vllm-iluvatar-bi-150", "taskType": "reinforcement_learning", "framework": "vllm", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex-iluvatar-bi150/vllm:0.8.3\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\napi: completion\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --gpu-memory-utilization\n - '0.9'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 241}}
{"templateId": "text-generation-llamacpp-ascend-910-b3", "taskType": "text-generation", "framework": "llamacpp", "targetGpu": "Ascend_910-b3", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\nframework: llamacpp\napi: completion\nmax_tokens: 1024\nmax_model_len: 2048\nsut_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build_ascend/bin/llama-server\n - --model\n - /model/Free_Sydney_V2_Mistral_7b.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '2048'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/Free_Sydney_V2_Mistral_7b.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '2048'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 41}}
{"templateId": "text-generation-llamacpp-ascend-910-b4", "taskType": "text-generation", "framework": "llamacpp", "targetGpu": "Ascend_910-b4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\napi: chat\nlang: zh\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command: [/workspace/llama.cpp/build_ascend/bin/llama-server, --model, /model/Gemma2-FT-ai-medical-chatbot.Q8_0.gguf,\n --alias, llm, --threads, '16', --n-gpu-layers, '128', --prio, '3', --min_p,\n '0.01', --ctx-size, '4096', --host, 0.0.0.0, --port, '3316', --jinja, --flash-attn,\n 'off']\nref_config:\n gpu_num: 1\n values:\n command: [/workspace/llama.cpp/build/bin/llama-server, --model, /model/Gemma2-FT-ai-medical-chatbot.Q8_0.gguf,\n --alias, llm, --threads, '16', --n-gpu-layers, '128', --prio, '3', --min_p,\n '0.01', --ctx-size, '4096', --host, 0.0.0.0, --port, '80', --jinja]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 73}}
{"templateId": "text-generation-llamacpp-iluvatar-bi-150", "taskType": "text-generation", "framework": "llamacpp", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command: [/app/llama-server, --model, /model/Math-Code-Llama3.1-8B.i1-Q4_0.gguf,\n --alias, llm, --threads, '20', --n-gpu-layers, '128', --ctx-size, '4096', --host,\n 0.0.0.0, --port, '8000', --jinja, --flash-attn, 'off', --no-mmap, --sync-to-temp]\nref_config:\n gpu_num: 1\n values:\n command: [/workspace/llama.cpp/build/bin/llama-server, --model, /model/Math-Code-Llama3.1-8B.i1-Q4_0.gguf,\n --alias, llm, --threads, '20', --n-gpu-layers, '128', --ctx-size, '4096', --host,\n 0.0.0.0, --port, '8000', --jinja, --flash-attn, 'off']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 547}}
{"templateId": "text-generation-llamacpp-mthreads-s4000", "taskType": "text-generation", "framework": "llamacpp", "targetGpu": "Mthreads_s4000", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\nframework: llamacpp\napi: completion\nmax_tokens: 1024\nmax_model_len: 2048\nsut_config:\n gpu_num: 1\n values:\n command:\n - /app/llama-server\n - --model\n - /model/MediQwen-4B.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '2048'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/MediQwen-4B.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '2048'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 145}}
{"templateId": "text-generation-llamacpp-hygon-k100-ai", "taskType": "text-generation", "framework": "llamacpp", "targetGpu": "hygon_k100-ai", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\ndocker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_NV_DOCKER_IMAGE\napi: completion\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command: [/app/llama-server, --model, /model/qwen-qwen3-embedding-0.6b-f16.gguf,\n --alias, llm, --threads, '20', --n-gpu-layers, '999', --prio, '3', --min_p,\n '0.01', --ctx-size, '2048', --host, 0.0.0.0, --port, '8000', --jinja, --flash-attn,\n 'off']\nref_config:\n gpu_num: 1\n values:\n command: [/workspace/llama.cpp/build/bin/llama-server, --model, /model/qwen-qwen3-embedding-0.6b-f16.gguf,\n --alias, llm, --threads, '20', --n-gpu-layers, '128', --ctx-size, '4096', --host,\n 0.0.0.0, --port, '8000', --jinja]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 4}}
{"templateId": "text-generation-sglang-sunrise-pt-200-x1", "taskType": "text-generation", "framework": "sglang", "targetGpu": "Sunrise_pt-200-x1", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex-sunrise/sglang:0.4.6.post2\nnv_docker_image: harbor.4pd.io/hardcore-tech/public/sglang:v0.5.8\nframework: sglang\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n values:\n command: [python3.10, -m, sglang.launch_server, --model-path, /model, --port,\n '17097', --served-model-name, llm, --context-length, '4096', --trust-remote-code,\n --host, 0.0.0.0, --disable-cuda-graph, --disable-radix-cache]\nref_config:\n gpu_num: 1\n values:\n command: [python3, -m, sglang.launch_server, --model-path, /model, --port, '8000',\n --served-model-name, llm, --context-length, '4096', --trust-remote-code, --host,\n 0.0.0.0]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 442}}
{"templateId": "text-generation-transformers-iluvatar-bi-100", "taskType": "text-generation", "framework": "transformers", "targetGpu": "Iluvatar_bi-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor-contest.4pd.io/sunjichen/funasr:ilu-mr100-service-0.2\nnv_docker_image: harbor-contest.4pd.io/sunjichen/funasr:a100-server-0.2\napi: completion\nframework: transformers\nnv_framework: vllm\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForCausalLM\",\n \"torch_dtype\": \"auto\",\n \"model_kwargs\": {\n \"trust_remote_code\": true,\n \"device_map\": \"auto\"\n },\n \"tokenizer_kwargs\": {\n \"use_fast\": true,\n \"trust_remote_code\": true\n },\n \"generate_kwargs\": {\n \"temperature\": 0.01,\n \"top_p\": 0.8,\n \"repetition_penalty\": 1.0,\n \"max_new_tokens\": 1024\n }\n }\n command: [python3, transformers_server.py]\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 612}}
{"templateId": "text-generation-transformers-iluvatar-bi-150", "taskType": "text-generation", "framework": "transformers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\napi: completion\nframework: transformers\nnv_framework: vllm\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForCausalLM\",\n \"torch_dtype\": \"auto\",\n \"model_kwargs\": {\n \"trust_remote_code\": true,\n \"device_map\": \"auto\"\n },\n \"tokenizer_kwargs\": {\n \"use_fast\": true,\n \"trust_remote_code\": true\n },\n \"generate_kwargs\": {\n \"temperature\": 0.01,\n \"top_p\": 0.8,\n \"repetition_penalty\": 1.0,\n \"max_new_tokens\": 1024\n }\n }\n command:\n - python3\n - transformers_server.py\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 555}}
{"templateId": "text-generation-transformers-iluvatar-mrv-100", "taskType": "text-generation", "framework": "transformers", "targetGpu": "Iluvatar_mrv-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor-contest.4pd.io/sunjichen/funasr:ilu-mr100-service-0.2\nnv_docker_image: harbor-contest.4pd.io/sunjichen/funasr:a100-server-0.2\napi: completion\nframework: transformers\nnv_framework: vllm\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForCausalLM\",\n \"torch_dtype\": \"auto\",\n \"model_kwargs\": {\n \"trust_remote_code\": true,\n \"device_map\": \"auto\"\n },\n \"tokenizer_kwargs\": {\n \"use_fast\": true,\n \"trust_remote_code\": true\n },\n \"generate_kwargs\": {\n \"temperature\": 0.01,\n \"top_p\": 0.8,\n \"repetition_penalty\": 1.0,\n \"max_new_tokens\": 1024\n }\n }\n command: [python3, transformers_server.py]\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 653}}
{"templateId": "text-generation-vllm-ascend-910-b3", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Ascend_910-b3", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\nframework: vllm\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '20644'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 49}}
{"templateId": "text-generation-vllm-ascend-910-b4", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Ascend_910-b4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\ndocker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_NV_DOCKER_IMAGE\napi: completion\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nlang: zh\nmax_model_len: 2048\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - llm\n - --max-model-len\n - '2048'\n - --dtype\n - auto\n - --gpu-memory-utilization\n - '0.95'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - llm\n - --max-model-len\n - '2048'\n - --dtype\n - auto\n - --gpu-memory-utilization\n - '0.95'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 81}}
{"templateId": "text-generation-vllm-biren-166m", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Biren_166m", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\ndocker_image: git.modelhub.org.cn:9443/enginex/xc-llm-biren166m:26.01\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\napi: completion\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nlang: zh\nmax_model_len: 4096\nsut_config:\n gpu_num: 4\n values:\n command: [/bin/bash, -ic, vllm serve /model --port 8000 --served-model-name llm\n --max-model-len 4096 --dtype auto --gpu-memory-utilization 0.9 --enforce-eager\n --trust-remote-code -tp 4 --host 0.0.0.0]\nref_config:\n gpu_num: 4\n values:\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '4096', --dtype, auto, --gpu-memory-utilization, '0.95', -tp, '4', --enforce-eager,\n --trust-remote-code]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 137}}
{"templateId": "text-generation-vllm-cambricon-mlu-370-x4", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Cambricon_mlu-370-x4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor.4pd.io/hardcore-tech/cambricon-mlu370-pytorch:v25.01-torch2.5.0-torchmlu1.24.1-ubuntu22.04-py310\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\nsut_config:\n values:\n gpu_num: 1\n env:\n - {name: MAX_MODEL_LEN, value: 4096}\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '4096', --trust-remote-code, --dtype, float16]\nref_config:\n values:\n cpu_num: 2\n gpu_num: 1\n env:\n - {name: MAX_MODEL_LEN, value: 4096}\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --trust-remote-code, --dtype, float16]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 361}}
{"templateId": "text-generation-vllm-iluvatar-bi-100", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Iluvatar_bi-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command: [/workspace/launch_service, --port, '80']\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 620}}
{"templateId": "text-generation-vllm-iluvatar-bi-150", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex-iluvatar-bi150/vllm:0.8.3\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\napi: completion\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --gpu-memory-utilization, '0.9', --enforce-eager, --trust-remote-code,\n -tp, '1']\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 563}}
{"templateId": "text-generation-vllm-iluvatar-mrv-100", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Iluvatar_mrv-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor.4pd.io/hardcore-tech/iluvatar/llm-infer-iluvatar-mr:v0\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\nmax_model_len: 4096\nlang: en\ntemperature: 0.4\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '20644', --served-model-name, llm, --max-model-len,\n '4096', --gpu-memory-utilization, '0.9', --enforce-eager, --trust-remote-code,\n -tp, '1']\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 661}}
{"templateId": "text-generation-vllm-kunlunxin-p-800", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Kunlunxin_p-800", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\ndocker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_NV_DOCKER_IMAGE\napi: completion\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nlang: zh\nmax_model_len: 2048\nsut_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '2048', --dtype, auto, --gpu-memory-utilization, '0.95', -tp, '1', --enforce-eager,\n --trust-remote-code]\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '2048', --dtype, auto, --gpu-memory-utilization, '0.95', -tp, '1', --enforce-eager,\n --trust-remote-code]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 177}}
{"templateId": "text-generation-vllm-kunlunxin-r-200-8f", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Kunlunxin_r-200-8f", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor.4pd.io/mic-llm-x/slx-infer-kunlunxin:release-0.1-pipe-6-commit-f2eb860f\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\nmax_model_len: 4096\napi: completion\nsut_config:\n gpu_num: 1\n values:\n command: [sh, -c, rm -rf /workspace/xtrt_engine/fp16-1xpu && mkdir -p /workspace/xtrt_engine/fp16-1xpu\n && python3 examples/qwen/build.py --hf_model_dir /model --output_dir /workspace/xtrt_engine/fp16-1xpu\n --dtype float16 --use_gpt_attention_plugin float16 --world_size 1 --tp_size\n 1 --remove_input_padding --opt_memory_use --paged_kv_cache --tokens_per_block\n 64 --version 1.5 && exec python3 -m xtrt_llm.vllm.entrypoints.openai.api_server\n --model /model --engine_dir /workspace/xtrt_engine/fp16-1xpu --port 8000 --served-model-name\n llm --max-model-len 4096 --trust-remote-code --dtype float16]\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 434}}
{"templateId": "text-generation-vllm-metax-c-500", "taskType": "text-generation", "framework": "vllm", "targetGpu": "MetaX_c-500", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex-metax/vllm:0.9.1\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command: [/opt/conda/bin/vllm, serve, /model, --port, '20644', --served-model-name,\n llm, --max-model-len, '4096', --gpu-memory-utilization, '0.9', -tp, '1', --enforce-eager,\n --trust-remote-code]\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', -tp, '1', --enforce-eager, --trust-remote-code]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 475}}
{"templateId": "text-generation-vllm-mthreads-s4000", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Mthreads_s4000", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\nframework: vllm\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --dtype\n - auto\n - --gpu-memory-utilization\n - '0.95'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --dtype\n - auto\n - --gpu-memory-utilization\n - '0.95'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 153}}
{"templateId": "text-generation-vllm-sunrise-pt-200-x1", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Sunrise_pt-200-x1", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "api: completion\nframework: vllm\nmax_model_len: 4096\nmax_tokens: 1024\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '4096', --trust-remote-code, --host, 0.0.0.0, --enforce-eager]\n env:\n - {name: VLLM_ALLOW_LONG_MAX_MODEL_LEN, value: 1}\nsut_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '17097', --served-model-name, llm, --max-model-len,\n '4096', --trust-remote-code, --host, 0.0.0.0, --enforce-eager]\n env:\n - {name: VLLM_ALLOW_LONG_MAX_MODEL_LEN, value: 1}\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 450}}
{"templateId": "text-generation-vllm-vastai-va16", "taskType": "text-generation", "framework": "vllm", "targetGpu": "Vastai_va16", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex/xc-llm-va16:26.03\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\napi: completion\nlang: zh\nmax_model_len: 2048\nsut_config:\n values:\n gpu_num: 1\n env:\n - {name: MAX_MODEL_LEN, value: 2048}\n command: [/bin/bash, -ic, vllm serve /model --port 8000 --served-model-name llm\n --max-model-len 2048 --gpu-memory-utilization 0.9 --enforce-eager --trust-remote-code\n -tp 1 --host 0.0.0.0]\nref_config:\n values:\n cpu_num: 2\n gpu_num: 1\n env:\n - {name: MAX_MODEL_LEN, value: 2048}\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '2048', --enforce-eager, --trust-remote-code, -tp, '1']\nmodel: llm\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 249}}
{"templateId": "text-generation-vllm-hygon-k100-ai", "taskType": "text-generation", "framework": "vllm", "targetGpu": "hygon_k100-ai", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex-hygon/vllm:0.9.2\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\nmax_model_len: 4096\napi: completion\nsut_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '20644', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --enforce-eager, --trust-remote-code, -tp, '1']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 12}}
{"templateId": "text-generation-vllm-016-iluvatar-bi-150", "taskType": "text-generation", "framework": "vllm-016", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm-016\nlang: en\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 571}}
{"templateId": "text-generation-vllm-customized-cambricon-mlu-370-x4", "taskType": "text-generation", "framework": "vllm-customized", "targetGpu": "Cambricon_mlu-370-x4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor.4pd.io/hardcore-tech/cambricon-mlu370-pytorch:v25.01-20260204\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm-customized\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '4096', --dtype, half, --gpu-memory-utilization, '0.95', -tp, '1', --enforce-eager,\n --trust-remote-code]\nref_config:\n gpu_num: 1\n values:\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '4096', --dtype, auto, --gpu-memory-utilization, '0.95', -tp, '1', --enforce-eager,\n --trust-remote-code]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 369}}
{"templateId": "text-generation-vllm-mlu-cambricon-mlu-370-x4", "taskType": "text-generation", "framework": "vllm-mlu", "targetGpu": "Cambricon_mlu-370-x4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor.4pd.io/hardcore-tech/cambricon-mlu370-pytorch:v25.01-torch2.5.0-torchmlu1.24.1-ubuntu22.04-py310\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm-mlu\nsut_config:\n values:\n gpu_num: 1\n env:\n - {name: MAX_MODEL_LEN, value: 4096}\n command: [vllm, serve, /model, --port, '8000', --served-model-name, llm, --max-model-len,\n '4096', --trust-remote-code, --dtype, float16]\nref_config:\n values:\n cpu_num: 2\n gpu_num: 1\n env:\n - {name: MAX_MODEL_LEN, value: 4096}\n command: [vllm, serve, /model, --port, '80', --served-model-name, llm, --max-model-len,\n '4096', --trust-remote-code, --dtype, float16]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 377}}
{"templateId": "text-generation-vllm-fix-tokenizer-sunrise-pt-200-x1", "taskType": "text-generation", "framework": "vllm_fix_tokenizer", "targetGpu": "Sunrise_pt-200-x1", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm_fix_tokenizer\ndocker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_NV_DOCKER_IMAGE\napi: completion\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nlang: zh\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command: [/opt/entrypoint.sh, /model, --port, '8000', --served-model-name, llm,\n --max-model-len, '4096', --gpu-memory-utilization, '0.9', --enforce-eager, --trust-remote-code]\nref_config:\n gpu_num: 1\n values:\n command: [/opt/entrypoint.sh, /model, --port, '8000', --served-model-name, llm,\n --max-model-len, '4096', --dtype, float16, --gpu-memory-utilization, '0.95',\n -tp, '1', --enforce-eager, --trust-remote-code]\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 458}}
{"templateId": "text-to-image-generation-diffusers-iluvatar-bi-100", "taskType": "text-to-image-generation", "framework": "diffusers", "targetGpu": "Iluvatar_bi-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: harbor-contest.4pd.io/zhanghao/t2i:bi100-v0.3\nnv_docker_image: harbor-contest.4pd.io/zhanghao/t2i:a100-v0.3\nframework: diffusers\nsut_config:\n gpu_num: 1\n values:\n env:\n - {name: DTYPE, value: fp16}\n command: [/workspace/entrypoint.sh, --port, '80']\nref_config:\n gpu_num: 1\n values:\n env:\n - {name: DTYPE, value: fp16}\n command: [/workspace/entrypoint.sh, --port, '80']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 628}}
{"templateId": "text-to-image-generation-diffusers-iluvatar-bi-150", "taskType": "text-to-image-generation", "framework": "diffusers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: diffusers\nsut_config:\n values:\n gpu_num: 1\n env:\n - {name: DTYPE, value: fp16}\n command: [/workspace/entrypoint.sh, --port, '80']\nref_config:\n values:\n gpu_num: 1\n env:\n - {name: DTYPE, value: fp16}\n command: [/workspace/entrypoint.sh, --port, '80']\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 580}}
{"templateId": "text-to-image-generation-diffusers-iluvatar-mrv-100", "taskType": "text-to-image-generation", "framework": "diffusers", "targetGpu": "Iluvatar_mrv-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: diffusers\nsut_config:\n values:\n gpu_num: 1\n env:\n - name: DTYPE\n value: fp16\n command:\n - /workspace/entrypoint.sh\n - --port\n - '80'\nref_config:\n values:\n gpu_num: 1\n env:\n - name: DTYPE\n value: fp16\n command:\n - /workspace/entrypoint.sh\n - --port\n - '80'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 669}}
{"templateId": "text-classification-transformers-iluvatar-bi-150", "taskType": "text_classification", "framework": "transformers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "classification_type: sentiment/emotion/toxicity/hate (根据文本分类的任务类型四选一)\nframework: transformers\nnv_framework: transformers\nmodelhub_options:\n srcRelativePath: leaderboard/modelHubXC/nlptown/bert-base-multilingual-uncased-sentiment\n mountPoint: /model\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForSequenceClassification\",\n \"tokenizer_class\": \"AutoTokenizer\",\n \"torch_dtype\": \"auto\"\n }\n command:\n - python3\n - transformers_server.py\nref_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForSequenceClassification\",\n \"tokenizer_class\": \"AutoTokenizer\",\n \"torch_dtype\": \"auto\"\n }\n command:\n - python3\n - transformers_server.py\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 588}}
{"templateId": "vision-classification-transformers-iluvatar-bi-150", "taskType": "vision_classification", "framework": "transformers", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: transformers\nnv_framework: transformers\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForImageClassification\",\n \"processer_class\": \"AutoImageProcessor\",\n \"torch_dtype\": \"auto\"\n }\n command:\n - python3\n - transformers_server.py\nref_config:\n gpu_num: 1\n values:\n env:\n - name: CONFIG_JSON\n value: |\n {\n \"model_class\": \"AutoModelForImageClassification\",\n \"processer_class\": \"AutoImageProcessor\",\n \"torch_dtype\": \"auto\"\n }\n command:\n - python3\n - transformers_server.py\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 596}}
{"templateId": "visual-multi-modal-llamacpp-ascend-910-b3", "taskType": "visual-multi-modal", "framework": "llamacpp", "targetGpu": "Ascend_910-b3", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\nframework: llamacpp\napi: completion\nmax_tokens: 1024\nmax_model_len: 2048\nsut_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build_ascend/bin/llama-server\n - --model\n - /model/Free_Sydney_V2_Mistral_7b.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '2048'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/Free_Sydney_V2_Mistral_7b.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '2048'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 57}}
{"templateId": "visual-multi-modal-llamacpp-ascend-910-b4", "taskType": "visual-multi-modal", "framework": "llamacpp", "targetGpu": "Ascend_910-b4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\napi: chat\nlang: zh\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build_ascend/bin/llama-server\n - --model\n - /model/Gemma2-FT-ai-medical-chatbot.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '16'\n - --n-gpu-layers\n - '128'\n - --prio\n - '3'\n - --min_p\n - '0.01'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '3316'\n - --jinja\n - --flash-attn\n - 'off'\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/Gemma2-FT-ai-medical-chatbot.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '16'\n - --n-gpu-layers\n - '128'\n - --prio\n - '3'\n - --min_p\n - '0.01'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '80'\n - --jinja\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 89}}
{"templateId": "visual-multi-modal-llamacpp-mthreads-s4000", "taskType": "visual-multi-modal", "framework": "llamacpp", "targetGpu": "Mthreads_s4000", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command:\n - /app/llama-server\n - --model\n - /model/glm-4-9b-chat-1m.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/glm-4-9b-chat-1m.Q8_0.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 161}}
{"templateId": "visual-multi-modal-llamacpp-hygon-k100-ai", "taskType": "visual-multi-modal", "framework": "llamacpp", "targetGpu": "hygon_k100-ai", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: llamacpp\nmax_model_len: 1024\nsut_config:\n gpu_num: 1\n values:\n command:\n - /app/llama-server\n - --model\n - /model/Qwen3-14B-BF16.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\nref_config:\n gpu_num: 1\n values:\n command:\n - /workspace/llama.cpp/build/bin/llama-server\n - --model\n - /model/Qwen3-14B-BF16.gguf\n - --alias\n - llm\n - --threads\n - '20'\n - --n-gpu-layers\n - '128'\n - --ctx-size\n - '4096'\n - --host\n - 0.0.0.0\n - --port\n - '8000'\n - --jinja\n - --flash-attn\n - 'off'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 22}}
{"templateId": "visual-multi-modal-vllm-ascend-910-b3", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "Ascend_910-b3", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_DOCKER_IMAGE\nframework: vllm\napi: completion\nlang: en\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.1\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '20644'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 65}}
{"templateId": "visual-multi-modal-vllm-ascend-910-b4", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "Ascend_910-b4", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\ndocker_image: PLACEHOLDER_DOCKER_IMAGE\nnv_docker_image: PLACEHOLDER_NV_DOCKER_IMAGE\napi: completion\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nlang: zh\nmax_model_len: 2048\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - llm\n - --max-model-len\n - '2048'\n - --dtype\n - auto\n - --gpu-memory-utilization\n - '0.95'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - llm\n - --max-model-len\n - '2048'\n - --dtype\n - auto\n - --gpu-memory-utilization\n - '0.95'\n - -tp\n - '1'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 97}}
{"templateId": "visual-multi-modal-vllm-iluvatar-bi-100", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "Iluvatar_bi-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: SERVED_MODEL_NAME\n value: vlm\n command:\n - /workspace/launch_service\n - --port\n - '80'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - vlm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 636}}
{"templateId": "visual-multi-modal-vllm-iluvatar-bi-150", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "Iluvatar_bi-150", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "docker_image: git.modelhub.org.cn:9443/enginex-iluvatar-bi150/vllm:0.8.3\nnv_docker_image: harbor.4pd.io/dooke/vllm/vllm/vllm-openai:v0.11.0\nframework: vllm\napi: completion\nmax_model_len: 4096\nmax_tokens: 1024\ntemperature: 0.7\nrepetition_penalty: 1.2\ntop_p: 0.9\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --gpu-memory-utilization\n - '0.9'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 604}}
{"templateId": "visual-multi-modal-vllm-iluvatar-mrv-100", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "Iluvatar_mrv-100", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n env:\n - name: SERVED_MODEL_NAME\n value: vlm\n command:\n - /workspace/launch_service\n - --port\n - '80'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - vlm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 677}}
{"templateId": "visual-multi-modal-vllm-metax-c-500", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "MetaX_c-500", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\nnv_framework: vllm_v10\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command:\n - /opt/conda/bin/vllm\n - serve\n - /model\n - --port\n - '8898'\n - --served-model-name\n - vlm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '8000'\n - --served-model-name\n - vlm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 483}}
{"templateId": "visual-multi-modal-vllm-mthreads-s4000", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "Mthreads_s4000", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framewok: vllm\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --served-model-name\n - llm\n - --trust-remote-code\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --gpu-memory-utilization\n - '0.5'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --served-model-name\n - llm\n - --trust-remote-code\n - --max-model-len\n - '4096'\n - --enforce-eager\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 169}}
{"templateId": "visual-multi-modal-vllm-hygon-k100-ai", "taskType": "visual-multi-modal", "framework": "vllm", "targetGpu": "hygon_k100-ai", "modelAddressTemplate": "{{MODEL_ADDRESS}}", "sampleModelAddress": "https://huggingface.co/Qwen/Qwen3.6-35B-A3B", "configParams": "framework: vllm\nmax_model_len: 4096\nsut_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '20644'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\nref_config:\n gpu_num: 1\n values:\n command:\n - vllm\n - serve\n - /model\n - --port\n - '80'\n - --served-model-name\n - llm\n - --max-model-len\n - '4096'\n - --enforce-eager\n - --trust-remote-code\n - -tp\n - '1'\n", "source": {"file": "/modelHub/网页版提交抓包样例.md", "line": 31}}

279
modelhub_submmit_api/templates/public_submit/template_index.json Normal file
View File

@@ -0,0 +1,279 @@
{
"sourceFile": "/modelHub/网页版提交抓包样例.md",
"totalExtractedSamples": 84,
"totalUniqueTemplates": 53,
"taskTypes": {
"asr": {
"funasr": [
{
"templateId": "asr-funasr-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
},
{
"templateId": "asr-funasr-iluvatar-mrv-100",
"targetGpu": "Iluvatar_mrv-100"
}
],
"sherpa-onnx": [
{
"templateId": "asr-sherpa-onnx-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
],
"transformers": [
{
"templateId": "asr-transformers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
]
},
"feature_emb": {
"sentence-transformers": [
{
"templateId": "feature-emb-sentence-transformers-metax-c-500",
"targetGpu": "MetaX_c-500"
}
]
},
"question_answering": {
"transformers": [
{
"templateId": "question-answering-transformers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
]
},
"reinforcement_learning": {
"llamacpp": [
{
"templateId": "reinforcement-learning-llamacpp-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
],
"transformers": [
{
"templateId": "reinforcement-learning-transformers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
],
"vllm": [
{
"templateId": "reinforcement-learning-vllm-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
]
},
"text-generation": {
"llamacpp": [
{
"templateId": "text-generation-llamacpp-ascend-910-b3",
"targetGpu": "Ascend_910-b3"
},
{
"templateId": "text-generation-llamacpp-ascend-910-b4",
"targetGpu": "Ascend_910-b4"
},
{
"templateId": "text-generation-llamacpp-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
},
{
"templateId": "text-generation-llamacpp-mthreads-s4000",
"targetGpu": "Mthreads_s4000"
},
{
"templateId": "text-generation-llamacpp-hygon-k100-ai",
"targetGpu": "hygon_k100-ai"
}
],
"sglang": [
{
"templateId": "text-generation-sglang-sunrise-pt-200-x1",
"targetGpu": "Sunrise_pt-200-x1"
}
],
"transformers": [
{
"templateId": "text-generation-transformers-iluvatar-bi-100",
"targetGpu": "Iluvatar_bi-100"
},
{
"templateId": "text-generation-transformers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
},
{
"templateId": "text-generation-transformers-iluvatar-mrv-100",
"targetGpu": "Iluvatar_mrv-100"
}
],
"vllm": [
{
"templateId": "text-generation-vllm-ascend-910-b3",
"targetGpu": "Ascend_910-b3"
},
{
"templateId": "text-generation-vllm-ascend-910-b4",
"targetGpu": "Ascend_910-b4"
},
{
"templateId": "text-generation-vllm-biren-166m",
"targetGpu": "Biren_166m"
},
{
"templateId": "text-generation-vllm-cambricon-mlu-370-x4",
"targetGpu": "Cambricon_mlu-370-x4"
},
{
"templateId": "text-generation-vllm-iluvatar-bi-100",
"targetGpu": "Iluvatar_bi-100"
},
{
"templateId": "text-generation-vllm-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
},
{
"templateId": "text-generation-vllm-iluvatar-mrv-100",
"targetGpu": "Iluvatar_mrv-100"
},
{
"templateId": "text-generation-vllm-kunlunxin-p-800",
"targetGpu": "Kunlunxin_p-800"
},
{
"templateId": "text-generation-vllm-kunlunxin-r-200-8f",
"targetGpu": "Kunlunxin_r-200-8f"
},
{
"templateId": "text-generation-vllm-metax-c-500",
"targetGpu": "MetaX_c-500"
},
{
"templateId": "text-generation-vllm-mthreads-s4000",
"targetGpu": "Mthreads_s4000"
},
{
"templateId": "text-generation-vllm-sunrise-pt-200-x1",
"targetGpu": "Sunrise_pt-200-x1"
},
{
"templateId": "text-generation-vllm-vastai-va16",
"targetGpu": "Vastai_va16"
},
{
"templateId": "text-generation-vllm-hygon-k100-ai",
"targetGpu": "hygon_k100-ai"
}
],
"vllm-016": [
{
"templateId": "text-generation-vllm-016-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
],
"vllm-customized": [
{
"templateId": "text-generation-vllm-customized-cambricon-mlu-370-x4",
"targetGpu": "Cambricon_mlu-370-x4"
}
],
"vllm-mlu": [
{
"templateId": "text-generation-vllm-mlu-cambricon-mlu-370-x4",
"targetGpu": "Cambricon_mlu-370-x4"
}
],
"vllm_fix_tokenizer": [
{
"templateId": "text-generation-vllm-fix-tokenizer-sunrise-pt-200-x1",
"targetGpu": "Sunrise_pt-200-x1"
}
]
},
"text-to-image-generation": {
"diffusers": [
{
"templateId": "text-to-image-generation-diffusers-iluvatar-bi-100",
"targetGpu": "Iluvatar_bi-100"
},
{
"templateId": "text-to-image-generation-diffusers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
},
{
"templateId": "text-to-image-generation-diffusers-iluvatar-mrv-100",
"targetGpu": "Iluvatar_mrv-100"
}
]
},
"text_classification": {
"transformers": [
{
"templateId": "text-classification-transformers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
]
},
"vision_classification": {
"transformers": [
{
"templateId": "vision-classification-transformers-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
}
]
},
"visual-multi-modal": {
"llamacpp": [
{
"templateId": "visual-multi-modal-llamacpp-ascend-910-b3",
"targetGpu": "Ascend_910-b3"
},
{
"templateId": "visual-multi-modal-llamacpp-ascend-910-b4",
"targetGpu": "Ascend_910-b4"
},
{
"templateId": "visual-multi-modal-llamacpp-mthreads-s4000",
"targetGpu": "Mthreads_s4000"
},
{
"templateId": "visual-multi-modal-llamacpp-hygon-k100-ai",
"targetGpu": "hygon_k100-ai"
}
],
"vllm": [
{
"templateId": "visual-multi-modal-vllm-ascend-910-b3",
"targetGpu": "Ascend_910-b3"
},
{
"templateId": "visual-multi-modal-vllm-ascend-910-b4",
"targetGpu": "Ascend_910-b4"
},
{
"templateId": "visual-multi-modal-vllm-iluvatar-bi-100",
"targetGpu": "Iluvatar_bi-100"
},
{
"templateId": "visual-multi-modal-vllm-iluvatar-bi-150",
"targetGpu": "Iluvatar_bi-150"
},
{
"templateId": "visual-multi-modal-vllm-iluvatar-mrv-100",
"targetGpu": "Iluvatar_mrv-100"
},
{
"templateId": "visual-multi-modal-vllm-metax-c-500",
"targetGpu": "MetaX_c-500"
},
{
"templateId": "visual-multi-modal-vllm-mthreads-s4000",
"targetGpu": "Mthreads_s4000"
},
{
"templateId": "visual-multi-modal-vllm-hygon-k100-ai",
"targetGpu": "hygon_k100-ai"
}
]
}
}
}