16 Commits

Author SHA1 Message Date
251f21ac8f add DOWNLOAD_ONLY mode: run zhoushasha downloads without submitting any validation task
All 498 models in the current list already had their hygon and bi150 slots
reserved in earlier rounds, so this round only needs the downloads. Without a
switch the download list would be empty, since phase 2 only downloads models
reserved in the same run.

DOWNLOAD_ONLY skips the reservation phase entirely and downloads all of
ALL_MODEL_IDS. Also splits the worker into _reserve_phase / _download_phase and
gates bi150 behind ENABLE_BI150_SUBMIT for symmetry with hygon.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 15:56:45 +08:00
fae53f5a2a add a dedicated bi150 fallback account and skip hygon this round
bi150 coverage lagged hygon: the zero-credit accounts each take only one model,
so once all 34 were used the remaining models got no bi150 at all. Add
zhangyuanxi as the fallback so bi150 can catch up to hygon's coverage, using a
separate account from zhoukaile so the two cards don't compete for one quota.

hygon is already submitted for every model in the current 498-model list, so
ENABLE_HYGON_SUBMIT is off this round; flip it back on with a new model list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 15:39:26 +08:00
1155ac4f4e reserve validation slots before downloading, to stop competitors front-running our downloads
The download queue (/adminApi/async/task/model-download-task) is readable by ANY
account, including low-privilege xc-Token ones, so our WAITING model ids leak.
Competitors submit those via mechanism B (which needs no download), their task
waits for the file, and once our zhoushasha download lands it runs and wins the
slot - leaving us with 60028 on our own download.

New flow:
  phase 1 (reserve, instant): bi150 via zero-credit accounts that have no bi150
    task yet, hygon via zhoukaile - both mechanism B, no download required
  phase 2 (download): zhoushasha downloads only the models we reserved

Startup now filters out accounts that already hold a bi150 task, which also makes
the rotation cursor idempotent across redeploys.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:33:04 +08:00
f8750be928 replace BI150_ACCOUNTS with 34 zero-credit referral accounts; refresh expired AUTH_TOKEN
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:41:08 +08:00
cea6924355 replace BI150_ACCOUNTS with 50 zero-credit referral accounts; advance rotation cursor only on successful submission; refresh AUTH_TOKEN 2026-09-08 14:48:55 +08:00
904893671f add bi150 submission via mechanism B with 24-account rotation (one model each, then fallback to l11223344); rotate model list (1901 models) 2026-09-02 11:58:56 +08:00
961ed40fc8 refresh expired AUTH_TOKEN (valid to 2026-09-07) 2026-08-31 11:26:44 +08:00
cbac5457ac rotate model list: comment out completed 8-14 batch, activate next batch (1936 models) 2026-08-27 14:03:26 +08:00
8556c7efc4 switch hygon submission to mechanism B (xc-Token, account l11223344); update model list (2040 models) 2026-08-14 15:39:57 +08:00
a9fe1f6393 update model list
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 17:03:42 +08:00
9aba1595db init 2026-07-14 16:44:33 +08:00
85f41bba58 Replace main.py with GGUF download + validation submit pipeline
Port logic from continuous_pipeline_download_gguf_zhoushasha_and_submit.py
into the strategy-deployable framework (health/status HTTP server,
STRATEGY_ID env, SIGTERM handling).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 15:45:51 +08:00
d6b0e416db update ppu 2026-07-13 19:43:35 +08:00
4dcfed6b6d update ppu 2026-07-13 18:38:55 +08:00
1e8cfacd8e uodate 2026-06-22 19:00:46 +08:00
d6cca90496 update main.py 2026-06-22 18:44:42 +08:00
4 changed files with 2887 additions and 421 deletions

2
.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
.DS_Store
__pycache__/

View File

@@ -1,6 +1,7 @@
FROM modelhubxc-4pd.tencentcloudcr.com/xc_agent_platform/python:3.11-slim FROM modelhubxc-4pd.tencentcloudcr.com/xc_agent_platform/python:3.11-slim
ENV PYTHONUNBUFFERED=1 ENV PYTHONUNBUFFERED=1
WORKDIR /app WORKDIR /app

View File

@@ -1,22 +1,24 @@
# xc_validation_strategy # xc_validation_strategy_gguf
批量向 ModelHub XC 平台提交模型验证任务的策略服务,之后保持 HTTP 服务存活供平台探活。 GGUF 模型下载 + 验证任务提交流水线策略服务:批量创建 GGUF 模型下载任务(最大并发 8
每个模型下载成功后立即提交 hygon / bi150 两个验证任务,之后保持 HTTP 服务存活供平台探活。
## 功能 ## 功能
- 自动登录 ModelHub 获取 Token - 自动登录 ModelHub 获取 Token(失败时回退到预设 Token
- 批量提交模型验证任务vLLM 框架Cambricon MLU-370-x8 - 按流水线批量创建 GGUF 模型下载任务HuggingFace 源,最大并发 8
- 提交结果写入 `submitted_validation_tasks.txt` - 每个模型下载成功后,立即提交 hygon_k100-ai 与 Iluvatar_bi-150 两个验证任务llamacpp 框架)
- 下载成功的模型 ID 写入 `downloaded_success_models.txt`
- 暴露 `/health``/status` 接口满足平台运行时契约 - 暴露 `/health``/status` 接口满足平台运行时契约
## 项目结构 ## 项目结构
``` ```
. .
├── main.py # 主入口HTTP 服务 + 提交逻辑 ├── main.py # 主入口HTTP 服务 + 下载/提交流水线
├── Dockerfile # 平台镜像构建配置 ├── Dockerfile # 平台镜像构建配置
├── requirements.txt # Python 依赖 ├── requirements.txt # Python 依赖
└── submitted_validation_tasks.txt # 运行后自动生成,记录提交结果 └── downloaded_success_models.txt # 运行后自动生成,记录下载成功的模型
``` ```
## 平台契约说明 ## 平台契约说明
@@ -26,4 +28,4 @@
- Dockerfile 位于仓库根目录,基于官方轻量基础镜像 - Dockerfile 位于仓库根目录,基于官方轻量基础镜像
- 暴露 8080 端口并实现 `GET /health` - 暴露 8080 端口并实现 `GET /health`
- 通过环境变量 `STRATEGY_ID` 获取策略 ID - 通过环境变量 `STRATEGY_ID` 获取策略 ID
- 正确处理 `SIGTERM` 信号,支持优雅停机 - 正确处理 `SIGTERM` 信号,支持优雅停机

3285
main.py

File diff suppressed because it is too large Load Diff