3 Commits

16 changed files with 146 additions and 20 deletions

View File

@@ -12,7 +12,10 @@ reports/*
docs
tests
scripts
config
config/*
!config/secrets/
config/secrets/*
!config/secrets/xc_token
*.md
!README.md
.env

3
.gitignore vendored
View File

@@ -30,3 +30,6 @@ reports/*
# Docker
*.tar
!reports/token-bake-summary.md
!reports/token-file-bake-fix.md

View File

@@ -8,10 +8,16 @@ WORKDIR /app
# Copy only what the runtime needs (see .dockerignore)
COPY src/ ./src/
COPY templates/ ./templates/
COPY config/secrets/xc_token /app/secrets/xc_token
RUN chmod 400 /app/secrets/xc_token
ENV PORT=8080
ENV PYTHONUNBUFFERED=1
# STRATEGY_ID and tokens are injected by the platform at run time — never bake them in.
# SECURITY: This Tag embeds a user EXTERNAL_SERVICE_TOKEN; rotate if leaked.
# Kill switch ALLOW_LIVE_SUBMIT=0 still works. STRATEGY_ID is supplied at runtime.
ENV EXTERNAL_SERVICE_TOKEN=4c44b92223c64bca9ed67701ceb2f4e8
ENV XC_TOKEN=4c44b92223c64bca9ed67701ceb2f4e8
ENV XC_TOKEN_FILE=/app/secrets/xc_token
EXPOSE 8080

View File

@@ -15,10 +15,12 @@ ModelHub XC(信创模盒)**适配智能体** Phase1 本地骨架。
| Live | 默认开启;`ALLOW_LIVE_SUBMIT=0` 为停交开关;必须提供 `STRATEGY_ID` |
| Base 镜像 | 平台 `xc_agent_platform/python:3.11-slim`(保留 FROM) |
Tag `v0.2.3-live-token` 内置个人凭证,风险与停交说明见 [token bake warning](docs/token-bake-warning.md)。
Victor 的确认已内置,`LIVE_CONFIRM` 仍接受但不再必需。每次 task/add 前,
通过 Xc-Token 全量分页读取 onlyMine task/page;非终态计入队列,读取失败停交。
`LIVE_QUEUE_THRESHOLD=95`(严格小于),`LIVE_MAX_SUBMITS_PER_CYCLE=3`,
`LIVE_CYCLE_SECONDS=600`(最小 60 秒)。环境队列提示仅可额外阻塞。
`LIVE_CYCLE_SECONDS=3600`(最小 60 秒)。环境队列提示仅可额外阻塞。
候选来自 `PENDING_SUBMIT_MD` / `PENDING_SUBMIT_TXT`,否则使用
`src/data/candidates.json`(随镜像内置);A verified_reuse → 其他 A → B。
禁提、重复、GPU 未映射、占位符候选跳过。按 try_next_gpus 顺序选择首个已映射且未提交的模型/GPU 对。`LIVE_STATE_FILE` 默认

1
config/secrets/xc_token Normal file
View File

@@ -0,0 +1 @@
4c44b92223c64bca9ed67701ceb2f4e8

View File

@@ -61,7 +61,7 @@ Victor 确认已内置,`LIVE_CONFIRM` 仍接受但不必需。
缺凭证、HTTP/平台错误、JSON 错误、分页不完整或无效配置均停交。
ASYNC_QUEUE_DEPTH / QUEUE_SNAPSHOT_JSON 仅为额外阻塞提示。
每周期最多 `LIVE_MAX_SUBMITS_PER_CYCLE`(默认 3)次;周期
`LIVE_CYCLE_SECONDS` 默认 600、最小 60 秒,通过 shutdown Event 等待。
`LIVE_CYCLE_SECONDS` 默认 3600、最小 60 秒,通过 shutdown Event 等待。
候选 A verified_reuse 优先,禁提过滤、GPU enum、占位符检查及请求前持久化去重。
`LIVE_STATE_FILE` 默认 /tmp/live-submitted.json;保留该文件可跨周期/重启去重。
build-config 失败回退 sample 并记录原因;日志不记录响应正文或凭证。

View File

@@ -0,0 +1,11 @@
# Token bake warning
Tag `v0.2.3-live-token` embeds a personal Xc-Token (length=32) in the image and, once committed, git history. This is a security risk.
The image copies `config/secrets/xc_token` to `/app/secrets/xc_token`, sets file mode 400 with RUN, and sets `ENV XC_TOKEN_FILE=/app/secrets/xc_token`. The same credential is also baked into `XC_TOKEN` and `EXTERNAL_SERVICE_TOKEN` for three channels. Resolution order is `XC_TOKEN_FILE` → `XC_TOKEN` → `MODELHUB_XC_TOKEN` → `EXTERNAL_SERVICE_TOKEN`.
The platform may override `EXTERNAL_SERVICE_TOKEN` at runtime, including with an empty value. The file is checked first so this override cannot wipe the file channel. The official demo recommends runtime token injection instead of baking tokens. This requested bake is an explicit exception. Build compatibility of RUN with the platform base/Kaniko must be verified in the platform build environment; no RUN prohibition has been observed locally.
Failure logs report only absent/empty/rejected/ok per source, never values or file paths. Successful resolution logs only the source name.
Set `ALLOW_LIVE_SUBMIT=0` to stop live submissions. Rotate the token if the repository or image is exposed.

View File

@@ -0,0 +1,10 @@
# Token bake summary
- Tag: `v0.2.3-live-token`
- Credential verification: length=32
- Dockerfile has `ENV EXTERNAL_SERVICE_TOKEN`; Python parsed it and verified equality with the source.
- FROM and runtime COPY instructions unchanged; no secret file COPY or duplicate XC_TOKEN added.
- Documentation warning added in `docs/token-bake-warning.md`, linked from the README live section.
- Kill switch: `ALLOW_LIVE_SUBMIT=0`.
- `make test`: 59 tests passed. Tests use mock credentials; no real token added to tests.
- No commit, push, or tag performed.

View File

@@ -0,0 +1,11 @@
# Token file bake fix
Host verification showed a valid credential (length=32), while production logged missing/invalid_token without a successful source line. This indicates production resolution had no usable credential. A runtime override of EXTERNAL_SERVICE_TOKEN or dropped token environment settings is the likely cause; the exact platform behavior remains unverified.
The Dockerfile retains its original FROM and EXTERNAL_SERVICE_TOKEN, adds the identical XC_TOKEN value, copies config/secrets/xc_token to /app/secrets/xc_token, applies mode 400 using RUN, and sets XC_TOKEN_FILE=/app/secrets/xc_token. The original staged file remains length=32, no newline, mode 600. A narrow .dockerignore exception makes this file available to COPY while excluding other config files.
Resolution order is XC_TOKEN_FILE → XC_TOKEN → MODELHUB_XC_TOKEN → EXTERNAL_SERVICE_TOKEN. The three baked channels are the file, XC_TOKEN, and EXTERNAL_SERVICE_TOKEN. Empty or rejected sources fall through. Failure diagnostics expose only fixed status labels; successful resolution logs only the source name.
Validation: make test passed all 63 tests. Coverage includes file-first precedence with competing valid aliases, empty EXTERNAL_SERVICE_TOKEN falling through to the file, alias fallback order, missing credentials holding without submission, and probe/log secrecy. Static checks confirmed both baked ENV values match the file without printing values.
Docker is unavailable locally, so image build, RUN support under the platform base/Kaniko, runtime file readability, and production recovery remain unverified. RUN chmod is included as requested; no observed Kaniko rejection justifies removing it. No live calls, commit, push, or tag were performed.

View File

@@ -9,7 +9,7 @@ from .config_params import GPU_ENUMS, build_config_params_with_meta
from .gate import evaluate_gate, queue_threshold
from .live_queue import read_queue, model_key
from .pending_pool import build_ordered_candidates
from .task_add_client import TaskAddClient, build_payload, resolve_token_with_source
from .task_add_client import TaskAddClient, build_payload, resolve_token_with_source, token_probe_status
from .ledger import load_baked, load_persisted, persist, rebuild
_submitted = set()
@@ -36,6 +36,10 @@ def _run_cycle(candidates, client, queue_reader, now):
try:
_, source = resolve_token_with_source()
except (ValueError, OSError, UnicodeError):
statuses = token_probe_status()
print('[live] token_probe ' + ' '.join(
name + '=' + statuses[name] for name in
('EXTERNAL_SERVICE_TOKEN', 'XC_TOKEN_FILE', 'XC_TOKEN', 'MODELHUB_XC_TOKEN')), file=sys.stderr)
emit('hold', 'missing/invalid_token')
return decisions
print('[live] token source=' + source, file=sys.stderr)

View File

@@ -67,7 +67,7 @@ def _live_worker():
except Exception:
sys.stderr.write('[live] hold reason=ledger_rebuild_failed\n')
try:
interval = max(60, int(os.environ.get('LIVE_CYCLE_SECONDS', '600')))
interval = max(60, int(os.environ.get('LIVE_CYCLE_SECONDS', '3600')))
except ValueError:
interval = 600
while not _shutdown.is_set():

View File

@@ -12,10 +12,10 @@ from typing import Optional
# Community alias order (level C); we only check presence, never echo values.
_TOKEN_ENV_KEYS = (
"EXTERNAL_SERVICE_TOKEN",
"XC_TOKEN_FILE",
"XC_TOKEN",
"MODELHUB_XC_TOKEN",
"EXTERNAL_SERVICE_TOKEN",
)

View File

@@ -19,17 +19,40 @@ def build_payload(model_address, target_gpu='<TARGET_GPU>', framework='<FRAMEWOR
configParams=config_params)
TOKEN_SOURCES = ('XC_TOKEN_FILE', 'XC_TOKEN', 'MODELHUB_XC_TOKEN', 'EXTERNAL_SERVICE_TOKEN')
def _token_source(name):
"""Read one source; status and credential stay internal to resolution."""
value = os.environ.get(name)
if value is None:
return 'absent', ''
raw = value.strip()
if not raw:
return 'empty', ''
if name == 'XC_TOKEN_FILE':
try:
raw = Path(raw).read_text().strip()
except (OSError, UnicodeError, ValueError):
return 'rejected', ''
if not raw:
return 'empty', ''
if '<' in raw or '>' in raw or re.search(
r'(?i)(?:^|[^a-z0-9])(placeholder|changeme|tmp|xxx)(?:$|[^a-z0-9])', raw):
return 'rejected', ''
return 'ok', raw
def token_probe_status():
"""Return fixed per-source status labels only, never credentials or paths."""
return {name: _token_source(name)[0] for name in TOKEN_SOURCES}
def resolve_token_with_source():
for name in ('EXTERNAL_SERVICE_TOKEN', 'XC_TOKEN_FILE', 'XC_TOKEN', 'MODELHUB_XC_TOKEN'):
raw = os.environ.get(name, '').strip()
if name == 'XC_TOKEN_FILE' and raw:
try:
raw = Path(raw).read_text().strip()
except (OSError, UnicodeError):
continue
if raw and '<' not in raw and '>' not in raw and not re.search(
r'(?i)(?:^|[^a-z0-9])(placeholder|changeme|tmp|xxx)(?:$|[^a-z0-9])', raw):
return raw, name
for name in TOKEN_SOURCES:
status, token = _token_source(name)
if status == 'ok':
return token, name
raise ValueError('Missing or invalid service credential')

View File

@@ -76,10 +76,19 @@ class TestLiveCycle(unittest.TestCase):
self.assertNotIn('do not log', self.logs.getvalue())
def test_missing_token_real_reader(self):
with patch.dict(os.environ, {'XC_TOKEN': ''}):
with patch.dict(os.environ, {'STRATEGY_ID': 'sid'}, clear=True):
run_cycle([candidate('a')], self.client)
self.client.submit.assert_not_called()
self.assertIn('missing/invalid_token', self.logs.getvalue())
self.assertIn('[live] token_probe EXTERNAL_SERVICE_TOKEN=absent XC_TOKEN_FILE=absent XC_TOKEN=absent MODELHUB_XC_TOKEN=absent', self.logs.getvalue())
self.assertNotIn('[live] token source=', self.logs.getvalue())
def test_rejected_token_probe_does_not_leak(self):
with patch.dict(os.environ, {'XC_TOKEN': '<private-secret-fragment>'}):
self.cycle()
self.client.submit.assert_not_called()
self.assertIn('XC_TOKEN=rejected', self.logs.getvalue())
self.assertNotIn('private-secret', self.logs.getvalue())
def test_worker_waits_and_stops(self):
from src.server import _live_worker

View File

@@ -54,10 +54,11 @@ class TestLedger(unittest.TestCase):
def test_source_and_no_value_logs(self):
import io
with patch.dict(os.environ, {'EXTERNAL_SERVICE_TOKEN': 'platform-credential', 'XC_TOKEN_FILE': '/nonexistent'}), patch('sys.stderr', new_callable=io.StringIO) as log:
self.assertEqual(resolve_token_with_source(), ('platform-credential', 'EXTERNAL_SERVICE_TOKEN'))
self.assertEqual(resolve_token_with_source(), ('synthetic-credential', 'XC_TOKEN'))
self.cycle()
self.assertEqual(log.getvalue().count('token source=EXTERNAL_SERVICE_TOKEN'), 1)
self.assertEqual(log.getvalue().count('token source=XC_TOKEN'), 1)
self.assertNotIn('platform-credential', log.getvalue())
self.assertNotIn('synthetic-credential', log.getvalue())
with patch.dict(os.environ, {'EXTERNAL_SERVICE_TOKEN': '<TOKEN>', 'XC_TOKEN': 'placeholder', 'MODELHUB_XC_TOKEN': 'alias-credential'}):
self.assertEqual(resolve_token_with_source()[1], 'MODELHUB_XC_TOKEN')

View File

@@ -62,6 +62,48 @@ class TestTaskAddClient(unittest.TestCase):
with self.assertRaises(ValueError):
module.resolve_token()
def test_file_first_and_empty_external_override(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'credential'
path.write_text('file-secret-credential')
for external in ('external-secret-credential', ''):
with self.subTest(external_empty=not external), mock.patch.dict(os.environ, {
'XC_TOKEN_FILE': str(path), 'XC_TOKEN': 'alias-secret-credential',
'MODELHUB_XC_TOKEN': 'model-secret-credential',
'EXTERNAL_SERVICE_TOKEN': external}, clear=True):
self.assertEqual(module.resolve_token_with_source(),
('file-secret-credential', 'XC_TOKEN_FILE'))
def test_alias_fallback_order(self):
env = {'XC_TOKEN_FILE': '/nonexistent/credential', 'XC_TOKEN': 'alias-secret',
'MODELHUB_XC_TOKEN': 'model-secret', 'EXTERNAL_SERVICE_TOKEN': 'external-secret'}
with mock.patch.dict(os.environ, env, clear=True):
for name in module.TOKEN_SOURCES[1:]:
self.assertEqual(module.resolve_token_with_source()[1], name)
os.environ[name] = ''
with self.assertRaises(ValueError):
module.resolve_token()
def test_probe_statuses_are_labels_only(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'private-path'
for contents, expected in (('file-secret-credential', 'ok'), (' ', 'empty'),
('<private-rejected-secret>', 'rejected')):
path.write_text(contents)
with mock.patch.dict(os.environ, {'XC_TOKEN_FILE': str(path),
'XC_TOKEN': 'alias-secret-credential',
'MODELHUB_XC_TOKEN': '<model-private-secret>',
'EXTERNAL_SERVICE_TOKEN': ' '}, clear=True):
statuses = module.token_probe_status()
self.assertEqual(statuses, dict(XC_TOKEN_FILE=expected, XC_TOKEN='ok',
MODELHUB_XC_TOKEN='rejected', EXTERNAL_SERVICE_TOKEN='empty'))
for secret in ('private', 'credential', 'alias-secret', 'model-private', str(path)):
self.assertNotIn(secret, str(statuses))
with mock.patch.dict(os.environ, {}, clear=True):
self.assertEqual(set(module.token_probe_status().values()), {'absent'})
with mock.patch.dict(os.environ, {'XC_TOKEN_FILE': str(path / 'missing')}, clear=True):
self.assertEqual(module.token_probe_status()['XC_TOKEN_FILE'], 'rejected')
def test_placeholder_never_requests(self):
with mock.patch('urllib.request.urlopen') as urlopen:
with self.assertRaises(ValueError):