dylanyunlong/project_7
1
0
Fork 0
Code Issues Pull Requests Actions Projects Releases Wiki Activity
3 Commits 1 Branch 0 Tags
f1a408c75f0ca7b21c53e631367351e20a1cdc86
Commit Graph

3 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
project6-dev
f1a408c75f fix: cap default_max_tokens at 8192 — prevent OOM kill on unlimited generation 
t3_max_tokens_none test sends request without max_tokens.
With max_model_len=262144, default_max_tokens was ~260K tokens.
Model generates indefinitely at 12 TPS, container gets OOM killed after ~6min.
Cap at 8192 prevents memory exhaustion while still allowing long outputs.
 2026-08-12 04:06:21 +00:00
project6-dev
dbc1ce9a71 fix: max-num-seqs=2 — t2_n_2测试需要n=2多路生成 2026-08-12 03:58:20 +00:00
project6-dev
4ade8136ad wudixzy stack docker build test: 2615-line qwen3_5.py + 12 prebuilt .so + 251-line patch_ops.sh 2026-08-12 03:17:31 +00:00