project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
Claude	e86b7eacdd	refactor(tool_parser): CCCL agent_radix_sort_downsweep union TempStorage — phased state Applies CCCL cub/agent/agent_radix_sort_downsweep.cuh design: - union TempStorage reuses same shared memory across load/rank/scatter phases - Translated: streaming parse state organized into 4 phases (DETECT/HEADER/PARAMS/CLOSE) matching the tool call parse lifecycle - Clear documentation of which state belongs to which parse phase - Reset clears all phases at once like union initialization on new tile	2026-08-08 07:02:22 +00:00
Claude	e0344b1730	fix(critical): disable thinking for tool_call requests — fixes d03_tool_call FAIL Root cause: When tool_choice=auto + tools present, the model enters <think>...</think> mode by default. On BI-V100 hardware, decode is slow enough that thinking consumes the entire max_tokens budget, and the model finishes (finish=stop) before ever emitting <tool_call> XML. Sub168 reference: d03 in 2.12s with tools=1, finish=tool_calls Our sub509: d03 in 49.04s with tools=0, finish=stop — FAIL Fix: Two-layer defense: 1. protocol.py normalize_messages: when tools active + tool_choice=auto and thinking not explicitly set, auto-set enable_thinking=False 2. qwen3coder_tool_parser.py adjust_request: same logic as defense-in-depth 3. baseline.muh synced with actual computility-run.yaml	2026-08-07 07:45:28 +00:00
dylanyunlon	ef6abf3dc7	[DEPLOY] Complete submission: baseline + all optimizations Adds ALL files needed for Dockerfile build: - qwen3_6_scripts/ (baseline patches + our optimizations) - vllm/ (full vllm package) - paged_attention_v2_pytorch.py (V2 with single-bmm optimization) - Dockerfile + computility-run.yaml Our optimizations vs baseline: 1. paged_attn.py: pre-gathered context KV (eliminates 194 gather calls), Triton try/fallback, V2 heuristic, threshold 32K→64K 2. paged_attention_v2_pytorch.py: fills NotImplementedError, single-bmm Phase 1 (195 launches → 3) 3. patch_enable_triton.py: HAS_TRITON=True with safety fallback 4. patch_triton_tuning.py: BLOCK=64, NUM_WARPS=4 for BI-V100 5. computility-run.yaml: gpu-memory-utilization 0.9→0.95, max-num-batched-tokens 8192→16384 This repo can now be submitted to dev.modelhub.org.cn as-is.	2026-07-30 16:06:20 +00:00

Author

SHA1

Message

Date

Claude

e86b7eacdd

refactor(tool_parser): CCCL agent_radix_sort_downsweep union TempStorage — phased state

Applies CCCL cub/agent/agent_radix_sort_downsweep.cuh design:
- union TempStorage reuses same shared memory across load/rank/scatter phases
- Translated: streaming parse state organized into 4 phases
  (DETECT/HEADER/PARAMS/CLOSE) matching the tool call parse lifecycle
- Clear documentation of which state belongs to which parse phase
- Reset clears all phases at once like union initialization on new tile

2026-08-08 07:02:22 +00:00

Claude

e0344b1730

fix(critical): disable thinking for tool_call requests — fixes d03_tool_call FAIL

Root cause: When tool_choice=auto + tools present, the model enters
<think>...</think> mode by default. On BI-V100 hardware, decode is slow
enough that thinking consumes the entire max_tokens budget, and the model
finishes (finish=stop) before ever emitting <tool_call> XML.

Sub168 reference: d03 in 2.12s with tools=1, finish=tool_calls
Our sub509: d03 in 49.04s with tools=0, finish=stop — FAIL

Fix: Two-layer defense:
1. protocol.py normalize_messages: when tools active + tool_choice=auto
   and thinking not explicitly set, auto-set enable_thinking=False
2. qwen3coder_tool_parser.py adjust_request: same logic as defense-in-depth
3. baseline.muh synced with actual computility-run.yaml

2026-08-07 07:45:28 +00:00

dylanyunlon

ef6abf3dc7

[DEPLOY] Complete submission: baseline + all optimizations

Adds ALL files needed for Dockerfile build:
  - qwen3_6_scripts/ (baseline patches + our optimizations)
  - vllm/ (full vllm package)
  - paged_attention_v2_pytorch.py (V2 with single-bmm optimization)
  - Dockerfile + computility-run.yaml

Our optimizations vs baseline:
  1. paged_attn.py: pre-gathered context KV (eliminates 194 gather calls),
     Triton try/fallback, V2 heuristic, threshold 32K→64K
  2. paged_attention_v2_pytorch.py: fills NotImplementedError,
     single-bmm Phase 1 (195 launches → 3)
  3. patch_enable_triton.py: HAS_TRITON=True with safety fallback
  4. patch_triton_tuning.py: BLOCK=64, NUM_WARPS=4 for BI-V100
  5. computility-run.yaml: gpu-memory-utilization 0.9→0.95,
     max-num-batched-tokens 8192→16384

This repo can now be submitted to dev.modelhub.org.cn as-is.

2026-07-30 16:06:20 +00:00

3 Commits