project_6

dylanyunlong/project_6

Fork 0

Commit Graph

Author	SHA1	Message	Date
dylanyunlon	32fd4299b3	[test+engine] 18→21 test cases + CCCL-informed improvements verify_functional.py: - TC-19 Idempotency: seed=42 temp=0 two requests must be identical (from CCCL catch2_test_device_reduce_deterministic.cu RFA pattern) - TC-20 Top-p boundary: top_p=1.0 and 0.01 edge cases (from CCCL catch2_test_device_topk_keys.cu k=1/k=N boundaries) - TC-21 Frequency penalty: freq_penalty=1.5 + presence_penalty=0.5 (from CCCL tuning_histogram.cuh privatized bin counting) model_runner.py: - Added CCCL cuda::experimental::graph_memory_resource design notes on CUDA Graph capture batch size optimization for BI-V100 CCCL sources read as input this session: - catch2_test_device_segmented_reduce_custom_policy_hub.cu (policy injection) - thrust/detail/random_bijection.h (Feistel cipher for sampling) - cudax/experimental/graph.cuh (CUDA Graph memory pools) - catch2_test_device_reduce_deterministic.cu (RFA determinism)	2026-08-06 06:33:18 +00:00
muh-engine	b0d597363a	[BUGFIX] qwen3_6_scripts/model_runner.py: fix max_decode_seq_len (deployment version) CRITICAL: patch_ops.sh deploys qwen3_6_scripts/ files, NOT vllm/ files. Previous bugfix only fixed vllm/worker/model_runner.py but the DEPLOYED version (qwen3_6_scripts/model_runner.py) still had the bug. Fix: max_decode_seq_len=max_encoder_seq_len → max_decode_seq_len=max_decode_seq_len This ensures CUDA graph capture correctly checks actual decode sequence length, not the encoder length (which is 0 for decoder-only Qwen3.6). Discovery from reading CCCL adjacent_difference custom_policy_hub test: the test showed that custom policy hubs OVERRIDE defaults. Our project has the same pattern: qwen3_6_scripts/ overrides vllm/ via patch_ops.sh. Therefore ALL fixes must go to qwen3_6_scripts/ to survive deployment. CCCL file: cub/test/catch2_test_device_adjacent_difference_custom_policy_hub.cu	2026-08-06 01:41:49 +00:00
dylanyunlon	327f9fbf40	[ARCH] Eliminate AST patch scripts — full file replacements only Deleted approach: patch_model_runner.py, patch_xformers_sdpa_seq.py did blind string replacement on base image files without reading full context. New approach: read complete base source files from vllm/, apply fixes with full context understanding, output complete modified files to qwen3_6_scripts/. Files now replaced as complete copies (not patched): - model_runner.py (1932 lines): prefix_cache_hit=False for Case 1 - xformers.py (821+80 lines): _run_sdpa_fallback + head_size>128 dispatch - arg_utils.py (1143 lines): disable auto chunked-prefill for 32K+ - logits_processor.py (157 lines): seq_groups=None guard patch_ops.sh rewritten: all python3 ./patch_*.py calls replaced with cp. Remaining python3 calls: patch_transformers_qwen3_5.py, patch_vllm_qwen3_5.py, patch_vllm_tool_parser.py — these register new model/parser classes in __init__.py files, which is additive (not modification of existing code).	2026-08-05 08:24:43 +00:00

Author

SHA1

Message

Date

dylanyunlon

32fd4299b3

[test+engine] 18→21 test cases + CCCL-informed improvements

verify_functional.py:
- TC-19 Idempotency: seed=42 temp=0 two requests must be identical
  (from CCCL catch2_test_device_reduce_deterministic.cu RFA pattern)
- TC-20 Top-p boundary: top_p=1.0 and 0.01 edge cases
  (from CCCL catch2_test_device_topk_keys.cu k=1/k=N boundaries)
- TC-21 Frequency penalty: freq_penalty=1.5 + presence_penalty=0.5
  (from CCCL tuning_histogram.cuh privatized bin counting)

model_runner.py:
- Added CCCL cuda::experimental::graph_memory_resource design notes
  on CUDA Graph capture batch size optimization for BI-V100

CCCL sources read as input this session:
- catch2_test_device_segmented_reduce_custom_policy_hub.cu (policy injection)
- thrust/detail/random_bijection.h (Feistel cipher for sampling)
- cudax/experimental/graph.cuh (CUDA Graph memory pools)
- catch2_test_device_reduce_deterministic.cu (RFA determinism)

2026-08-06 06:33:18 +00:00

muh-engine

b0d597363a

[BUGFIX] qwen3_6_scripts/model_runner.py: fix max_decode_seq_len (deployment version)

CRITICAL: patch_ops.sh deploys qwen3_6_scripts/ files, NOT vllm/ files.
Previous bugfix only fixed vllm/worker/model_runner.py but the DEPLOYED
version (qwen3_6_scripts/model_runner.py) still had the bug.

Fix: max_decode_seq_len=max_encoder_seq_len → max_decode_seq_len=max_decode_seq_len

This ensures CUDA graph capture correctly checks actual decode sequence
length, not the encoder length (which is 0 for decoder-only Qwen3.6).

Discovery from reading CCCL adjacent_difference custom_policy_hub test:
the test showed that custom policy hubs OVERRIDE defaults. Our project
has the same pattern: qwen3_6_scripts/ overrides vllm/ via patch_ops.sh.
Therefore ALL fixes must go to qwen3_6_scripts/ to survive deployment.

CCCL file: cub/test/catch2_test_device_adjacent_difference_custom_policy_hub.cu

2026-08-06 01:41:49 +00:00

dylanyunlon

327f9fbf40

[ARCH] Eliminate AST patch scripts — full file replacements only

Deleted approach: patch_model_runner.py, patch_xformers_sdpa_seq.py did
blind string replacement on base image files without reading full context.

New approach: read complete base source files from vllm/, apply fixes with
full context understanding, output complete modified files to qwen3_6_scripts/.

Files now replaced as complete copies (not patched):
  - model_runner.py (1932 lines): prefix_cache_hit=False for Case 1
  - xformers.py (821+80 lines): _run_sdpa_fallback + head_size>128 dispatch
  - arg_utils.py (1143 lines): disable auto chunked-prefill for 32K+
  - logits_processor.py (157 lines): seq_groups=None guard

patch_ops.sh rewritten: all python3 ./patch_*.py calls replaced with cp.
Remaining python3 calls: patch_transformers_qwen3_5.py, patch_vllm_qwen3_5.py,
patch_vllm_tool_parser.py — these register new model/parser classes in
__init__.py files, which is additive (not modification of existing code).

2026-08-05 08:24:43 +00:00

3 Commits