Logo
Explore Help
Register Sign In
dylanyunlong/project_6
1
0
Fork 0
You've already forked project_6
Code Issues Pull Requests Actions Projects Releases Wiki Activity
Files
a2a5dd8f00fef4fb54f794f73640fcfdf3dda2e8
project_6/vllm/model_executor/__init__.py

14 lines
466 B
Python
Raw Normal View History

[DEPLOY] Complete submission: baseline + all optimizations Adds ALL files needed for Dockerfile build: - qwen3_6_scripts/ (baseline patches + our optimizations) - vllm/ (full vllm package) - paged_attention_v2_pytorch.py (V2 with single-bmm optimization) - Dockerfile + computility-run.yaml Our optimizations vs baseline: 1. paged_attn.py: pre-gathered context KV (eliminates 194 gather calls), Triton try/fallback, V2 heuristic, threshold 32K→64K 2. paged_attention_v2_pytorch.py: fills NotImplementedError, single-bmm Phase 1 (195 launches → 3) 3. patch_enable_triton.py: HAS_TRITON=True with safety fallback 4. patch_triton_tuning.py: BLOCK=64, NUM_WARPS=4 for BI-V100 5. computility-run.yaml: gpu-memory-utilization 0.9→0.95, max-num-batched-tokens 8192→16384 This repo can now be submitted to dev.modelhub.org.cn as-is.
2026-07-30 16:06:20 +00:00
from vllm.model_executor.parameter import (BasevLLMParameter,
PackedvLLMParameter)
from vllm.model_executor.sampling_metadata import (SamplingMetadata,
SamplingMetadataCache)
from vllm.model_executor.utils import set_random_seed
__all__ = [
"SamplingMetadata",
"SamplingMetadataCache",
"set_random_seed",
"BasevLLMParameter",
"PackedvLLMParameter",
]
Reference in New Issue Copy Permalink
Powered by Gitea Version: 1.24.3 Page: 68ms Template: 1ms
English
Bahasa Indonesia Deutsch English Español Français Gaeilge Italiano Latviešu Magyar nyelv Nederlands Polski Português de Portugal Português do Brasil Suomi Svenska Türkçe Čeština Ελληνικά Български Русский Українська فارسی മലയാളം 日本語 简体中文 繁體中文(台灣) 繁體中文(香港) 한국어
Licenses API