EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[BugFix][0.18.0][310p] fix post-sampling not working in graph mode on 310p (#8077)

Browse Source 
### What this PR does / why we need it?

Enabling temperature in post-processing on 310P devices can cause the
service to stall and eventually hang. We first traced the issue to a
timeout where the temperature-related `div` operator was waiting for
results from a sub-stream. After investigating the preceding operators,
we finally identified the root cause as the `q.exponential_()` operator,
which is not well supported on 310P and triggers an internal issue in
the `add` kernel.

### Does this PR introduce _any_ user-facing change?
NA

### How was this patch tested?
This patch was thoroughly tested locally（accuracy-dataset test and
stress test）. It is not easy to design a proper unit test for this case,
and I appreciate your understanding.

Signed-off-by: Tflowers-0129 <2906339855@qq.com>
This commit is contained in:
Shaoxu Cheng
2026-04-09 16:31:38 +08:00
committed by GitHub
parent 0d1424d81a
commit 82e17f693a
3 changed files with 74 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
vllm_ascend/_310p/sample/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
from vllm_ascend._310p.sample.sampler import AscendSampler310
__all__ = ["AscendSampler310"]