EngineX/xc-llm-ascend
3
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

[Bugfix] fix env variable in dbo (#1284)

Browse Source 
### What this PR does / why we need it?
Fix env variable in dbo to enable dbo in DeepSeek-V3 model. Besides, we
have fixed an known issue in deepseek-dbo.


### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
This patch can be tested with newly added e2e tests:
[tests/multicard/test_offline_inference_distributed.py](https://github.com/vllm-project/vllm-ascend/pull/1285/files#diff-7cd2e6b1bda6b8ad1bedb3276971fe7064aeae4dc0efd41c301c4ede2158c57e).
It can be verified with pytest.

---------

Signed-off-by: zhuohuan <zxdu1997@gmail.com>
This commit is contained in:
zxdukki
2025-06-23 09:07:57 +08:00
committed by GitHub
parent 21fb68a03a
commit f04c6763d8
4 changed files with 41 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
vllm_ascend/models/deepseek_dbo.py
View File

@@ -641,7 +641,7 @@ class CustomDeepseekDBODecoderLayer(DeepseekV2DecoderLayer):
if self.mlp.tp_size > 1:
num_token, _ = hidden_states[i].shape
padded_num_tokens = (self.mlp.tp_size - num_token %
padded_num_tokens = (self.mlp.tp_size - num_tokens[i] %
self.mlp.tp_size) % self.mlp.tp_size
if padded_num_tokens > 0:
hidden_states[i] = nn.functional.pad(
@@ -851,7 +851,8 @@ class CustomDeepseekDBOModel(nn.Module):
if VLLM_ASCEND_ENABLE_DBO and self.can_run_ms()
else self.end_layer - self.start_layer)
for i in range(self.start_layer, self.start_layer + num_normal_layers):
moe_start_layer = self.start_layer + num_normal_layers
for i in range(self.start_layer, min(moe_start_layer, self.end_layer)):
layer = self.layers[i]
hidden_states, residual = layer(
positions, hidden_states, residual,
@@ -859,8 +860,7 @@ class CustomDeepseekDBOModel(nn.Module):
self.start_layer] if kv_caches is not None else None,
attn_metadata)
moe_start_layer = self.start_layer + num_normal_layers
if moe_start_layer != self.end_layer:
if moe_start_layer < self.end_layer:
# if we enable multistream/dbo, process sparse layers here
hidden_states, residual = self._forward_ms_layers(
positions=positions,