[Misc] Remove CP Redundant Variables after FIA operator enables for CANN 8.5 (#6013)
### What this PR does / why we need it?
PCP/DCP splits the kv-cache onto different cards. After introducing the
parameter cp-kv-cache-interleave-size, the first size tokens will be
cached at Card 0, and so on.
However, if there are too few tokens, some cards will not store the
key-value pairs, resulting in values of 0, corrupted values, and
precision issues. Currently, additional operations are introduced to
avoid this precision problem.
After we integrate FIA operator in mla_cp._forward_decode and CANN
updates to 8.5.0, we now can remove these additional operations.
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
passed all CI by CANN 8.5.0
- vLLM version: v0.13.0
- vLLM main:
2c24bc6996
Signed-off-by: dsxsteven <dsxsteven@sina.com>
Signed-off-by: dsxsteven <36877507+dsxsteven@users.noreply.github.com>
This commit is contained in:
@@ -137,7 +137,6 @@ class TestAscendMLADecodeMetadata(TestBase):
|
||||
seq_lens_list = [2, 3]
|
||||
attn_mask = None
|
||||
cp_seq_len = torch.tensor([2, 3])
|
||||
batch_seq_mask = torch.tensor([[1, 1, 0, 0], [1, 1, 1, 0]])
|
||||
|
||||
metadata = AscendMLADecodeMetadata(input_positions=input_positions,
|
||||
block_table=block_table,
|
||||
@@ -145,8 +144,7 @@ class TestAscendMLADecodeMetadata(TestBase):
|
||||
max_seq_lens=max_seq_lens,
|
||||
seq_lens_list=seq_lens_list,
|
||||
attn_mask=attn_mask,
|
||||
cp_seq_len=cp_seq_len,
|
||||
batch_seq_mask=batch_seq_mask)
|
||||
cp_seq_len=cp_seq_len)
|
||||
|
||||
self.assertIs(metadata.input_positions, input_positions)
|
||||
self.assertIs(metadata.block_table, block_table)
|
||||
@@ -155,7 +153,6 @@ class TestAscendMLADecodeMetadata(TestBase):
|
||||
self.assertEqual(metadata.seq_lens_list, seq_lens_list)
|
||||
self.assertIsNone(attn_mask)
|
||||
self.assertIs(metadata.cp_seq_len, cp_seq_len)
|
||||
self.assertIs(metadata.batch_seq_mask, batch_seq_mask)
|
||||
|
||||
|
||||
class TestAscendMLAMetadata(TestBase):
|
||||
|
||||
Reference in New Issue
Block a user