[Feature] add multi-rank support for Lora (#4492)

Co-authored-by: rudy152 <czh1137892874@gmail.com>
This commit is contained in:
chaobo jia
2025-03-29 00:38:44 +08:00
committed by GitHub
parent 6dea5c96bf
commit ef9a378a20
16 changed files with 292 additions and 97 deletions

View File

@@ -25,6 +25,12 @@ class LoRABatchInfo:
# The index of lora adapter used by each sequence, in shape (bs,)
weight_indices: torch.Tensor
# ranks of each lora adapter, in shape (lora_num,)
lora_ranks: torch.Tensor
# scaling of each lora adapter, in shape (lora_num,)
scalings: torch.Tensor
class LoRAType(Enum):
LORA_A = 0