saltyfish66 e41549c3d6 fix: fix illegal cuda memory access at fused_moe_kernel (#4727) ...

Co-authored-by: yuethe <yuethe@tencent.com>

2025-04-03 00:07:32 -07:00

..

configs

Clean up import vllm in quantization/__init__.py (#4834)

2025-03-28 10:34:10 -07:00

connector

feat(remote_model): support variable remote backend for model loader (#3964)

2025-03-14 00:40:44 -07:00

constrained

Fix retract for page size > 1 (#4914)

2025-03-30 02:57:15 -07:00

disaggregation

[PD] Remove invalid parameter (#4721)

2025-03-24 13:14:16 -07:00

distributed

Fix error due to CustomAllreduce setup failure (#4815)

2025-03-27 18:52:10 -07:00

entrypoints

[VLM RLHF] Take Image input for verl vlm rollout (#4915)

2025-04-01 20:03:17 -07:00

layers

fix: fix illegal cuda memory access at fused_moe_kernel (#4727)

2025-04-03 00:07:32 -07:00

lora

[Feature] add multi-rank support for Lora (#4492)

2025-03-28 09:38:44 -07:00

managers

Update tokenizer_manager.py (#5008)

2025-04-02 13:56:19 -07:00

mem_cache

bug fix for hicache host eviction (#4989)

2025-04-02 00:33:50 -07:00

metrics

Fix retract for page size > 1 (#4914)

2025-03-30 02:57:15 -07:00

model_executor

[Feature] Support DeepEP Low Latency (#4767)

2025-04-01 09:23:25 -07:00

model_loader

[Misc] Fix issues reported by torchfix (#4837)

2025-03-27 20:10:32 -07:00

models

[Feature] Support DeepEP Low Latency (#4767)

2025-04-01 09:23:25 -07:00

openai_api

refactor: multimodal data (#4754)

2025-03-31 09:57:51 -07:00

platforms

[feat] interface for platforms abstraction (#4928)

2025-03-31 00:04:21 -07:00

sampling

[Fix] use torch.cat instead of torch.concat to prevent entering the Autograd backends. (#4466)

2025-03-16 00:02:47 -07:00

speculative

Add Eagle Speculative Decoding to FA3 Backend (#4951)

2025-04-02 13:09:02 -07:00

_custom_ops.py

use sgl custom all reduce (#4441)

2025-03-18 00:46:41 -07:00

aio_rwlock.py

A better aio rwlock that guarantees the order (#2547)

2024-12-22 15:44:32 -08:00

code_completion_parser.py

Feature: support code completion (#3612)

2025-03-16 18:26:19 -07:00

conversation.py

model: Minicpmo (#3023)

2025-03-24 20:08:40 -07:00

custom_op.py

sgl scaled_fp8_quant support output padding (#4861)

2025-04-02 23:53:57 +08:00

function_call_parser.py

Remove empty tool function name (#4704)

2025-03-27 22:23:30 -07:00

hf_transformers_utils.py

[Bug Fix] Add partial rotary factor support for Phi-4 and upgrade to transformers v4.50.0 (#3984)

2025-03-22 14:27:39 -07:00

mm_utils.py

docs: fix module docstrings and copyright headers (#2077)

2024-11-22 22:16:53 +08:00

model_parallel.py

Make torch TP composable with torchao (#2436)

2024-12-11 04:21:42 -08:00

patch_torch.py

Patch PyTorch's bug that cross-process tensor transfer will lead to wrong device (#4565)

2025-03-27 00:22:33 -07:00

reasoning_parser.py

Reasoning parser (#4000)

2025-03-03 21:16:36 -08:00

server_args.py

[Feature] Support DeepEP Low Latency (#4767)

2025-04-01 09:23:25 -07:00

server.py

Misc fixes for eagle (flush_cache, CPU overhead) (#3014)

2025-01-20 20:27:38 -08:00

torch_memory_saver_adapter.py

Warn users when release_memory_occupation is called without memory saver enabled (#4566)

2025-03-26 00:18:14 -07:00

utils.py

[VLM RLHF] Take Image input for verl vlm rollout (#4915)

2025-04-01 20:03:17 -07:00

warmup.py

Support penalty in overlap mode; return logprob with chunked prefill; improve benchmark scripts (#3988)

2025-03-03 00:12:04 -08:00