SGLang for DeepSeek on Ampere
author of the fork, 2026
A fork of the SGLang inference engine that runs DeepSeek V4-Flash on older NVIDIA Ampere cards, past the requirements of more recent hardware.
SGLang serves large language models, and DeepSeek V4-Flash uses a numeric format that only recent NVIDIA cards support in hardware. This fork runs the model on the older Ampere generation, with software fallbacks in the attention kernels, a conversion script that requantizes the model’s weights into compatible formats, a matching loader, a container image, and correctness tests.