vLLM for DeepSeek on consumer GPUs
author of the fork, 2026
A modified version of vLLM that runs the DeepSeek V4 model on consumer and older NVIDIA cards.
vLLM serves large language models. This modified version runs DeepSeek V4 on consumer and older NVIDIA cards, adding a converter that requantizes the checkpoint into formats those cards support, changes to the attention path, a mapping from standard reasoning-effort settings onto the model’s three levels, and container images wired for Thunderbolt networking.