vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2x A5000/3090) — Qwen3.6-35B-A3B FP8 ~244 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B AWQ, 256K ctx. 321 patches: TurboQuant k8v4 KV, MTP/DFlash spec-decode, FULL cudagraph, hybrid GDN. vLLM pin dev424 + SNDR Control Center GUI.
cuda nvidia moe gdn ampere structured-output long-context fp8 vllm llm-inference qwen speculative-decoding tool-calling qwen3 rtx-3090 runtime-patches dflash turboquant ampere-sm86 rtx-a5000
-
Updated
Jun 27, 2026 - Python