Matrix A - minicpm5_2b - run 37875011623 │ c1@126k │ c8@14k │ c64@1.6k │ c256@0.8k engine │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── vllm │ 9456/5832 132/83.4 132/84 13644/22125 7.6/12.0 44/27 │ 5175/4438 79.7/68.1 799/695 3018/3527 12.5/14.7 466/398 │ 872/854 78.8/67.9 5493/4692 2605/2652 12.7/14.7 4076/3627 │ 220/217 29.5/26.8 8598/7978 4502/4534 33.9/37.3 5974/5621 sglang │ 9601/5832 132/84.5 132/84 13438/22123 7.5/11.8 44/27 │ 5862/4561 82.6/69.4 819/711 2732/3491 12.1/14.4 492/402 │ 949/884 76.7/67.2 5364/4650 2191/2389 13.0/14.9 4044/3605 │ 247/241 26.4/23.6 7506/6843 4054/4187 37.9/42.4 5098/4695 cells: tkv/bf16 · TTFT,TPOT=ms median · prefill=prompt tok / TTFT · decode/stream=1000/TPOT · decode total=all streams over their decode spans · agg=all streams @ 900 tok, prefill INCLUDED · best / worst>20% / worst-close / plain mid