Matrix A - minicpm5_2b - run 37874964979 │ c1@126k │ c8@14k │ c64@1.6k │ c256@0.8k engine │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s │ prefill tok/s decode/stream decode total TTFT TPOT agg@900 tok/s ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── arbi │ 9744/5776 126/80.6 126/81 13242/22337 7.9/12.4 44/27 │ 5640/4828 82.9/61.4 804/626 2832/2682 12.1/16.3 487/374 │ 905/887 74.7/55.8 5165/3957 2408/2361 13.4/17.9 3918/3114 │ —/— —/— —/— —/— —/— —/— sglang │ 9598/5857 132/84.5 132/84 13443/22030 7.5/11.8 44/28 │ 5867/4566 82.6/69.5 819/711 2728/3487 12.1/14.4 492/402 │ 945/885 76.7/67.2 5360/4652 2197/2390 13.0/14.9 4038/3607 │ 247/242 26.3/23.6 7498/6845 4058/4170 38.0/42.4 5093/4697 cells: tkv/bf16 · TTFT,TPOT=ms median · prefill=prompt tok / TTFT · decode/stream=1000/TPOT · decode total=all streams over their decode spans · agg=all streams @ 900 tok, prefill INCLUDED · best / worst>20% / worst-close / plain mid