Why repeated identical prompts look faster: prefix caching
Re-sending the same prompt can look much faster because engines reuse computed KV for a shared prefix. That is a real effect — and a trap if you ignore it in a benchmark.
Lab notebook
What I ran, measured, and found.
Newest first. Each entry is one experiment on my own machines.
Why Docker can't use the Mac GPU
On Apple Silicon, Docker runs Linux guests. Those guests do not get Metal, so GPU-backed inference has to run on the host.
Running vLLM on a Mac
I got vLLM running on Apple Silicon with the Metal plugin and measured it against plain Transformers on the same model and questions.