Track A
Independent R&D
Making computation fast on real chips.
AI models · GPU code · small devices · scientific simulation. Measured on hardware I own, with numbers you can verify.
Scope
Research tracks
My R&D asks one question: how does computation really run on real chips? I work on it through AI inference, GPU programming, small devices and scientific computing.
Latest result
vLLM on Apple Silicon
vLLM vs plain Transformers on an Apple Silicon Mac: same model, same questions, honest numbers.
Single request · tok/s
192 vs 182
vLLM (123–222) · Transformers (181–188)
10 one by one · tok/s
203 vs 181
vLLM (197–213) · Transformers (180–186)
100 together · tok/s
4,133
vLLM only (2,647–5,571). Transformers not run at this batch.
Median of 3 runs. Machine: MacBook Pro, Apple M5 Pro, 24 GB. Model: SmolLM2-360M-Instruct (bfloat16). At 100 requests together, median throughput was about 23× the sequential Transformers run.
Caveats:small sample (three runs, one Mac, one small model); results vary between runs (the 100-request case ranged about 2×); answer lengths differ between engines — compare tokens/sec and requests/sec, not wall time alone.
Field notes
Latest notebook
Why repeated identical prompts look faster: prefix caching
Re-sending the same prompt can look much faster because engines reuse computed KV for a shared prefix. That is a real effect — and a trap if you ignore it in a benchmark.
Why Docker can't use the Mac GPU
On Apple Silicon, Docker runs Linux guests. Those guests do not get Metal, so GPU-backed inference has to run on the host.
Running vLLM on a Mac
I got vLLM running on Apple Silicon with the Metal plugin and measured it against plain Transformers on the same model and questions.
Contact