Oli Wilkins
Senior Inference Engineer
I build LLM inference platforms on Kubernetes.
vLLM, NVIDIA Dynamo, and GPUs that should be working harder than they are.
Experience
Likezero
Sep 2024 — PresentSenior AI Engineer
I built and own the LLM inference platform on AWS and Kubernetes: vLLM, NVIDIA Dynamo, disaggregated prefill/decode, KV-cache-aware routing, and GPU autoscaling. My async Python infrastructure has handled hundreds of millions of requests.
Applied Data Science Partners
Aug 2023 — Sep 2024Senior AI Engineer
I delivered three LLM projects in defence and education, and led generative AI and software engineering work in cyber defence.
Liveminds and Maeving
Jul 2023 — Sep 2024AI & Data Engineer Consultant
I built a production RAG summarisation product for Liveminds and an analytics platform for Maeving.
Projects
h8s
181 GitHub starsHomernetes is a Talos OS based K8s cluster for my homelab.
argo-workflowsargocdcert-managerciliumgrafanahomelabkubernetesprometheustalos-linux.dotfiles
42 GitHub starsMy personal Nix dotfiles for Linux and MacOS.
configdotfilesneovimnixshellzshinferlace
0 GitHub starsOne model inference engine written by hand for RTX 4080 Lovelace GPU architecture.
cudagpuinferencellmpythonpytorch
Services
A few things I host on my Kubernetes homelab.