kiln
A WebGPU inference engine running Whisper in-browser
- Role
- Personal project
- Year
- 2026
- Built with
- TypeScript · WebGPU · WGSL
Overview
kiln is a from-scratch WebGPU inference engine for transformer models, built around what WebGPU actually is rather than ported from a native runtime. It runs OpenAI's Whisper speech-to-text entirely in the browser on hand-written WGSL kernels, with no ONNX, no WASM runtime and no server.
Why
Running models in the browser costs nothing in server GPUs, and the data never leaves the device. Most of that work today goes through general-purpose graph compilers. kiln goes the other way: a small set of kernels designed around WebGPU's limits. There's no f64, f16 is optional, workgroup memory is small, and every dispatch carries overhead that punishes lots of tiny kernels.
Kernels
Matmul is tile-parametric and tuned by sweeping. Decoding uses a dedicated GEMV with int4 weights. Attention is a fused FlashAttention-style kernel, and there's a key-parallel decode kernel. QKV, GELU and residual-into-LayerNorm are fused too, which cut a decode step from 96 dispatches to 52.
Measure, don't guess
Every kernel has a plain-JS reference and a test that compares the two on a real GPU, about 270 tests in all. Kernels are timed with GPU timestamp queries. On the same GPU and browser, the README reports whisper-tiny running 5.3× faster than Transformers.js with identical accuracy at f32, and 6.5× faster at 4-bit.
Tech Stack
- TypeScript
- WebGPU
- WGSL
- Vitest
- Playwright