Description
Trillion-parameter models normally need a rack of GPUs, so most people just call an API. kimi-k3-in-c is a portable C99 inference engine that runs the 2.78-trillion-parameter Kimi K3 on a single CPU with 8.24GB peak RAM, with no BLAS, framework or GPU.
The whole engine is 176KB and streams the checkpoint from disk. More memory only buys speed, so an ordinary 8GB laptop gets the same answer, just slower per token.
Tiny memory:8.24GB peak for a trillion-scale MoE.
No dependencies:Pure C99, no GPU.
Disk streaming:1.56TB checkpoint read on demand.
Portable:Same output on any machine.
The whole engine is 176KB and streams the checkpoint from disk. More memory only buys speed, so an ordinary 8GB laptop gets the same answer, just slower per token.
Features
Tiny memory:8.24GB peak for a trillion-scale MoE.
No dependencies:Pure C99, no GPU.
Disk streaming:1.56TB checkpoint read on demand.
Portable:Same output on any machine.
