Why are GPUs so important for modern AI?

GPUs, or graphics processing units, were originally designed to render complex 3D graphics for video games, a job that requires performing huge numbers of simple mathematical calculations simultaneously across millions of pixels. It turns out that training AI models involves an extremely similar kind of workload: massive amounts of matrix multiplication done in parallel, so researchers realized GPUs could train neural networks far faster than traditional processors built for handling tasks one after another.

Storing files is the job of hard drives or solid-state drives, not GPUs, which are specialized for calculation rather than data storage. Keeping computers cool is handled by fans and heat sinks, and while GPUs do generate significant heat during heavy use, cooling is not their function.

This unexpected crossover from gaming hardware to AI has been so consequential that demand from AI companies has, at times, driven shortages and price spikes in GPUs that gamers themselves also rely on for their consoles and PCs.