GPUs were designed to draw video-game graphics, which means doing the same maths on millions of pixels at once. Neural networks need the same kind of massively parallel maths, so GPUs became the engine of modern AI. Their supply and price shape the whole AI industry.

How it works

A CPU has a small number of powerful cores built to handle varied tasks one after another. A GPU has thousands of simpler cores that perform the same operation on lots of data simultaneously. AI models are mostly giant grids of numbers, and both training and inference come down to multiplying large matrices, exactly the work GPUs excel at. Modern data-centre GPUs add specialised tensor cores for these matrix operations and support lower-precision number formats, such as 16-bit or 8-bit, which trade a little precision for big speed gains. Memory is just as important as raw speed. A model's weights must fit in the GPU's video memory (VRAM); at 16-bit precision each parameter takes two bytes, so a 7-billion-parameter model needs roughly 14 GB for its weights alone. Large data-centre GPUs use high-bandwidth memory and fast links to work together, and the biggest models are trained on clusters of thousands of them. Nvidia dominates AI GPUs, helped by its CUDA software platform, with alternatives from AMD and custom chips such as Google's TPUs.

Why it matters

GPU cost is a large part of why AI tools charge what they do, use credits or limit usage. For creators and businesses, GPUs matter in two ways. First, if you run AI locally, such as image generation or an open-weight model, your GPU's VRAM decides what you can run and how fast. Second, cloud GPU rental lets you render, train or batch-process without buying hardware. Understanding GPUs also makes AI industry news about chip shortages and data centres easier to follow.

Where you will see it

  • Training and fine-tuning AI models in data centres.
  • Running image generators or open-weight language models on a local PC.
  • Speeding up video editing, effects and rendering.
  • Renting cloud GPUs for batch jobs such as transcription or upscaling.

Limits and common misconceptions

  • High-end AI GPUs are expensive and power-hungry, and supply has often been tight.
  • VRAM is often the real bottleneck; a fast GPU with too little memory cannot load large models.
  • Not essential for using AI apps; cloud tools run on the provider's GPUs, not yours.

How it differs from related terms

  • GPU vs Inference Inference is the act of running a trained model to get an answer; a GPU is the hardware that commonly performs it.
  • GPU vs Parameters Parameters are the numbers inside a model; the GPU stores them in memory and does the maths on them, so more parameters need more GPU memory.

Frequently asked questions

Why are GPUs used for AI instead of CPUs?

AI models rely on huge numbers of simple multiplications that can run at the same time. A GPU's thousands of cores handle this parallel work far faster than a CPU's few powerful cores. CPUs still run AI, but much more slowly for large models.

How much VRAM do I need to run AI locally?

It depends on the model and settings. Image generators often run on GPUs with around 8 to 12 GB, with more allowing larger images and faster batches. For language models, a rough guide is two bytes per parameter at 16-bit precision, less with quantised versions.

What is the difference between a GPU and a TPU?

A GPU is a general parallel processor originally built for graphics and adapted for AI. A TPU, or tensor processing unit, is a chip Google designed specifically for machine learning and offers mainly through its cloud. Both accelerate the matrix maths at the heart of AI.

Related terms

All AI terms