Definition
A GPU (graphics processing unit) is a computer chip originally designed to render video game graphics that has become the workhorse of artificial intelligence. Unlike a normal processor, which handles a few complex tasks at a time, a GPU performs thousands of simple calculations in parallel. Neural networks are made of exactly that kind of arithmetic, so GPUs train and run AI models tens or hundreds of times faster than ordinary chips.
Most companies never buy a GPU. They rent them indirectly, by paying for AI APIs or by renting GPU instances in the cloud when they run open-weight models themselves. GPU capacity, however, sits behind every AI cost you see: model prices, rate limits and latency all trace back to how much GPU time your requests consume. For a CFO, the GPU is the reason AI has a marginal cost, unlike traditional software.
Demand for AI hardware has made GPU supply a strategic issue for countries and companies, with Nvidia the dominant supplier and cloud providers and labs designing their own accelerator chips. Efficiency gains mean each generation delivers more work per euro. The misconception is that you need your own GPUs to do serious AI. Unless you have strict data residency needs or very high steady volumes, renting through APIs is cheaper and simpler.
In practice
A healthcare software company with strict data residency requirements rents a small cluster of GPUs in an EU data centre to run an open-weight model on patient records, instead of sending data to a public API.
Why it matters
GPUs are the physical constraint behind AI pricing and availability. Understanding this explains why AI costs scale with usage and why owning hardware is rarely the right call for a mid-size company.
Frequently asked questions
- Why are GPUs used for AI instead of normal processors?
- Because AI models consist of huge numbers of simple, repetitive calculations that can run at the same time. GPUs have thousands of small cores built for exactly that, while a normal CPU has a few powerful cores optimised for sequential tasks. The result is a massive speed and energy advantage for AI.
- Should my company buy GPUs for AI?
- Usually not. Buying hardware makes sense only with very high, predictable volumes or hard data residency rules. Most companies get better economics and flexibility from AI APIs or from renting GPU capacity in the cloud by the hour when they need it.