Definition
Latency is the time it takes for a request to travel to a system and for the response to come back. It is measured in milliseconds and it is the technical name for waiting. Some of it is physics: a signal crossing an ocean takes tens of milliseconds regardless of how good the software is. The rest is engineering: how many steps a request passes through, how far the data centre is from the user and how efficiently each step does its work.
In a company, latency decides how a product feels. A page that loads in 300 milliseconds feels instant, one that takes 3 seconds feels broken, and users abandon before they consciously notice why. It shows up in search rankings, checkout conversion, support tickets and in how often your own team uses the internal tools you paid for. Engineers reduce latency with CDNs, caching, faster databases, fewer network hops and by placing servers close to users.
Latency is especially relevant for AI products in 2026. A large language model may take several seconds to produce a full answer, so teams stream the response word by word, use smaller faster models for simple tasks and reserve the large reasoning models for problems that justify the wait. The misconception is that latency is a purely technical concern. It is a product decision: for a chat interface, a voice agent or a trading system, latency is the feature.
In practice
A voice-based customer service agent that took three seconds to reply felt unusable, with callers talking over it. Rebuilding the pipeline to respond in under a second, using a faster model for the first words, changed the product from a demo into something customers accepted.
Why it matters
Users experience latency as trust. If your product feels slow, they assume it is also unreliable. Ask for the numbers, in milliseconds, for the five actions your customers perform most, and set targets.
Frequently asked questions
- What is a good latency for a website?
- Under about 200 milliseconds for the server to start responding feels fast, and the whole page should be usable within 2 to 3 seconds on mobile. Google's Core Web Vitals recommend a Largest Contentful Paint under 2.5 seconds. Anything slower costs conversion and search visibility.
- What is the difference between latency and bandwidth?
- Latency is delay, how long a single request takes to get a response. Bandwidth is capacity, how much data can move per second. A connection can have high bandwidth and still feel slow if latency is high, which is why distance to the server matters even on a fast line.