What the heck is an AI token anyway?

Nvidia CEO Jensen Huang’s remarks at the company’s GTC conference had a recurring theme: AI tokens.

In a conversation with analysts on Tuesday, Huang framed future computers as “manufacturing equipment” that will produce tokens. He said that tokens will become a core line item in corporate budgets, like laptops or software subscriptions.

With all this talk about tokens, what exactly are they?

Tokens are units of text — a word or a part of a word — that determine how AI work is measured and priced. A short word could be a single token, while longer words can be split into several. A rule of thumb is that one token is about four characters.

Large language models like OpenAI’s ChatGPT or Anthropic’s Claude track how many tokens are processed when a user inputs text, and how many are generated in response — and AI giants bill companies accordingly. The more text you work with, the more tokens it takes for AI models to process. Unlike existing software costs, which are…