Despite declining token costs, businesses continue to see their AI tab climb.
The price of a token — the chunks of text that large language models (LLMs) use to generate responses to prompts and that AI companies use to bill their customers — has collapsed, noted Jeff Barrington, managing director of Windsor Drake , an investment banking and M&A advisory firm in Toronto.
"They're down about 98% since early 2024, yet enterprise AI bills keep rising," he told TechNewsWorld. "Cheaper compute just gets consumed faster, and 73% of enterprises blew past their original AI budgets last year."
"Uber handed 5,000 engineers a coding assistant in December and burned its entire annual AI budget by April," he added.
One reason for the growing consumption: Companies are moving from pilots to production, and from chatbots to agents. "An agent does not answer one prompt," Barrington explained. "It fans that prompt out into planning, tool calls, retrieval and verification, which can turn a single instruction into hundreds of calls and five to 30 times the tokens."
"Feeding in messy, unstructured data makes it worse, because the model burns tokens just parsing the mess," he said.
"Clean, structured documents cut the bill in reverse — less to interpret, fewer tokens," he continued. "Inference is now about 85% of enterprise AI budgets, so that is where the money goes."
AI flipped software from a fixed-cost license to a metered utility, asserted Jon Knisley, head of AI enablement and value at Abbyy , a global intelligent automation company.
"Every prompt, retrieval, and loop is on the meter, so spend rises with adoption," he told TechNewsWorld. "But the real jump is architecture."
Moving from a chatbot to an agent raises token consumption per task by one to two orders of magnitude," he said.
He added that a large portion of a business's AI bill is waste. "In agentic workloads, actual output is often only 5% to 15% of the tokens consumed," he explained. "The rest is context overhead."
"PDFs are a perfect example," he noted. "They were designed for rendering, not understanding, so every PDF that enters a pipeline forces the model to burn tokens reconstructing layout before it can extract the facts. You're paying a reasoning model to do a formatting job."
"Token consumption is also accelerating because organizations are expanding the number of workflows where AI is delivering value. As AI becomes embedded across more business functions and everyday tasks, token usage naturally increases because more interactions flow through these models," explained Edward Rothschild, CTO of Adronite , an AI coding and software intelligence company in Seattle.
He added that newer generations of AI models often consume more tokens than their predecessors, even when completing tasks that older models were already capable of handling.
"In many cases, this is because they're designed to reason more deeply, taking additional steps, evaluating more context, and performing more internal processing before producing an answer," he told TechNewsWorld. That extra reasoning can improve quality, but it also increases token consumption.
The big message here is that the consumption of AI tokens is exploding as companies move beyond simple chatbots into copilots — autonomous agents that repeatedly read documents, retrieve data, call tools, and communicate with other systems, observed Mark N. Vena, president and principal analyst at SmartTech Research , a technology advisory firm…