Jensen Huang has a test to see if it’s worth keeping an engineer, and it comes with a token budget. Speaking on the All-In Podcast at the closing ceremony of GTC 2026, Nvidia’s chief executive said “I would be very concerned” if a $500,000 engineer’s annual AI token consumption was less than half of his salary. He confirmed that NVIDIA is working toward an annual token charge of $2 billion for its engineering division.
He was describing trade-offs that most companies are already making in less-glamorous ways. Money that once paid people is increasingly being paid in tokens. The four hyperscalers will collectively spend about $700 billion in capital spending in 2026, nearly double last year, while data from outplacement firm Challenger, Gray & Christmas lists AI as the top reason for job cuts in the U.S. for a record fourth month in a row.
An internal meta memo seen by Reuters said the company’s heavy investments were offset by cutting 8,000 positions in May during a quarter in which sales rose 33%. Layoffs are not a survival strategy for companies like this. They’re making loans.
The problem is that the loan isn’t buying what it promised. Gartner surveyed 350 executives at companies with over $1 billion in revenue, all of whom have deployed AI agents or automation, and found that approximately 80% have cut headcount with no correlation to improved profits. Analyst Helene Poitevin’s verdict was straightforward: “Layoffs may create budget space, but they don’t generate profits.”
Uber learned the token side of that lesson the expensive way, equipping 5,000 engineers with AI coding tools in December and burning through its entire 2026 AI budget by April. Chief Operating Officer Andrew MacDonald acknowledged that even though 70% of committed code is generated by AI, there is a lack of connection to everything that customers notice: “That connection just doesn’t exist yet.”
Juxtaposing these two failures highlights the real problem. Companies treated token invoices as fixed and labor as flexible, even when the opposite was true. Once a pay cut occurs, institutional knowledge is introduced. It turns out that the token budget would be bent in six places if someone bothered to design it.
Where the token budget changes
The cheapest solution is also the least attractive. It means you stop paying to process the same text over and over again. Prompt caching, now standard across major API providers, reduces the cost of repeated input by up to 90% under Anthropic and OpenAI’s public pricing because static content, such as system instructions and reference documentation, is processed once and reloaded a fraction of the time.
Security firm ProjectDiscovery documented that rebuilding prompts increased cache hit rates from 7% to 84%, reducing total LLM spend by 59-70% while delivering 9.8 billion tokens from cache. This single engineering exercise recovered more money than most AI-induced rounds of attrition would save.
The next lever is to route the work to the appropriately sized model. Although the provider’s own price list shows that flagship models cost five times more per token than smaller models, many production workloads default to routine classification and summarization being sent to the most expensive tier. Batch processing adds an additional 50% discount for those that do not require real-time answers.
Search expansion generation approaches the problem from a different angle by sending only relevant slices of the knowledge base to the model, rather than the entire knowledge base, and uses prompt compression to trim redundant examples that would bloat every call. The open-weight model further reduces costs and handles everyday workloads at a fraction of Frontier API prices for teams willing to manage their infrastructure.
These measures are simply the equivalent of AI turning off the lights in an empty room, and Uber’s $1,500 per month cap per engineer, imposed after being exceeded in April, is early evidence that spending discipline will eventually arrive. Companies that get ahead of the curve simply choose to do so before budget constraints hit them.
The other half of the modifications are done by humans
Optimizing token billing only matters if the savings are spent on something productive, and the strongest evidence points to people. According to Poitevin’s research, organizations that increased ROI were those that used AI to augment their workforce rather than replace them.
Klarna ran a controlled experiment on everyone’s behalf and replaced around 700 customer service roles with OpenAI-powered assistants before customer satisfaction declined. CEO Sebastian Siemiatkowski told Bloomberg that “the result is a decline in quality, and it’s not sustainable,” something most executives don’t acknowledge out loud.
Fintechs are currently running a mixed model where AI absorbs routine tasks and rehired humans handle everything that requires judgment. Gartner expects this pattern to spread, predicting that by 2027, half of companies that cut customer service staff due to AI will rehire them.
Optimization logic makes one investment in people more urgent than optional. Employment of software developers between the ages of 22 and 25 is down nearly 20% from 2024 levels, even as the older population has grown, according to Stanford University’s Institute for Human-Centered AI. This means that companies are running out of training ground for senior engineers who will need to command all these systems within five years.
A company that just designed 60% off token billing has the budget to keep hiring lower-tier talent. Whether that happens depends on leadership decisions, not economic ones.
Nvidia’s Huang’s provocation will resonate at the company’s earnings conference, and capital spending will likely continue to rise. The companies leading the way are not the ones that spent the most on tokens or the ones that cut the most jobs to buy tokens. It will be a company that realizes from the beginning that the token budget is a flexible line, narrows the budget on engineering rather than people, and spends the difference on the people who bring value to the token.
(Image courtesy: kate.sade)
See: AI fees per token now on GitHub Copilot
Want to learn more about AI and big data from industry leaders? Check out the AI & Big Data Expos in Amsterdam, California, and London. This comprehensive event is part of TechEx and co-located with other major technology events such as Cyber Security & Cloud Expo. Click here for more information.
AI News is brought to you by TechForge Media. Learn about other upcoming enterprise technology events and webinars.
