<\/p>\n
Quick answer:<\/strong> Jensen Huang has a test for whether an engineer is worth keeping, and it comes with a token budget attached. Speaking on the All-In Podcast at the close of GTC 2026, the Nvidia chief executive said that if a 0,000 engineer\u2019s annual AI token consumption came in under half their salary, \u201cI am going to.<\/p>\n<\/div>\n He was describing a trade-off most companies have already made with less fanfare: money that once paid people increasingly pays for tokens. The four largest hyperscalers have guided<\/a> roughly $700 billion in combined 2026 capital expenditure, nearly double last year, while data from outplacement firm Challenger, Gray & Christmas shows<\/a> AI as the most-cited reason for US job cuts for a record fourth consecutive month.<\/p>\n An internal Meta memo obtained by Reuters<\/em> described May\u2019s cuts of 8,000 roles as offsetting the company\u2019s substantial investments, in a quarter when revenue grew 33%. The layoffs at companies like these aren\u2019t survival measures. They\u2019re financing.<\/p>\n The trouble is that the financing hasn\u2019t bought what it promised. Gartner surveyed 350 executives at companies with over $1 billion in revenue, all deploying AI agents or automation, and found roughly 80% had cut headcount with no correlation<\/a> to improved returns. Analyst Helen Poitevin\u2019s verdict was blunt: \u201cWorkforce reductions may create budget room, but they do not create return.\u201d<\/p>\n Uber learned the token side of that lesson the expensive way, giving 5,000 engineers AI coding tools in December and exhausting its entire 2026 AI budget by April. Chief Operating Officer Andrew Macdonald conceded that despite 70% of committed code being AI-generated, the connection to anything customers notice is missing: \u201cThat link is not there yet.\u201d<\/p>\n Put those two failures side-by-side and the actual problem comes into focus. Companies treated the token bill as fixed and the workforce as flexible, when the opposite is true. Payroll cuts happen once and take institutional knowledge with them. A token budget, it turns out, bends in half a dozen places if anyone bothers to engineer it.<\/p>\n The cheapest fix is also the least glamorous: stop paying to process the same text repeatedly. Prompt caching, now standard across the major API providers, cuts the cost of repeated input by up to 90% under Anthropic\u2019s and OpenAI\u2019s published pricing, because static content like system instructions and reference documents gets processed once and reread at a fraction of the rate.<\/p>\n Security firm ProjectDiscovery documented raising its cache hit rate from 7% to 84% by restructuring prompts, cutting its total LLM spend by 59 to 70% while serving 9.8 billion tokens from cache. That single engineering exercise recovered more budget than most AI-attributed layoff rounds save.<\/p>\n The next lever is routing work to the right-sized model. Providers\u2019 own price lists show flagship models costing five times their smaller siblings per token, yet plenty of production workloads send routine classification and summarisation to the most expensive tier by default. Batch processing adds a further 50% discount for anything that doesn\u2019t need a real-time answer.<\/p>\n Retrieval-augmented generation attacks the problem from another angle by sending the model only the relevant slice of a knowledge base rather than the whole thing, and prompt compression trims the redundant examples that inflate every call. Open-weight models reduce costs further still, handling routine workloads at a fraction of frontier API prices for teams willing to manage the infrastructure.<\/p>\n These measures are simply the AI equivalent of turning off the lights in empty rooms, and Uber\u2019s $1,500 monthly cap per engineer \u2013 imposed after the April overrun \u2013 is early evidence that spending discipline arrives eventually. The companies getting ahead are simply choosing it before the budget forces it.<\/p>\n Optimising the token bill only matters if the savings go somewhere productive, and the strongest evidence points at people. Poitevin\u2019s research found the organisations that improved ROI were those using AI to amplify their workforce rather than replace it.<\/p>\n Klarna ran the controlled experiment on everyone\u2019s behalf, replacing roughly 700 customer service roles with an OpenAI-powered assistant before customer satisfaction fell. Chief Executive Sebastian Siemiatkowski told Bloomberg<\/em> what few executives admit aloud: \u201cThe result was lower quality, and that\u2019s not sustainable.\u201d<\/p>\n The fintech now runs a blended model, with AI absorbing routine volume while rehired humans handle everything requiring judgment. Gartner expects the pattern to spread, predicting that by 2027 half the companies that cut customer service staff for AI will rehire them.<\/p>\n There\u2019s one workforce investment the optimisation logic makes urgent rather than optional. Stanford University\u2019s Institute for Human-Centered AI found<\/a> employment for software developers aged 22 to 25 fell nearly 20% from 2024 levels even as older cohorts grew, which means companies are removing the training ground for the senior engineers they\u2019ll need directing all these systems in five years.<\/p>\n A business that has just engineered 60% off its token bill has the budget room to keep hiring at the bottom rung. Whether it does is a leadership decision, not a financial one.<\/p>\n Nvidia\u2019s Huang\u2019s provocation will keep echoing through earnings calls, and the capex numbers will keep climbing. The companies that come out ahead won\u2019t be the ones that spent the most on tokens or cut the most people to afford them\u2014they\u2019ll be the ones that noticed the token budget was the flexible line all along, squeezed it with engineering rather than headcount, and spent the difference on the people who make the tokens worth anything.<\/p>\nWhere the token budget bends<\/strong><\/h3>\n
The other half of the fix is human<\/strong><\/h3>\n