Hello and welcome to Eye on AI. In this edition…The growing debate about AI's impact on productivity; Anthropic debuts the first hybrid reasoning model, Claude 3.7 Sonnet; AI models are prone to cheating their way to goals and that should worry us; a new U.K. report highlights the need for AI regulation.
In the U.S., Elon Musk and his minions at the newly created Department of Government Efficiency have been using AI to attempt to identify budget cuts across the federal government. Some of these cuts—such as slashing anything having to do with diversity or that touched on climate change—seem more driven by ideology than any actual drive for efficiency. Musk and his deputies have summarily dismissed thousands of federal workers and are threatening to fire thousands more, with AI being used again to help decide who to cull. And it seems that one reason Musk and his team think so many of these jobs can be eliminated without derailing vital government services is a belief that in many cases AI can take on the tasks these workers once performed.
But Musk’s faith in AI’s current abilities is not matched so far by evidence from the private sector, where there is a growing debate about exactly how much of a productivity boost AI is providing.
Doubts about generative AI are growing in some quarters
Whereas once CEOs were eager to play up vague notions of how AI was helping them boost their bottom lines, some are now trying to tamp down expectations. Brian Chesky, Airbnb’s CEO, told investors and analysts on a company earnings call last week that AI coding assistants have not led to a “fundamental step change in productivity yet.” (Chesky’s comments are perhaps particularly telling because he is known to be a close friend of OpenAI CEO Sam Altman. So you might expect him to have a more boosterish view.)
But many execs say the productivity boost is real
Chesky’s view is in stark contrast to what many other executives have been saying about the technology. Ashok Srivastava, Intuit’s senior vice president and chief data officer, told me at the end of last year that the financial and tax software giant had seen an “eight-fold increase in development velocity” (as measured by new software and version releases) over the past four years as the company has deployed more and more AI across its own technology platform. Generative AI coding assistants have helped boost that productivity by an additional 15%, he said.
The company has also seen that using generative AI to help categorize customer inquiries and coach human customer support agents has resulted in an 11% increase in the efficiency of its “customer success” teams—which help Intuit’s customers better use the company’s products, such as QuickBooks, TurboTax, and Mailchimp. And some of the new generative AI features are resulting in real gains for Intuit’s customers too, Srivastava said. He noted that using a QuickBooks AI-powered feature that generates automatic invoice reminders had resulted in Intuit customers getting paid 45% faster than before.
Success theater?
Those numbers certainly sound impressive. But it’s still a little difficult to tell from the outside what’s real and what is simply what Edward Achtner, the head of generative AI at the bank HSBC, termed “success theater,” in reference to other company’s AI claims. If it is theater, though, HSBC is also on the stage. Its CEO, George Elhedery, told investors just last week that the bank was rolling out generative AI applications in customer service and, yes, coding assistants for its engineers.
Solow’s paradox redux
Of course, productivity gains from new technologies, especially digital ones, can take a long time to become evident. In 1987, economist Robert Solow famously quipped that “you can see the computer age everywhere except in productivity statistics.” (A phenomenon soon named Solow’s Paradox.) But starting eight years later, in 1995, labor productivity did begin to accelerate significantly and remained elevated for the next 10 years as internet adoption flourished. Economists speculate that there is a significant lag in how long it takes companies to adopt new technologies, but also in how long it takes to train workers to use them effectively and to reconfigure work practices to get the largest productivity gains from them. While AI adoption rates are very rapid compared to previous technologies, we are still just two years into the generative AI revolution. So it may take a few years yet to see what the gains really are.
With that, here’s more AI News.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
