Good morning!
A new study from Boston Consulting Group’s Henderson Institution think tank and academics at four top business schools, including Harvard Business School and MIT Sloan School of Management, provides some early insight into the benefits of generative AI when used for the right tasks—and the destruction it can create when used for the wrong ones.
Researchers experimented on more than 750 BCG consultants, assessing ChatGPT-4's efficacy in assisting them on two different tasks. The first task, “creative product innovation,” asked the test subjects to brainstorm ideas for a new product, develop a business case, test and launch the product, and create a marketing campaign. The second task, “business problem solving,” asked subjects to fix a fictitious company’s challenges, assess business performance data, and analyze an interview transcript with company executives.
The findings show a 40% performance boost for consultants using the chatbot for the creative product project, compared to the control group that did not use ChatGPT, but a 23% decline in performance when used for business problem-solving.
The chatbot likely performed better with the creative project because it’s easier for large language models (LLMs) to develop innovative, novel, or valuable ideas due to the large swath of data they’re trained on. Meanwhile, the chatbot decreased business problem-solving performance because LLMs are more prone to error when analyzing quantitative and qualitative data to solve complex problems.
The researchers found that ChatGPT enhanced the performance of nearly all (90%) consultants who used it for the creative product assignment. But that boost was more prominent among low-performing employees, who saw a 43% performance increase when using the chatbot versus a 17% increase for top-performing employees.
There was also a 41% decline in the diversity of thought when using generative AI for the creative product assignment because ChatGPT churned out similar responses to prompts.
For HR leaders, the findings underscore the need for strategic workforce planning, namely, the human capabilities a company will need and how AI can best drive that performance.
“We're just scratching the surface of this revolution. And this is why it's fascinating to be a CHRO at the moment because you will be in a position to really lead on [human-AI interactions],” says Francois Candelon, global director of BCG’s Henderson Institute. “To experiment, to understand how it works, who I should hire, what capabilities I need over time, and so on. So it's a lot of work for CHROs.”
Paige McGlauflin
paige.mcglauflin@fortune.com
@paidion
