Hello and welcome to December’s special edition of Eye on A.I.
Is ChatGPT the end of trust? That’s what some people are suggesting after the release of OpenAI’s chatbot ChatGPT, which is shockingly good at mimicking human writing in almost any format, from computer code and poetry to blog posts and polemical essays. Much of what the chatbot spits out is factually accurate. But much of it isn’t. And the problem is that there is no easy way for a user to ensure that ChatGPT’s responses are accurate. ChatGPT expresses both fact and fiction with equal confidence and style.
Never mind that the written word has had trust issues since the very beginning of writing. (Ancient scribes were often propagandists and fabulists after all.) There does seem to be something different about the way ChatGPT can create fluent and confident answers to almost any question in less than a second—and right now since OpenAI isn’t charging for it, it does so at zero cost to the user. Before, creating a convincing fraud would take time and serious effort. But tools like ChatGPT mean that the marginal cost of creating misinformation has essentially dropped to zero. That means we are likely to see an explosion of it.
Some say we have already seen the first victim of this misinformation eruption: Stack Overflow, a site that provides community-sourced answers to people’s computer coding questions, had to bar users from submitting answers created by ChatGPT after being overwhelmed with responses created by the chatbot. The problem, Stack Overflow said, is that the answers seemed very convincing, but were actually wrong, and it was taking their community moderators too long to vet all the answers and discover the flaws.
Things are going to get a lot worse if one of the new advocates for open-sourcing A.I. models decides to build a ChatGPT clone and make it fully available for free as an open-source project. (Right now OpenAI still controls the model behind ChatGPT and users can only query the model through an interface that OpenAI could shut down, or impose charges to use, at any time. Its terms of use also bar people from using the bot to run misinformation campaigns.) Already Ehmad Mostaque, the former hedge fund manager who runs Stability AI, the company that helped train and open-source the popular text-to-image system Sable Diffusion, has asked his Twitter followers whether Stability should create an open source version of ChatGPT.
As part of its release of ChatGPT, OpenAI also released an A.I. system that can detect whether text was created using ChatGPT. The open-source A.I. startup Hugging Face hosts an interface to that ChatGPT detector on its website and, in experiments, Casey Fiesler, a professor of information science at the University of Colorado at Boulder, said on Twitter that when she fed the detector five student-written essays and five created using ChatGPT, it flagged all five ChatGPT-made ones with 99.9% confidence. But some researchers say they doubt the detector will work on all future versions of the A.I. system, or will work for any similar, but not identical, large language models that others train. Earlier research on large language models had found that A.I. systems were poor at differentiating between A.I.-created and human-written text.
One area where many people think ChatGPT and similar systems will have an immediate and profound effect is education. Many are saying such systems mean the end of using any kind of write-at-home essay or report for student assessments. It might mean the end of college application essays and term papers. The Atlantic had a very good piece examining this last week. I asked a friend of mine who is a university professor what he thought and he answered unequivocally that the term paper was finished. He said he thought professors will have to rely solely on proctored exams where students are asked to hand write their essays (or type on computers that they can prove are not attached to the Internet.)
Kevin Scott, Microsoft’s chief technology officer, said at Fortune’s Brainstorm A.I. conference in San Francisco last week that teachers who were wringing their hands about ChatGPT were making “a pedagogical mistake” in confusing the essay, which he said was simply “an artifact,” with the important information that the teacher is actually trying to ensure that the student has learned.
He seemed to say that ChatGPT would no more destroy the teaching of humanities than the calculator had destroyed the teaching of mathematics. “In a sense, nothing really is changing here other than you have this tool, and the student themselves has to become the teacher to the model,” he said, meaning that the student will still have to go over the answer that the large language model produces and ensure that it is not making up information. The student, for now, would still have to provide accurate citations of where the information was coming from. “Your job is: be the editor for this thing, be the teacher, coax it into getting you the output that you really need. That’s the important thing, that's the special thing about us. The thing is just a tool.”
Scott is not alone in the view that ChatGPT could actually be great for education. But I think Scott and others are missing something here. Teachers use essays for more than just assessing what facts a student has learned. That may be true in elementary school. But at the high school and, certainly at the university level, teachers use essays not simply to see what facts students know but if they can use those facts to make a well-reasoned argument. The facts are simply supporting evidence. They are necessary but not sufficient in order to get top marks. Teachers also use essays to assess how well the student is able to express their ideas in writing—how graceful his or her prose is, can the student come up with original and appropriate metaphors, etc.
Perhaps most importantly, it is difficult to separate the act of composition from the act of thinking—by writing a person is forced to structure their thoughts, refine their ideas, marshal evidence, and consider counter-arguments. There’s a lot of learning that takes place in the act of composition itself. Most of that disappears when ChatGPT or its successor bots can simply churn out page after page of well-written and well-structured prose and the student is reduced to being a mere fact-checker and annotator. We write not merely to convey information, but to conjure and refine it.
Read on for a few more A.I.-related stories from the past week. Fortune’s newsletters, including Eye on A.I., are going on hiatus for the holidays. The next Eye on A.I. will be in your inboxes on Jan. 10. In the meantime, happy holidays and a happy new year to you all! See you in 2023.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
