It’s a clash of tech titans.
Sam Altman, CEO of OpenAI, says that regulatory intervention by governments will be critical to mitigate the risks of increasingly powerful [AI] models.” Legendary tech investor Marc Andreesen demurs, arguing that such regulation will likely entrench the power of the market leaders and deter innovation.
It’s an old truism of military intelligence that when your worst case forecast and your best case are too far apart, you need more information. That’s why the right regulatory response is to start with registration and mandatory disclosure of the latest measures of AI risks and performance characteristics. You can’t regulate what you don’t understand.
These disclosures should initially be based on current best practices in the management of AI systems. Good work is already being done here, but the results are published inconsistently and irregularly.
Disclosures should be mandatory, consistent, regular, and independently audited, much as we require public companies to disclose and audit their financials.
The work that Timnit Gebru, Margaret Mitchell and their coauthors have done on the disclosure of the performance characteristics and risks of trained AI models (“Model Cards for Model Reporting”) and training data (“Datasheets for Datasets”) are a very good first draft of something like the Generally Accepted Accounting Principles that guide financial reporting. Might we call them “Generally Accepted AI Management Principles”?
It’s essential that these generally accepted AI management principles are created in close cooperation with the creators of AI systems, so they reflect actual best practice rather than a set of rules imposed from without by regulators and advocates.
But they can’t be developed solely by the tech companies themselves. In his book Voices in the Code, James G. Robinson (now Director of Policy for OpenAI) points out that every algorithm makes moral choices, and explains why those choices must be hammered out in a participatory and accountable process.
Robinson also calls for disclosure, with a twist.
He believes that part of disclosure should be predictions about the impact of algorithms (including AI), followed by auditing to understand whether those predictions are correct. His case study in the book isn’t AI proper, but the kidney transplant matching algorithm, where predictions of expected outcomes are clearer and more measurable, but this idea can still be adapted for AI.
The model card for GPT-4, for example, reads much like the registration statement in preparation for an IPO. What we need now is the forward guidance, and then retrospective accounts of whether it has been met, much as happens in the quarterly and annual financial reports required by public companies by the SEC.
What might such forward guidance consist of? Well, if the model card highlights various risks, and promises that the company is working to counter them, what are the measurements that indicate success or failure? If GPT-4 “hallucinates’,” what is the hallucination rate? Is it going up or down? If “red teaming” has indicated that humans can get around safeguards against providing harmful information, how many attempts were there? What percentage was countered, and what percentage succeeded?
This kind of reporting is certainly possible for large, centralized models like those from OpenAI, Google, and Microsoft, since they keep track of user requests and use them for further training of the model. It’s more difficult for distributed open source models, but for the moment at least, those are less powerful.
We are a long way from knowing what all of the best practices are, but much as accounting standards are based on a consensus view of what good financial reporting contains, adopted widely by industry and then codified into formal reporting and auditing, understanding what companies are actually doing (or not doing) to manage the risks of the AI models they develop is a good place to start.
Tim O’Reilly is founder and CEO of O’Reilly
