While a growing number of organizations have articulated ethics principles to guide their AI development process, it can be difficult for those outside of an organization to verify whether the organization’s AI systems reflect those principles in practice. This ambiguity makes it harder for stakeholders such as users, policymakers, and civil society to scrutinize AI developers’ claims about properties of AI systems and could fuel competitive corner-cutting, increasing social risks and harms. The report describes existing and potential mechanisms that can help stakeholders grapple with questions like:

The 10 mechanisms highlighted in the report are listed below, along with recommendations aimed at advancing each one. (See the report⁠(opens in a new window) for discussion of how these mechanisms support verifiable claims as well as relevant caveats about our findings.)

We and our co-authors will be doing further research on these mechanisms and OpenAI will be looking to adopt several of these mechanisms in the future. We hope that this report inspires meaningful dialogue, and we are eager to discuss additional institutional, software, and hardware mechanisms that could be useful in enabling trustworthy AI development. We encourage anyone interested in collaborating on these issues to connect with the corresponding authors and visit the report website⁠(opens in a new window).

Miles Brundage⁠ (OpenAI)Shahar Avin⁠ (Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence)Jasmine Wang⁠ (Mila, University of Montreal)Haydn Belfield⁠ (Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence)Gretchen Krueger⁠ (OpenAI)

Gillian Hadfield (OpenAI, University of Toronto, Schwartz Reisman Institute for Technology and Society)Heidy Khlaaf (Adelard)Jingying Yang (Partnership on AI)Helen Toner (Center for Security and Emerging Technology)Ruth Fong (University of Oxford)Tegan Maharaj (Mila, Montreal Polytechnic)Pang Wei Koh (Stanford University)Sara Hooker (Google Brain)Jade Leung (Future of Humanity Institute)Andrew Trask (University of Oxford)Emma Bluemke (University of Oxford)Jonathan Lebensold (Mila, McGill University)Cullen O’Keefe (OpenAI)Mark Koren (Stanford Centre for AI Safety)Théo Ryffel (École Normale Supérieure [Paris])JB Rubinovitz (Remedy.AI)Tamay Besiroglu (University of Cambridge)Federica Carugati (Center for Advanced Study in the Behavioral Sciences)Jack Clark (OpenAI)Peter Eckersley (Partnership on AI)Sarah de Haas (Google Research)Maritza Johnson (Google Research)Ben Laurie (Google Research)Alex Ingerman (Google Research)Igor Krawczuk (École Polytechnique Fédérale de Lausanne)Amanda Askell (OpenAI)Rosario Cammarota (Intel)Andrew Lohn (RAND Corporation)David Krueger (Mila, Montreal Polytechnic)Charlotte Stix (Eindhoven University of Technology)Peter Henderson (Stanford University)Logan Graham (University of Oxford)Carina Prunkl (Future of Humanity Institute)Bianca Martin (OpenAI)Elizabeth Seger (University of Cambridge)Noa Zilberman (University of Oxford)Seán Ó hÉigeartaigh (Leverhulme Centre for the Future of Intelligence, Centre for the Study of Existential Risk)Frens Kroeger (Coventry University)Girish Sastry (OpenAI)Rebecca Kagan (Center for Security and Emerging Technology)Adrian Weller (University of Cambridge, Alan Turing Institute)Brian Tse (Future of Humanity Institute, Partnership on AI)Elizabeth Barnes (OpenAI)Allan Dafoe (Future of Humanity Institute)Paul Scharre (Center for a New American Security)Ariel Herbert-Voss (OpenAI)Martijn Rasser (Center for a New American Security)Shagun Sodhani (Mila, University of Montreal)Carrick Flynn (Center for Security and Emerging Technology)Thomas Gilbert (University of California, Berkeley)Lisa Dyer (Partnership on AI)Saif Khan (Center for Security and Emerging Technology)Yoshua Bengio (Mila, University of Montreal)Markus Anderljung (Future of Humanity Institute)