AI 'Godfather' Backs New Oversight Plan for OpenAI and Anthropic

In a significant development for artificial intelligence governance, the AI Evaluator Forum has unveiled a detailed proposal aimed at enhancing the oversight of advanced AI systems developed by companies such as OpenAI and Anthropic. This initiative, backed by prominent figures in the AI community, including the esteemed computer science pioneer Geoffrey Hinton, articulates essential criteria for third-party evaluators to effectively assess and mitigate potential risks associated with rapidly evolving AI technologies. The forum's recommendations underscore the critical need for independent, transparent, and scientifically rigorous evaluation processes, marking a pivotal step towards ensuring the responsible development of AI.
The genesis of this new framework can be traced to recent commitments made by OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei. Both leaders had expressed openness to integrating external evaluators into their organizations, granting them unprecedented access to internal operations and data to scrutinize AI risks. This willingness from industry giants emerged amidst growing public apprehension regarding the safety and ethical implications of advanced AI. The AI Evaluator Forum's letter, released this past Friday, directly responds to these commitments, translating the broad concept of external evaluation into concrete, actionable guidelines. The letter specifies that for these evaluations to be credible, evaluators must maintain scientific objectivity, operate with complete transparency, and be fully independent from the companies they are assessing. Crucially, robust protections against any form of interference from the evaluated firms are also deemed indispensable.
A key tenet of the proposed plan involves empowering evaluators with the ability to communicate directly and without filters with the companies' boards and other high-level oversight bodies. This direct line of communication is intended to ensure that critical findings and concerns are not diluted or suppressed. Furthermore, the forum advocates for evaluators to have the freedom to publicly disclose their findings, fostering an environment of accountability and public trust. To prevent any potential retaliation, the letter stresses the importance of shielding evaluators from adverse actions by AI labs. Equal access to relevant systems, data, tools, and physical spaces, comparable to that afforded to internal company assessors, is also stipulated as a prerequisite for thorough and effective evaluation. The forum also highlights the necessity of incorporating diverse perspectives and specialized expertise among evaluators, particularly concerning varied types of risks, including those in biological sciences and issues related to control.
Among the organizations comprising the AI Evaluator Forum are METR, a non-profit AI evaluation body that recently examined a security incident involving OpenAI and Hugging Face, and the AI Verification and Evaluation Research Institute (AVERI), which is co-led by Miles Brundage, a former OpenAI employee. The collective expertise and prior engagements of these groups lend significant weight to the forum's recommendations. The discussions around AI safety and oversight have been dynamic, with other prominent tech leaders also contributing their views. Microsoft CEO Satya Nadella, for instance, has voiced his support for the concept of 'embedded evaluators,' acknowledging the benefits of integrating independent scrutiny into AI development processes. This broad consensus among key stakeholders signals a collective recognition of the importance of establishing robust oversight mechanisms to navigate the complex challenges and opportunities presented by advanced artificial intelligence.