Dive Brief:
- The U.S. and U.K. will work together to develop tests for advanced AI models and build a standard approach to AI safety testing as part of a nonbinding agreement signed by both nations Monday.
- The collaboration will accelerate the U.S. and U.K. AI Safety Institutes’ efforts to mitigate risks associated with AI “across the full spectrum,” a result of commitments agreed upon during the AI Safety Summit in the U.K. last November, according to the announcement.
- “Our partnership makes clear that we aren’t running away from these concerns — we're running at them,” U.S. Secretary of Commerce Gina Raimondo said in a statement. “Because of our collaboration, our institutes will gain a better understanding of AI systems, conduct more robust evaluations and issue more rigorous guidance.”
Dive Insight:
Mitigating AI risks is a shared priority for leaders around the world. But for regulators to reach their goals, they have to cultivate expertise.
Enforcers require an understanding of the technology at a deeper level to adequately craft rules and evaluate models, but the swift pace of innovation and novelty challenges regulators, who are playing catch-up without a clear guide.
Most vendors release research evaluating models’ scores on popularly used tests.
Public forums also give insight into model behavior, such as the chatbot arena created by the Large Model Systems Organization. The crowdsourced open platform has collected more than 500,000 human preference votes to rank the LLMs. Claude 3 Opus is tied for the top spot with two OpenAI GPT-4 preview models, as of March 29.
The goal of the U.S. and UK AI Safety Institute partnership is to create a standard approach to AI testing so that regulators can use the evaluations to enforce impending safety and security requirements.
Despite a lack of formal regulation, the U.S. is making progress on its goal to reduce AI’s harm and harness its potential. Federal agencies have since completed all of the tasks the administration ordered within 150 days.
To keep momentum, the White House is pushing to hire 100 AI professionals by this summer.
Meanwhile, enterprises are weighing the pros and cons of adopting large language models and generative AI tools, but IT leaders have confronted challenges.
The perceived pressure to rapidly adopt generative AI, combined with the lack of technical know-how in most organizations, creates ample opportunities for missteps. The landscape has pushed CIOs to rethink talent strategies, level up their skills and train employees.
