Dive Brief:
- OpenAI is continuing its focus on increasing human control over AI models in anticipation of superintelligent systems, the company announced Friday.
- The leading AI startup plans to dole out $10 million in grants to support technical research around the topic, OpenAI said in a blog post. Academic labs, nonprofits and individual researchers can apply for grants between $100,000 to $2 million.
- OpenAI is “particularly interested” in funding research related to the impact of human reviewers when an AI system is much smarter, understanding model interpretability, using AI systems to monitor other AI systems and adversarial robustness among other research areas, according to the blog post.
Dive Insight:
Regulators, providers and enterprise leaders have been focused on ways to future-proof AI as the technology develops.
The White House’s executive order aimed to put guardrails around AI models that surpassed a specific computing threshold, according to an October announcement. The initial obligations of the European Union’s AI Act, announced earlier this week, included a requirement allowing citizens the right to launch complaints about AI systems and receive explanations about decisions based on high-risk AI systems that impact them.
OpenAI expects AI systems to become significantly harder to manage within the next decade, according to the company’s blog post.
“Today, we align AI systems to ensure they are safe using reinforcement learning from human feedback,” OpenAI said. “Existing alignment techniques like [reinforcement learning from human feedback] that rely on human supervision may no longer be sufficient.”
Foundational models, like OpenAI’s GPT, are often referred to as black boxes due to their complexity and the challenging nature of interpreting their inner workings. As foundational models become more common additions to enterprise and government tech stacks, researchers are working to advance existing oversight resources.
“If there is continued rapid progress in AI capabilities, society may need to repeatedly reach agreement on new best practices for each more capable class of AI systems, in order to incentivize speedy adoption of new practices that address these systems’ greater risks,” OpenAI researchers said in a paper published Friday.
