OpenAI established a super intelligent AI control group but then neglected it

A research team at OpenAI, whose mission is to find ways to control “super-intelligent” AI systems, has not received enough computing resources needed to do their work. This, along with other disagreements, caused several key team members to quit, including team leader Jan Leike.

Leike believes that OpenAI should focus more on ensuring the safety of the next generations of AI. He is concerned that OpenAI is prioritizing new product launches over addressing important safety issues.

OpenAI did not immediately respond to the issue of resources promised and allocated to this research group.

The research team was founded last July with the ambitious goal of solving the core technical challenges of controlling super-intelligent AI over the next four years. However, as new product launches took up more of OpenAI leadership’s attention, the research team had to fight for more resources.

Over the past few months my team has been sailing against the wind. Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done.

— Jan Leike (@janleike) May 17, 2024

Although the group published several safety studies and funded millions of dollars in grants to outside researchers, they felt sidelined.

Leike warns that building machines that are smarter than humans is a potentially risky endeavor. He believes that over the years, safety culture has been overlooked in favor of flashy products.

We’re really grateful to Jan for everything he's done for OpenAI, and we know he'll continue to contribute to the mission from outside. In light of the questions his departure has raised, we wanted to explain a bit about how we think about our overall strategy.

First, we have… https://t.co/djlcqEiLLN

— Greg Brockman (@gdb) May 18, 2024

Disagreements between OpenAI co-founder Ilya Sutskever and CEO Sam Altman further complicate the situation. Sutskever once asked for Altman to be fired because he was concerned that the CEO had not been honest with the board. However, under pressure from investors and employees, Altman was reinstated, and Sutskever did not return to work.

After Leike quit, Altman admitted there was still much work to be done and promised to deal with it. However, specific commitments are still unclear.

OpenAI, instead of maintaining a separate team, will disperse safety researchers into different departments. This makes many people worry that OpenAI’s AI development will not be as safe as before.