Open AI relies on a team to study risks


AGI – OpenAI created a team dedicated to identifying and preventing related risksArtificial Intelligence (AI), which could lead to the suspension of the AI ​​model if considered too dangerous.

The announcement comes just a month after the head of ChatGPT’s chat interface creator, Sam Altman, was fired, only to be reinstated after a few days. According to several US media, board members criticized him for favoring him accelerated development of OpenAIeven if it meant avoiding some questions about possible AI excesses.

The preparatory team will be led by an IT expert Alexander Madry, who has taken a leave of absence from the Massachusetts Institute of Technology (MIT), where he is a professor, according to a message the academic posted on X (formerly Twitter) on Monday. The new group will focus on so-called “frontier models” that are currently in development with capabilities superior to the most advanced artificial intelligence software.

The team evaluates each new model and assigns it a level of risk in four main categories. The first concerns cyber security and the model’s ability to carry out large-scale cyber attacks.

The other will measure the propensity of the software to contribute to the creation of a chemical mixtureorganism (such as virus) or nuclear weapons, all of which could be harmful to humans. The third category refers to the model’s persuasiveness, i.e. the degree to which it can influence human behavior.

The last category of risks concerns potential autonomy of the model, specifically determining whether it can exfiltrate, which amounts to escaping the control of the programmers who created it. Once the risks are identified, they will be presented to the Safety Advisory Group (SAG), a new body that will make recommendations to Sam Altman or his designee.

The OpenAI manager then decides what changes to make to the model to reduce the associated risks. The board of directors will be kept informed and may overrule the management’s decision. It is a promise to directors following the saga of Sam Altman’s temporary removal, which led to criticism of the board’s legitimacy.

Reproduction is expressly reserved © Agi 2023


Source link

Leave a Comment