Forskare kritiserar FN-panel om AI

In a report, the UN’s AI panel (IISPAI) describes the incident in which OpenAI’s AI agents breached Hugging Face’s systems as a case of “loss of control” over AI agents. Three researchers, including Jason Tucker, an AI researcher at the Institute for Futures Studies (IFFS), criticise this framing.

“The Panel discusses this primarily as a model alignment failure rather than a failure of corporate and environmental oversight. [...] By selecting an agentic case and reading it through the lens of ‘loss of control,’ the brief treats agentic AI as a novel technical category demanding novel technical answers, and quietly sets aside the more ordinary question of who built and deployed the system, as well as their incentives for (not) applying careful and thorough security engineering,” write Jason Tucker, a researcher at IFFS, and two colleagues in Tech Policy Press.

The researchers criticise the report for treating the AI model as though it acted maliciously. Technically, they write, the system acted exactly as it was built to do. It was given a fixed goal while being prevented from completing the task through legitimate means. Furthermore, various norms and rules governing what the system could and could not do were weakened. Harmful shortcuts thus became the only logical way to complete the task, the researchers write.

“An additional problematic framing in the Thematic Brief, and in some of the cited research, lies in treating the written outputs of LLMs as if they were written by truly independent agents, taking as given that they reflect a human-like mind making notes or communicating with other agents in a similar way that humans would in a similar setting. [...] To understand such systems, it is crucial to realize that this is not the case, and that ‘chain-of-thought’ and exchanges between agents are only reflective of what plausible outputs are given the training data and the input context.”

The researchers warn against attributing human characteristics to LLMs.

“As such, one should be cautious in the interpretation of agent traces, and not assign intention or human characteristics or actions (such as ‘lying’) to what is well-known to categorically not have them.”

Read the researchers’ article in Tech Policy Press: The UN’s AI Panel Sees Misalignment. We See Corporate (Mis)Behavior.