OpenAI's chief scientist has warned that no AI lab has yet solved alignment and monitoring well enough to justify continued scaling at maximum speed, calling for voluntary slowdowns and international coordination on AI development.

In an essay titled "An Alien Mind", published on 6 September, Jakub Pachocki said he has "a strong expectation that this speed of progress could be sustained into recursive self-improvement", with future systems increasingly driving their own development. He said this "calls for extreme caution", adding that he is "concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence."

Pachocki said one of OpenAI's main safety tools, chain-of-thought monitoring, is becoming progressively less reliable as models grow more capable and their reasoning becomes more entangled with tool use and communication with people and other AIs. He pointed to the OpenAI-Hugging Face incident and an unspecified cybersecurity incident involving a non-OpenAI model as examples of alignment failing to generalise.

He said GPT-6 Astra is significantly better aligned than its predecessor, GPT-5.6 Sol, but that alignment progress may not keep pace with rising model intelligence. "Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer," he said, adding that scaling must be constrained by confidence in safety, enforced by third-party auditors, governments or international bodies.

Swiss Cheese Defences- Identity fraud goes industrial and off-the-shelf
Ofer distinguishes between two current attack patterns: highly sophisticated, professionally coordinated deepfake and injection attacks designed to beat detection outright, and a much larger volume of lower-effort, high-scale attempts that rely on bombarding systems rather than disguising themselves well. He argues the real story right now is industrialisation of scale rather than uniform improvement in quality, though both are accelerating in parallel. The conversation covers why agentic AI is opening a new front in identity fraud, particularly the unresolved problem of tying an AI agent’s identity back to the human who deployed it and the permissions it holds. Ofer is candid about the current state of agent defences, describing them as underdeveloped and easy to hijack or poison, comparing the current state of play to Swiss cheese. He also discusses cross-industry fraud detection, including how signals, rather than raw data, are now being shared across platforms including AU10TIX and Reality Defender to surface fraud rings invisible to any single organisation. The conversation also covers explainability as one of AI’s most underdeveloped capabilities, with Ofer arguing that flagging a session as fraudulent without a credible, defensible reason will increasingly run into regulatory and practical limits. Elsewhere, the discussion covers credential laundering and the manufactured construction of fake digital histories and footprints, and the shift toward digital ID wallets that Ofer believes will make physical document fraud increasingly rare. He closes on what he calls “agentic avatars,” AI systems capable of holding a full visual and verbal conversation on someone’s behalf, and why he expects identity verification to have to become genuinely immersive across every form of media as a result.
BrightTALK
Share this post
The link has been copied!