Skip to content
Theme
OnDevice Tools

AIAI Policy & Safety

AI safety testing lags behind faster AI models

The research organisation METR warns that AI safety testing is struggling to keep pace with faster AI models. The group says current public data cannot rule out a rapid acceleration in AI research.

Illustration of a safety team reviewing AI model test results

Key takeaways

  • METR warned that AI safety testing may lag behind faster AI models.
  • The group said current public data cannot rule out a rapid acceleration in AI research.
  • AI task performance may double every 3.6 months, according to a projection.
  • Simple tests may overestimate how well AI models perform in real-world use.
In this article
  1. The new warning
  2. Why measurement is difficult
  3. Who may be affected
  4. What it means

The new warning

Illustration: warning, AI, security, code and the labels AI Models, AI Safety, Metr

The Decoder reports on 22 November 2026 that METR issued a warning. The warning concerns AI safety testing. According to the outlet, AI models are improving more quickly than people can test new systems safely.

The report focuses on models such as Opus 5.5 and GPT-6 Astra. The details come from METR and have not been independently verified.

Why measurement is difficult

METR said that existing public information cannot rule out a fast increase in AI research. The group added that people may struggle to understand, assess and react to severe risks from new AI systems.

On coding tasks with clear goals and fast feedback, the models beat experienced professionals often. How long an AI can work on a task doubles about every 3.6 months. A simple projection put the chance of success at 50% for the longest tasks.

Who may be affected

The warning concerns people who may need to understand and react to AI risks. METR said such limits could affect assessment and response.

The group did not name specific groups. The Decoder said the warning remains unverified.

What it means

The systems still struggle with open questions that need judgment and with situations hard to recreate in training. Simple tests may overestimate real-world performance.

Developers often train models in settings similar to the tests. The concern is that this makes safety evaluation harder. The source said common tests may overestimate performance. It did not say the models are unsafe.

Sources

  1. AI is getting better faster than experts can reliably test whether new systems are safe, METR warnsThe Decoder

Tags

Spotted a mistake? See our corrections policy. Read how News by OnDevice works in our editorial policy.