tl;dr

When prompted, current models can sandbag ML experiments and research decisions without being detected by zero-shot prompted monitors. Claude 3.5 Sonnet (new) can only sandbag effectively when seeing a one-shot example, while Claude 3.7 Sonnet can do this without an example (zero-shot). We are not yet worried about sabotage in today's models, as the sandbagging we observe today would be noticed by humans.

Twitter thread about the blog post here.

Introduction

AI models might soon assist with large parts of AI and AI safety research, completing multiple hours of work independently. These automated researchers will likely not be sufficiently capable to cause immediate catastrophic harm. However, they will have the opportunity to engage in sabotage, including sabotaging the research we rely on to make more powerful AI models safe. For example, a malicious automated researcher might slow down AI safety research by purposefully making promising experiments fail.

In this work we aim to ensure that automated researchers are not sabotaging research without detection. It may be hard to determine whether a powerful model is aligned or simply faking alignment. We instead pessimistically assume that an automated researcher is actively misleading our research. We defend against the automated researcher’s sabotage by adding a second model that monitors the automated researcher’s outputs for signs of manipulation. We then explore which strategies the saboteur could use to bypass these defenses, and how good current models are at implementing these strategies.

We use a strong, untrusted model to act as the potentially malicious automated researcher. This simulates a worst-case scenario where the alignment of a deployed state-of-the-art model is compromised despite pre-deployment testing. If we use the misaligned automated researcher to monitor and flag flawed research, it might sabotage these predictions as well, so we use weaker models as monitors. Weaker models are more trustworthy because they stem from an earlier, time-tested generation of models and because misalignment and alignment faking might only emerge with greater model capabilities. Our approach is similar to recent work on AI Control.

We study two broad classes of strategies a malicious AI system could use to sabotage research:

• In experiment sandbagging, a malicious automated researcher tries to manipulate experiments to slow down a targeted area of safety-relevant research.
• In research decision steering, the automated researcher aims to manipulate arguments and discussion to favor a certain machine learning (ML) solution. For example, it may exaggerate one interpretation of an experiment and downplay the other.

For experiment sandbagging, we let the automated researcher implement a solution to ML tasks by writing and executing Python code, similar to Claude Code. The goal of these tasks is typically to achieve a high score on a benchmark. We then let a weak monitor analyze the automated researcher’s full transcript and rate its suspiciousness. The malicious automated researcher’s secret goal is to reduce task performance while keeping the monitor’s suspiciousness low.

We largely conducted this research before the release of Claude 3.7 Sonnet. We report headline results for Claude 3.7 but otherwise discuss Claude 3.5 Sonnet (new). When we write "Claude 3.5 Sonnet" we always refer to Claude 3.5 Sonnet (new). Qualitatively, our conclusions about the two models are similar, with the key difference being that Claude 3.7 is effective at sandbagging zero-shot. We summarize our results on Claude 3.7 in the appendix.

We find:

• Unlike Claude 3.5 Sonnet, Claude 3.7 Sonnet is almost capable of zero-shot sabotage without using a scratchpad, reducing task performance to regular Claude 3.5 Sonnet with only a slight increase in monitor suspiciousness.
• One-shot prompting with an example of successful, subtle sandbagging enables models to effectively sandbag, achieving lower performance without any increase in suspiciousness. Claude 3.5 Sonnet achieves its best sandbagging performance when prompted to imitate a weaker model and given a one-shot example of effective subtle sandbagging. Claude 3.7 Sonnet also benefits from one-shot prompting, with both sabotage and weak model imitation prompts resulting in viable strategies with different tradeoffs on the Pareto frontier.
• Giving models a secret scratchpad does not seem to have a consistent effect on sabotage performance, with the effect varying between models and different sabotage strategies.
• Zero-shot prompted monitors don’t reach a high-enough true positive rate to reliably prevent one-shot sandbagging at inference time. However, the noisy monitoring signal might be sufficient to train away sandbagging.

We also share some preliminary results on steering research decisions. We consider a debate setting where a model-based judge predicts which of a pair of ML task solutions would achieve a higher benchmark score. We measure how easily the judge is swayed by manipulating free-text justifications accompanying each solution. We find:

• It is easy to steer the judge’s decision by making a justification less convincing (another form of sandbagging). Exaggerating the other justification is not effective.
• Prompted monitors struggle to detect manipulated justifications, even more so than for experiment sandbagging.
• The judge’s decisions do not improve from reading honest debates between stronger models. However, we do not train the judge, which might be necessary to see such improvements.

Overall, our results suggest that current models are likely already able to sandbag experiments without being detected by zero-shot prompted monitors. Since we only study prompting for elicitation, these results are only a lower bound on model capabilities and no-holds-barred sandbagging performance would likely be stronger. We are not yet worried about sabotage in today's models, as the sandbagging we observe today would be noticed by humans. In practice, sandbagging without detection by humans or monitors would require additional capabilities such as better situational awareness and planning abilities.

Because sandbagging might be one of the first ways AIs may reduce our ability to ensure the safety of future models, we are excited about further work tracking this capability and developing mitigations beyond the simple monitoring studied here.

Read the full blog post here.

If you’re interested in working with us on these and related questions, we are hiring Research Engineers and Research Scientists.