Davis Brown

See our blog summary of part of this paper, where we find widespread cheating in agent evaluations.

We design an agent, "Meerkat," that performs quite well on a number of asynchronous monitoring settings, ie monitoring after an agent has already executed some actions. Meerkat identifies misuse campaigns, reward hacking, code sabotage, and developer cheating on real agent evals more effectively than standard monitors or naive agents.

In general, an attacker attempting LLM misuse doesn't compute jailbreaks. They instead control the context the model can see by splitting up a harmful task into small, benign-seeming tasks across independent queries-- this is called a decomposition attack. To better understand cyber and biosecurity misuse threats, we build an automated misuse evaluation pipeline, Benchmarks for Stateful Defenses (BSD). We find that decomposition attacks are as strong as the best jailbreaks, but much harder to catch. We argue that stateful defenses, which store context from previous user sessions, are needed to subvert decomposition attacks.

Project website

We build datasets and tasks desgined to evaluate a model's ability to generate new research-level algebraic combinatorics conjectures. Many of the problems we select are currently open and pose strong challenges to both frontier models and mathematicians using models-as-tools. For each problem, we provide a large amount of associated data and train narrow models to both serve as baselines / objects of study for interpretability.