Paper Feed: July 2025

AI Security

• Guillotine: Hypervisors for Isolating Malicious AIs
  James Mickens, Sarah Radway, Ravi Netravali (2025)
• Confidential Inference Systems: Design principles and security risks
  Anthropic (2025)

Evals / Benchmarking

• Beyond benchmark scores: Analyzing o3-mini's mathematical reasoning
  Anson Ho, Jean-Stanislas Denain, Elliot Glazer (2025)
• The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements
  Bingchen Zhao, [...], Jakob Foerster, Yoram Bachrach (2025)
• Soft Prompts for Evaluation: Measuring Conditional Distance of Capabilities
  Ross Nordby (2025)

Science of DL / Interpretability

• MAGIC: Near-Optimal Data Attribution for Deep Learning
  Andrew Ilyas, Logan Engstrom (2025)
• Hidden Breakthroughs in Language Model Training
  Sara Kangaslahti, Elan Rosenfeld, Naomi Saphra (2025)
• Stochastic Parameter Decomposition
  Lucius Bushnaq, Dan Braun, Lee Sharkey (2025)
• Dense SAE Latents Are Features, Not Bugs
  Xiaoqing Sun, [...], Mrinmaya Sachan, Max Tegmark (2025)
• The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions
  Gül Sena Altıntaş, Devin Kwok, Colin Raffel, David Rolnick (2025)

AI Safety

• Distillation Robustifies Unlearning
  Bruce W. Lee, [...], Alex Cloud, Alexander Matt Turner (2025)
  
  Why this is notable
  See also AI safety techniques leveraging distillation by Ryan Greenblatt.
• Cost-Effective Constitutional Classifiers via Representation Re-use
  Hoagy Cunningham, Alwin Peng, [...], Mrinank Sharma (2025)
• Two proposed projects on abstract analogies for scheming
  Julian Stastny (2025)
• Jailbreak Strength and Model Similarity Predict Transferability
  Rico Angell, Jannik Brinkmann, He He (2025)

Scaling / Systems

• Basic facts about GPUs
  Damek Davis (2025)
• LLMs now accept longer inputs, and the best models can use them more effectively
  Greg Burnham, Tom Adamczewski (2025)
• How Long Contexts Fail
  Drew Breunig (2025)
• Expertise
  David Autor, Neil Thompson (2025)
• Slow corporations as an intuition pump for AI R&D automation
  Ryan Greenblatt, Eli Lifland (2025)