Paper Feed: October 2025

AI Safety & Alignment

• Building and evaluating alignment auditing agents
  Trenton Bricken, Rowan Wang, Sam Bowman, Euan Ong, Johannes Treutlein, Jeff Wu, Evan Hubinger, Samuel Marks (July 24, 2025)
  
  Why this is notable
  See also: https://alignment.anthropic.com/2025/openai-findings/
• AI Safety Claims Analysis
  Zach Stein-Perlman (2025)
• Eliciting Harmful Capabilities by Fine-Tuning on Safeguarded Outputs
  Anonymous (under submission, 2025)
• Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
  Usman Anwar, Tim Bakker, Dana Kianfar, Cristina Pinneri, Christos Louizos (2025)
• Narrow finetuning is different
  Alex Cloud, Stewy Slocum (2025)
• Towards a Typology of Strange LLM Chains-of-Thought
  1a3orn (2025)
• Prospects for studying actual schemers
  Ryan Greenblatt, Julian Stastny (2025)
• Rogue Internal Deployments via External APIs
  Fabien Roger, Buck Shlegeris (2025)

Evals / Benchmarking

• Evaluating the Unseen Capabilities: How Many Theorems Do LLMs Know?
  Xiang Li, Jiayi Xin, Qi Long, Weijie J. Su (2025)
• Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Tasks
  Angelica Chen, Samuel D. Stanton, [...], Nathan C. Frey (2024)

Science of DL / Interpretability

• EvoLM: In Search of Lost Language Model Training Dynamics
  Zhenting Qi, Fan Nie, [...], Hanlin Zhang (2025)
  
• Reinforcement Learning Finetunes Small Subnetworks in Large Language Models
  Sagnik Mukherjee, Lifan Yuan, Dilek Hakkani-Tur, Hao Peng (2025)
• DataRater: Meta-Learned Dataset Curation
  Dan A. Calian, Gregory Farquhar, [...], David Silver (2025)
• Learning to Interpret Weight Differences in Language Models
  Avichal Goel, Yoon Kim, Nir Shavit, Tony T. Wang (2025)
• Resolved: Will Sparse Autoencoders be successfully used on a downstream task in the next year and beat baselines?
  Neel Nanda (2025)
  
  Why this is notable
  See the Manifold Market for details.
• Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs
  Ziqian Zhong, Aditi Raghunathan (2025)
• Fresh in memory: Training-order recency is linearly encoded in language model activations
  Dmitrii Krasheninnikov, Richard E. Turner, David Krueger (2025)

Scaling / Systems / Reasoning

• A taxonomy for next-generation reasoning models
  Nathan Lambert (2025)
• Barbarians at the Gate: How AI is Upending Systems Research
  Audrey Cheng, Shu Liu, [...], Ion Stoica (2025)
• Agents Built From Alloys
  Albert Ziegler (July 17, 2025)

Technical AI Governance

• Technical Options for Flexible Hardware-Enabled Guarantees
  James Petrie, Onni Aarne (2025)
• Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment
  Mauricio Baker, Gabriel Kulp, Oliver Marks, Miles Brundage, Lennart Heim (2025)
• Focus transparency on risk reports, not safety cases
  Ryan Greenblatt (2025)

Miscellaneous