3

The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models
Pretraining filtering is one of the most common safeguards for open-weight language models. Here we study its effectiveness, durability …
Aaron Maiwald, Roderick McNeill, Jaeyoung Lee, Matthew A. Clarke, Lennart Justen, Alejandro Acelas, Ivo Andrews, Giles Harper-Donnelly, Adam Moore, Sophie Rose, Oliver M. Crook, Felix Jackson
PDF DOI
The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models
Selective Generalization: Improving Capabilities While Maintaining Alignment
Training to improve capabilities may cause undesired changes in model behavior. For example, training models on oversight protocols or …
Ariana Azarbal, Matthew A. Clarke, Jorio Cocolla, Cailley Factor, Alex Cloud
PDF Code Dataset Project
Selective Generalization: Improving Capabilities While Maintaining Alignment
MAGELLAN: Automated Generation of Interpretable Computational Models for Biological Reasoning
Computational models have become essential tools for understanding signalling networks and their non-linear dynamics. However, these …
Matthew A. Clarke, Charlie George Barker, Yuxin Sun, Theodoros I. Roumeliotis, Jyoti S. Choudhary, Jasmin Fisher
PDF Code Project DOI
MAGELLAN: Automated Generation of Interpretable Computational Models for Biological Reasoning
Compositionality and Ambiguity: Latent Co-occurrence and Interpretable Subspaces
Sparse AutoEncoder (SAE) latents show promise as a method for extracting interpretable features from large language models (LM), but …
Matthew A. Clarke, Hardik Bhatnagar, Joseph Bloom
PDF Code Dataset Project Video
Compositionality and Ambiguity: Latent Co-occurrence and Interpretable Subspaces