Protein Language Models

The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models
Pretraining filtering is one of the most common safeguards for open-weight language models. Here we study its effectiveness, durability …
Aaron Maiwald, Roderick McNeill, Jaeyoung Lee, Matthew A. Clarke, Lennart Justen, Alejandro Acelas, Ivo Andrews, Giles Harper-Donnelly, Adam Moore, Sophie Rose, Oliver M. Crook, Felix Jackson
PDF DOI
The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models