Analysis of the first Genetic Engineering Attribution Challenge [article]

Oliver M. Crook, Kelsey Lane Warmbrod, Greg Lipstein, Christine Chung, Christopher W. Bakerlee, T. Greg McKelvey Jr., Shelly R. Holland, Jacob L. Swett, Kevin M. Esvelt, Ethan C. Alley, William J. Bradshaw
2021 arXiv   pre-print
The ability to identify the designer of engineered biological sequences -- termed genetic engineering attribution (GEA) -- would help ensure due credit for biotechnological innovation, while holding designers accountable to the communities they affect. Here, we present the results of the first Genetic Engineering Attribution Challenge, a public data-science competition to advance GEA. Top-scoring teams dramatically outperformed previous models at identifying the true lab-of-origin of engineered
more » ... sequences, including an increase in top-1 and top-10 accuracy of 10 percentage points. A simple ensemble of prizewinning models further increased performance. New metrics, designed to assess a model's ability to confidently exclude candidate labs, also showed major improvements, especially for the ensemble. Most winning teams adopted CNN-based machine-learning approaches; however, one team achieved very high accuracy with an extremely fast neural-network-free approach. Future work, including future competitions, should further explore a wide diversity of approaches for bringing GEA technology into practical use.
arXiv:2110.11242v1 fatcat:6tqm4pj6ynfjlaqu45pixwuxvm