Assessing the behavior of machine learning methods to predict the activity of antimicrobial peptides

• Published in: Revista FI-UPTC Vol. 26; no. 44; pp. 167 - 180
• Main Authors: Camacho, Francy Liliana, Torres-Sáez, Rodrigo, Ramos-Pollán, Raúl
• Format: Journal Article
• Language: English; Portuguese
• Published: Universidad Pedagógica y Tecnológica de Colombia 01.01.2017
• Subjects: antimicrobial peptides; Computer and Systems Engineering; ENGINEERING, MULTIDISCIPLINARY; learning curves; machine learning; statistical stability; support vector regression; Curvas de aprendizagem; Estabilidade estatística; Regressão de Vetores de Suporte; machine learning; statistical stability; curvas de aprendizaje; estabilidad estadística; support vector regression; learning curves; regresión de vectores de soporte; Aprendizagem de máquina; aprendizaje de máquina; Peptídeos antimicrobianos; péptidos antimicrobianos; antimicrobial peptides
• ISSN: 0121-1129; 2357-5328
• DOI: 10.19053/01211129.v26.n44.2017.5834

This study demonstrates the importance of obtaining statistically stable results when using machine learning methods to predict the activity of antimicrobial peptides, due to the cost and complexity of the chemical processes involved in cases where datasets are particularly small (less than a few hundred instances). Like in other fields with similar problems, this results in large variability in the performance of predictive models, hindering any attempt to transfer them to lab practice. Rather than targeting good peak performance obtained from very particular experimental setups, as reported in related literature, we focused on characterizing the behavior of the machine learning methods, as a preliminary step to obtain reproducible results across experimental setups, and, ultimately, good performance. We propose a methodology that integrates feature learning (autoencoders) and selection methods (genetic algorithms) thorough the exhaustive use of performance metrics (permutation tests and bootstrapping), which provide stronger statistical evidence to support investment decisions with the lab resources at hand. We show evidence for the usefulness of 1) the extensive use of computational resources, and 2) adopting a wider range of metrics than those reported in the literature to assess method performance. This approach allowed us to guide our quest for finding suitable machine learning methods, and to obtain results comparable to those in the literature with strong statistical stability.