Ramifications of Evolving Misbehaving Convolutional Neural Network Kernel and Batch Sizes

Deep-learners have many hyper-parameters including learning rate, batch size, kernel size - all playing a significant role toward estimating high quality models. Discovering useful hyper-parameter guidelines is an active area of research, though the state of the art generally uses a brute force, uni...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/ACM Machine Learning in HPC Environments (MLHPC) pp. 106 - 113
Main Authors Coletti, Mark, Lunga, Dalton, Berres, Anne, Sanyal, Jibonananda, Rose, Amy
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.11.2018
Subjects
Computer architecture
Convolution
convolutional neural networks
deep learning
evolutionary algorithms
Evolutionary computation
hyper-parameters
Kernel
optimization
satellite imagery
Satellites
settlement detection
Tuning
Online AccessGet full text

Cover

More Information
Summary:Deep-learners have many hyper-parameters including learning rate, batch size, kernel size - all playing a significant role toward estimating high quality models. Discovering useful hyper-parameter guidelines is an active area of research, though the state of the art generally uses a brute force, uniform grid approach or random search for finding ideal settings. We share the preliminary results of using an alternative approach to deep learner hyper-parameter tuning that uses an evolutionary algorithm to improve the accuracy of a deep-learner models used in satellite imagery building footprint detection. We found that the kernel and batch size hyper-parameters surprisingly differed from sizes arrived at via a brute force uniform grid approach. These differences suggest a novel role for evolutionary algorithms in determining the number of convolution layers, as well as smaller batch sizes in improving deep-learner models.
DOI:10.1109/MLHPC.2018.8638644