Efficient Neural Network Space with Genetic Search

Abstract

We present a novel neural architecture search space and its search strategy with an evolutionary algorithm. It aims to find a set of inverted bottleneck structure blocks, which takes a low-dimensional input representation followed by a compressing layer. Primitive operation layers constitute flexible inverted bottleneck blocks and can be assembled in evolutionary operation. Because the bottleneck structure confines the search space, the proposed evolutionary search algorithm can easily find a competitive neural network despite its small population size. During the search process, we designed to evaluate a model to avoid local minimums: such implementation helped the algorithm to discard local minimums and find better models. We conducted experiments on image classification of Fashion-MNIST, and we discovered an efficiently optimized neural network achieving 6.76 for an error rate with 356K parameters. © 2020, Springer Nature Singapore Pte Ltd.