Publication
E-DNAS: Differentiable neural architecture search for embedded systems
Conference Article
Conference
International Conference on Pattern Recognition (ICPR)
Edition
25th
Pages
4704-4711
Doc link
https://doi.org/10.1109/ICPR48806.2021.9412130
File
Abstract
Designing optimal and light weight networks to fit in resource-limited platforms like mobiles, DSPs or GPUs is a challenging problem with a wide range of interesting applications, e.g. in embedded systems for autonomous driving. While most approaches are based on manual hyperparameter tuning, there exist a new line of research, the so-called NAS (Neural Architecture Search) methods, that aim to optimize several metrics during the design process, including memory requirements of the network, number of FLOPs, number of MACs (Multiply-ACcumulate operations) or inference latency. However, while NAS methods have shown very promising results, they are still significantly time and cost consuming. In this work we introduce E-DNAS, a differentiable architecture search method, which improves the efficiency of NAS methods in designing light-weight networks for the task of image classification. Concretely, E-DNAS computes, in a differentiable manner, the optimal size of a number of meta-kernels that capture patterns of the input data at different resolutions. We also leverage on the additive property of convolution operations to merge several kernels with different compatible sizes into a single one, reducing thus the number of operations and the time required to estimate the optimal configuration. We evaluate our approach on several datasets to perform classification. We report results in terms of the SoC (System on Chips) metric, typically used in the Texas Instruments TDA2x families for autonomous driving applications. The results show that our approach allows designing low latency architectures significantly faster than state-of-the-art.
Categories
computer vision.
Author keywords
Deep Learning, Neural Architecture Search, Convolutional Meta Kernels.
Scientific reference
J. Garcia, A. Agudo and F. Moreno-Noguer. E-DNAS: Differentiable neural architecture search for embedded systems, 25th International Conference on Pattern Recognition, 2021, Milan, Italy (Virtual), pp. 4704-4711.
Follow us!