Estimating Optimal Depth of VGG Net with Tree-Structured Parzen Estimators

Abstract

Deep convolutional neural networks (CNNs) have shown astonishing
performances in variety of fields. However, different architectures
of the networks are required for different datasets, and finding
right architecture for given data has been a topic of great interest in
computer vision communities. One of the most important factors of
the CNNs architecture is the depth of the networks, which plays a
significant role in avoiding over-fitting. Grid Search is widely used
for estimating the depth, but it requires huge computation time. Motivated
by this, a method for finding an optimal architecture depth is
introduced, which is based on a hyper-parameter optimizer called
Tree-Structured Parzen Estimators (TPE). In this work, we show
that the TPE is capable of estimating the CNNs architecture depth
with an accuracy of 83.33% with CIFAR-10 dataset and 60.00%
with CIFAR-100 dataset while it reduces the computation time by
more 70% compared to the Grid Search.