TY - JOUR
AU - Shafiee, Mohammad Javad
AU - Fieguth, Paul
AU - Wong, Alexander
PY - 2016/10/03
Y2 - 2024/09/17
TI - StochasticNet in StochasticNet
JF - Journal of Computational Vision and Imaging Systems
JA - J. Comp. Vis. Imag. Sys.
VL - 2
IS - 1
SE - Articles
DO - 10.15353/vsnl.v2i1.106
UR - https://openjournals.uwaterloo.ca/index.php/vsl/article/view/106
SP -
AB - <p>Deep neural networks have been shown to outperform conventional<br />state-of-the-art approaches in several structured prediction<br />applications. While high-performance computing devices such as<br />GPUs has made developing very powerful deep neural networks<br />possible, it is not feasible to run these networks on low-cost, lowpower<br />computing devices such as embedded CPUs or even embedded<br />GPUs. As such, there has been a lot of recent interest<br />to produce efficient deep neural network architectures that can be<br />run on small computing devices. Motivated by this, the idea of<br />StochasticNets was introduced, where deep neural networks are<br />formed by leveraging random graph theory. It has been shown<br />that StochasticNet can form new networks with 2X or 3X architectural<br />efficiency while maintaining modeling accuracy. Motivated by<br />these promising results, here we investigate the idea of Stochastic-<br />Net in StochasticNet (SiS), where highly-efficient deep neural networks<br />with Network in Network (NiN) architectures are formed in<br />a stochastic manner. Such networks have an intertwining structure<br />composed of convolutional layers and micro neural networks<br />to boost the modeling accuracy. The experimental results show<br />that SiS can form deep neural networks with NiN architectures that<br />have 4X greater architectural efficiency with only a 2% drop<br />in accuracy for the CIFAR10 dataset. The results are even more<br />promising for the SVHN dataset, where SiS formed deep neural<br />networks with NiN architectures that have 11.5X greater architectural<br />efficiency with only a 1% decrease in modeling accuracy.</p>
ER -