Stochastic Receptive Fields in Deep Convolutional Networks

Abstract

Deep convolutional neural networks (ConvNets) have rapidly grown
in popularity due to their powerful capabilities in representing and
modelling the high-level abstraction of complex data. However,
ConvNets require an abundance of data to adequately train network
parameters. To tackle this problem, we introduce the concept
of stochastic receptive fields, where the receptive fields are
stochastic realizations of a random field that obey a learned distribution.
We study the efficacy of incorporating layers of stochastic
receptive fields to a ConvNet to boost performance without the
need for additional training data. Preliminary results showing an
improvement in accuracy ( 2% drop in test error) was achieved by
adding a layer of stochastic receptive fields to a ConvNet compared
to adding a layer of fully-trained receptive fields, when training with
a small training set consisting of 20% of the STL-10 dataset.