Mitigating Architectural Mismatch During the Evolutionary Synthesis of Deep Neural Networks

Abstract

Evolutionary deep intelligence has recently shown great promise
for producing small, powerful deep neural network models via the
organic synthesis of increasingly efficient architectures over suc-
cessive generations. Existing evolutionary synthesis processes,
however, have allowed the mating of parent networks independent
of architectural alignment, resulting in a mismatch of network struc-
tures. We present a preliminary study into the effects of architec-
tural alignment during evolutionary synthesis using a gene tagging
system. Surprisingly, the network architectures synthesized using
the gene tagging approach resulted in slower decreases in perfor-
mance accuracy and storage size; however, the resultant networks
were comparable in size and performance accuracy to the non-
gene tagging networks. Furthermore, we speculate that there is a
noticeable decrease in network variability for networks synthesized
with gene tagging, indicating that enforcing a like-with-like mating
policy potentially restricts the exploration of the search space of
possible network architectures.