TY - JOUR AU - Li, Fan AU - Neverova, Natalia AU - Wolf, Christian AU - Taylor, Graham PY - 2016/10/03 Y2 - 2024/03/29 TI - Modout: Learning to Fuse Modalities via Stochastic Regularization JF - Journal of Computational Vision and Imaging Systems JA - J. Comp. Vis. Imag. Sys. VL - 2 IS - 1 SE - Articles DO - 10.15353/vsnl.v2i1.103 UR - https://openjournals.uwaterloo.ca/index.php/vsl/article/view/103 SP - AB - <p>Model selection methods based on stochastic regularization such<br />as Dropout have been widely used in deep learning due to their<br />simplicity and effectiveness. The standard Dropout method treats<br />all units, visible or hidden, in the same way, thus ignoring any a priori<br />information related to grouping or structure. Such structure is<br />present in multi-modal learning applications, where subsets of units<br />may correspond to individual modalities. In this abstract we describe<br />Modout, a model selection method based on stochastic regularization,<br />which is particularly useful in the multi-modal setting.<br />Different from previous methods, it is capable of learning whether<br />or when to fuse two modalities in a layer. Evaluation of Modout<br />on the Montalbano gesture recognition dataset demonstrates improved<br />performance compared to other stochastic regularization<br />methods, and is on par with a state-of-the-art carefully designed<br />fusion architecture.</p> ER -