STeW: Real-time Video Facial Emotion Classification via a Compact Sliding Temporal Windowed Deep Neural Network

Abstract

The real-time classification of human facial expressions presents a
challenging task, even for humans. Individuals with Autism Spectrum Disorder (ASD) have an even greater difficulty in detecting
and interpreting these facial expressions, which can lead to an
increased risk of depression and loneliness due to a disconnect
with society. This study explores a compact Sliding Temporal Windowed (STeW) deep neural network architecture for real-time video
facial emotion classification. The proposed STeW architecture is
designed to provide a balance between speed and the leveraging
of temporal characteristics to capture transient nuances of facial
expressions. A more difficult dataset (which we call BigFaceX) is
proposed by combining and modifying the extended Cohn-Kanade
(CK+), BAUM-1, and the eNTERFACE public datasets, and used to
evaluate the proposed STeW network. Experimental results show
that the proposed STeW network architecture can achieve noticeably higher accuracy when compared to the highly compact mini-
Xception network, thus illustrating the potential for leveraging this
approach to achieve real-time video facial emotion classification.