Automated Failure Detection in Computer Vision Systems

Abstract

Human validation of computer vision systems increase their operating
costs and limits their scale. Automated failure detection can
mitigate these constraints and is thus of great importance to the
computer vision industry. Here, we apply a deep neural network
to detect computer vision failures on vehicle detection tasks. The
proposed model is a convolution neural network that estimates the
output quality of a vehicle detector. We train the network to learn
to estimate a pixel-level F1 score between the vehicle detector and
human annotated data. The model generalizes well to testing data,
providing a mechanism for identifying detection failures.