Research Spotlight Archive
Title: Video Anomaly Detection: From Surveillance to YouTube
Funding: National Science Foundation, National Geospatial Agency
Background: Video camera networks have proliferated in the U.S. and abroad, appearing everywhere from airports to border crossings to city streets. Today more than 30 million surveillance cameras produce nearly 4 billion hours of video footage each week, and this continuous stream of data exceeds the processing capacity of human analysts. Although automatic algorithms searching the data for suspicious activity exist, they are often inadequate, especially in busy, urban areas. This is further compounded by algorithm complexity that requires significant computing power, usually available at a central server. An additional consequence of this is network congestion as all video streams need to be transmitted for processing.
Description: The simplest form of anomaly detection is background subtraction during which objects that move or appear in/disappear from the camera’s field of view are deemed anomalous. Although this may be sufficient for restricted-area access control where a mere detection of movement is sufficient, it is not enough for the detection of unusual behavior. However, the background subtraction results capture the moving object dynamics while stripping the photometric content. We leverage this for an abnormal behavior detection framework that can detect abnormally high activity, departure from typical activity, etc., all within the same formulation. By aggregating motion labels for a given pixel from all frames of a training sequence, we build a low-dimensionality representation against which we test an aggregate computed from the observed sequence. If the observed aggregate either exceeds or departs from the training aggregate, abnormality is declared. This simple strategy leads to remarkably robust results.
Results: The developed algorithm, called behavior subtraction, has been tested on a wide variety of video sequences, from highly-cluttered traffic scenes captured by vibrating cameras (see image above) to notoriously-difficult boats-on-a-river scenes where specular light reflections off the water surface are difficult to account for. Not only is the algorithm effective at detecting anomalous dynamics in the scene, but it is also efficient in that it requires relatively low-power computing hardware and its memory footprint is minimal compared to state-of-the-art algorithms. Both these characteristics are very desirable for the so-called edge implementation, i.e., implementation in the camera instead of a central server.
Publications: P. M. Jodoin, J. Konrad, and V. Saligrama, “Modeling background activity for behavior subtraction,” in ACM/IEEE International Conference on Distributed Smart Cameras, September 2008.
P. M. Jodoin, J. Konrad, V. Saligrama, and V. Veilleux-Gaboury, “Motion detection with an unstable camera,” in Proceedings of the IEEE International Conference on Image Processing, October 2008.
P. M. Jodoin, V. Saligrama, and J. Konrad, “Behavior subtraction,” in Proc. SPIE Visual Communications and Image Process., vol. 6822, pp. 10.1-10.12, Jan. 2008.
J. McHugh, J. Konrad, V. Saligrama, and P. M. Jodoin, “Foreground-adaptive background subtraction,” IEEE Signal Processing Letters, vol. 16, pp. 390-393, May 2009.
V. Saligrama, J. Konrad, and P. M. Jodoin, “Video anomaly identification: A statistical approach,”IEEE Signal Processing Magazine, vol. 27, pp. 18-33, September 2010.