TR#479: Probabilistic Parsing in Action Recognition

Yuri A. Ivanov and Aaron F. Bobick

This paper describes a probabilistic syntactic approach to the detection and recognition of temporally extended activities and interactions between multiple agents. The paper develops a mechanism for stochastically parsing parallel input streams within a single Stochastic Context-Free Grammar (SCFG) parser and for enforcing multi-class interleaved consistency. A complete system consisting of an adaptive tracker, an event generator, and the parser performs segmentation and labeling of a surveillance video of a parking lot; the system correctly identifies activities such as pick-up and drop-off, which involve person-vehicle interactions.