TR#508: Parametric Hidden Markov Models for Gesture Recognition

Andrew D. Wilson and Aaron F. Bobick

Appears in IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(9), September 1999.

A new method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the 2-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually-derived and directly measured 3-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques as well as greater robustness in parameter estimation with respect to noise in the input features. Lastly, we extend the PHMM to handle arbitrary smooth (non-linear) dependencies. The non-linear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction.

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by the authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

Postscript . PDF . Full list of tech reports