Face expression recognition
We developed basic techniques for motion analysis of human facial
deformations. A system that simultaneously tracks the image motion
of a human face and measures the nonrigid motion of face regions
such as the eyes, eyebrows and mouth was developed The system was
applied initially to the problem of recognition of the so-called six universal
facial expressions, originally identified by Darwin and more recently investigated
systematically by the psychologists Ekman and Friesen .
The approach was applied to an extensive database of both laboratory
image sequences and ``real'' video clips digitized from movies and television
talk shows. The rules employed for recognizing facial expressions
were drawn from the psychology literature.
Gaze recognition and face expression generation
We developed a fast, robust, and low cost pupil detection technique
that uses two infra red (IR) time multiplexed light sources, synchronized
with the camera frame rate . One light source is placed very close
to the camera's optical axis, and the second source is placed off-axis.
The pupil appears bright in the camera image during on-axis illumination
(similar to the red eye effect from flash photography), and dark
when illumination is off-axis. Our experiments using a real-time implementation
of the system show that this technique is very robust, and able to
detect pupils using wide field of view low cost cameras under different
illumination conditions, even for people with glasses. We combine
this pupil detector with scene background models to enhance the robustness.
Additionally we use motion information to detect when new people
appear in the scene allowing for a shift of attention between people.