Computer spots behavior patterns

European researchers, coordinated by the Computer Vision Center (CVC) of Universitat Autònoma de Barcelona (UAB), have developed HERMES, a cognitive computational system claimed to recognize and predict human behavior. Applications for the system could include intelligent surveillance and accident prevention.

The Engineer reports that HERMES (Human Expressive Graphic Representation of Motion and their Evaluation in Sequences) analyzes human behavior based on video sequences captured at three different focus levels: the individual as a relatively distant object; the individual’s body at medium length so as to be able to analyze body postures; and the individual’s face, which allows a detailed study of facial expressions.

The information obtained is processed by computer vision and artificial intelligence algorithms, which permits the system to learn and recognize movement patterns.

HERMES is said to offer two important innovations in the field of computer vision. The first is the description of, in natural language, movement captured by the cameras, through simple and precise phrases that appear on the computer screen in real time, together with the frame number in which the action is taking place. The system uses an avatar to talk and describes this information in different languages.

The second innovation is the possibility to analyze and discover potentially unusual behavior — based on the movements it recognizes — and give off warning signals. For example, HERMES sends a signal to the control center of an underground station after capturing the image of someone trying to cross the tracks, or alerts a medical center if an elderly person living alone falls over.

The HERMES project, carried out as part of the 6th European Research Framework Program, was coordinated by Juan José Villanueva, emeritus professor of the Department of Computer Science at UAB. Project collaborators included researchers from Oxford University’s Active Vision Laboratory.