Capturing the human action semantics using a query-by-example

The paper describes a method for extracting hu man action semantics in video’s using queries-by-exam ple. Here we consider the indexing and the matching proble ms of content-based human motion data retrieval. The query formulation is based on trajectories that ma y be easily built or extracted by following relevant points on a video, by a novice user too. The so real ized trajectories contain high value of action semant ics. The semantic schema is built by splitting a trajecto ry in time ordered sub-sequences that contain the feat ures of extracted points. This kind of semantic representat ion allows reducing the search space dimensionality and, being human-oriented, allows a selective recogni tion of actions that are very similar among them. A neural network system analyzes the video semantic sim ilarity, using a two-layer architecture of multilayer perceptrons, which is able to learn the semantic schem a of the actions and to recognize them.