Description

This project presents an efficient deep-learning method that lets the robot IVO recognise continuous gestures performed in front of it in real-life situations.

Still gestures recognition is tested with shallow networks to evaluate the project availability and then a temporal component is added to design the proposed method, achieving the recognition of more natural and intuitive gesture classes.

First, MediaPipe framework extracts features from the images in order to find the landmarks for each of the hand joints. Then, some feature representation techniques are tested to avoid scalability or translation problems. Next, the key frames are selected to input their data into a neural network. Different network configurations are tested to find the optimal hyperparameters.

State-of-the-art experiments are performed to refine the designed method characteristics and to have it ready to work on real situations. A batch approach is combined with the best model of the designed method to execute real-life experiments with sequences of continuous gestures including non-gesture instances, actions without meaning that are performed in intuitive human to human communication like nose scratching.

The obtained results overperform state-of-the-art outcomes regarding its accuracy. Using the batch approach with the proposed method recognises gestures executed in front of the robot IVO and runs at 10 frames per second with an unoptimized code.