Context-based Data Augmentation for Improved Ballet Pose Recognition

As computer vision technology continually advances and expands into various application fields, modern methods enable researchers to extract the most important and relevant key features from visual data. Human action recognition is one area that has gained much interest due to its potential in a variety of domains. Ballet, an art form filled with movements and poses, is a domain where the analysis of key features is particularly relevant. At a fundamental level, well-defined features used in a computer vision pipeline, generally provide more accurate predictions. This paper presents a feature engineering and pose analysis approach with OpenPose that produces pose feature templates from distance data. The calculated distance data is used to explore feature augmentation strategies for improved ballet pose recognition results. In addition to the OpenPose distance feature data, this paper investigates geometric and spatial feature approaches to determine optimal feature configurations for the ballet pose recognition task. A ballet dataset of eight distinct poses with multiple dancers was used for the study. The results demonstrate that the proposed approach provides a way to derive baseline pose skeletons from which statistical metrics are derived to aid in determining a valid ballet pose feature space. The augmentation of calculated OpenPose distance feature data yields improved results where the leading SVM classifier for the OpenPose feature space achieves an accuracy of 99.713%. The ensemble-based feature extraction approach which uses MobileNetV3 along with Augmented OpenPose features yielded an excellent Area Under the Curve (AUC) score of 99.999%. The proposed study, therefore, provides a valuable approach that results in state-of-the-art performance for the ballet pose recognition task.