Action Unit Recognition: Leveraging Weak Supervision with Large Loss Rejection

Acquiring high-quality annotated datasets is challenging and costly, leading to the advent of Weakly Supervised Learning (WSL) techniques. These methods enable the utilisation of existing noisy labels for model development, making them highly valuable. WSL techniques have demonstrated success across various domains, prompting exploring their applicability to facial Action Unit (AU) recognition. Different from the prevalence of image datasets, facial AU annotated datasets are scarce given that the annotation process is both time- and labour-intensive and requires the expertise of a subject matter expert. This study employs a Weakly Supervised Multi-Label (WSML) classification approach incorporating Large Loss Rejection (LLR) to train an AU recognition model utilising existing limited AU annotated datasets. The AU sample labels from the Extended Denver Intensity of Spontaneous Facial Action Database (DISFA+) were amended to reflect inaccurate labels. The LLR mechanism identified and rejected samples with substantial errors, preventing the model from learning from these erroneous labels. This step was crucial in ensuring the model’s accuracy, given the prevalence of introduced inaccurate labelling into the sample data. The performance of the proposed LLR model was compared against a standard AU recognition model using exact AU labels and an AU recognition model using inexact labels (i.e., trained on emotion labels and fine-tuned for AU recognition). The AU recognition model using inaccurate labels and the LLR approach exhibited promising results with a subset accuracy of 69% and a weighted average F1-score of 0.65 for AU recognition. Furthermore, cross-dataset testing on the KDEF dataset resulted in the recognition of relevant AU annotations. These findings underscore the potential of LLR-based weak supervision in addressing the data annotation challenges encountered in facial AU recognition.