Estimating Student Learning Affect Using Facial Emotions ∗

The current COVID-19 pandemic has seen a lot of higher institutions of learning embracing the e-learning systems. Although these e-learning systems promise to deliver solutions to teaching and learning in this pandemic era, a key challenge is motivating the learner to engage with the e-learning system continuously. Most e-learners quickly get bored and lose motivation in the course of learning. While there exist many strategies such as chatrooms and sporadic question and answer sessions to keep learners involved in e-learning platforms, they have always achieved minimal connectedness among e-learners. Facial emotions have been identified as an effective tool for interpreting learning experience in learners. This study, therefore, examines the use of facial emotions expressed by learners to interpret their learning affect in an e-learning session. This work also explores a standardized mapping mechanism between facial emotions exhibited and their respective learning affects. The study identifies the physical changes in the face of a learner and uses it to estimate their facial emotions and then based on the mapping mechanism, maps emotional states to a student's learning affect. Experiments include the use of a convolutional neural network for the classification of seven facial emotions. The research study tests different network architectures to find optimal architecture, using the FER2013 dataset. Results from the mapping are statistically analyzed and compared with responses provided by participants who participated in the live testing of the system. Results show that facial emotions, which are a form of non-verbal communication, can be used to estimate the learning affect of a student and provides a new avenue to enhance the current e-learning platforms.