Affects Analysis: A Temporal Approach to Estimate Students' Learning

Christine Bukola Asaju, Hima Vadapalli

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

The dissimilarity between real and virtual environments is getting closer rapidly. More people are working on computers to accomplish a variety of tasks. Tasks like shopping, banking, learning, teaching, etc. are all being carried out online. The education sector is one of the sectors that has adapted to the needs and obstacles created by the impossibility to present a face to face sessions, which was necessitated because of the global pandemic. These obstacles have led to more options for online teaching and learning. Unfortunately, the underlying experiences, feelings, emotions, and moods of students during learning are not adequately captured by teachers, unlike in the classroom environment where a teacher can observe students through visual and audio cues such as eye gaze, body language, facial expressions, speech, etc. Facial emotion expressions recognition during online learning has been a subject of research in the field of machine learning, but most have not adequately considered analyzing a student's affect states with the possibility of providing appropriate feedback. This work, therefore, intends to contribute to the existing knowledge by proposing a temporal approach to estimate a student's learning affects. A model, that comprises of a video input of facial emotion expressions, feature extraction using ResNet-50 pretrained CNN model, and a bi-directional Long Short-Term Memory classifier to detect and classify affect states of boredom, confusion, frustration, and engagement for estimating student's learning is proposed. The baseline model was trained, validated, and tested using the DAiSEE dataset and shows a test accuracy of 88% on 6546 test samples. The work further maps the affect states into positive and negative learning affects. It is anticipated that such a model will serve as feedback to the teachers and enhance the estimation of the students' learning, especially during e-learning sessions.

Original languageEnglish
Title of host publication2021 3rd International Multidisciplinary Information Technology and Engineering Conference, IMITEC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665417495
DOIs
Publication statusPublished - 2021
Externally publishedYes
Event3rd International Multidisciplinary Information Technology and Engineering Conference, IMITEC 2021 - Windhoek, Namibia
Duration: 23 Nov 202125 Nov 2021

Publication series

Name2021 3rd International Multidisciplinary Information Technology and Engineering Conference, IMITEC 2021

Conference

Conference3rd International Multidisciplinary Information Technology and Engineering Conference, IMITEC 2021
Country/TerritoryNamibia
CityWindhoek
Period23/11/2125/11/21

Keywords

  • Affect states
  • Bidirection Long Short-Term Memory
  • Convolutional Neural Network
  • Temporal data

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Information Systems
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'Affects Analysis: A Temporal Approach to Estimate Students' Learning'. Together they form a unique fingerprint.

Cite this