Authors:
Yuan An、Samarth Kolanupaka、Jacob An、Matthew Ma、Unnat Chhatwal、Alex Kalinowski、Michelle Rogers、Brian Smith
Paper:
https://arxiv.org/abs/2408.10492
Introduction
Engaging students and ensuring the retention of knowledge are critical aspects of effective teaching. However, many lectures fail to achieve these goals. Despite extensive research in cognitive science and neuroscience suggesting effective teaching strategies, their application in real classrooms remains limited. The advent of artificial intelligence (AI) offers a promising avenue to bridge this gap. This paper introduces a novel knowledge graph-supported intelligent lecturing assistant (ILA) system designed to help teachers enhance student learning by integrating insights from cognitive science, neuroscience, and established pedagogical best practices. By leveraging AI, this system aims to empower instructors and create more interactive and engaging learning environments, ultimately contributing to improved student learning outcomes.
Related Work
Knowledge graphs (KGs) represent knowledge in a domain using a graphical structure consisting of vertices and links. They have gained significant attention for their ability to represent complex information and enhance various AI applications. In educational contexts, KGs facilitate the organization and retrieval of educational content, enhance personalized learning, and support intelligent tutoring systems. They enable the development of adaptive learning systems that can tailor content to individual learners’ needs.
Intelligent educational systems (IES) and intelligent tutoring systems (ITS) have leveraged KGs to address several key aspects of learning, including exercise recommendation, student interactions diagnosis, personalized learning guidance, and automatic grading. The integration of voice sentiment analysis in educational technology builds upon foundational research in affective computing and speech processing. Previous studies have demonstrated that emotional cues in speech can significantly impact communication efficacy and audience engagement. In educational settings, a teacher’s emotional expressiveness has been linked to student motivation, comprehension, and retention of information.
Research Methodology
Constructing Knowledge Graph for Intelligent Lecturing System
The knowledge graph, which represents the course content, schedule, objectives, and assessment, is the foundation of the proposed intelligent lecturing assistant. The key requirements for the construction and representation of this knowledge graph include:
- Accurate Representation: The KG must accurately capture the detailed content, schedule, objectives, and assessment materials of the course.
- Automatic Construction: The KG should be automatically constructed for a wide range of topics across various educational contexts.
- Support for Effective Teaching: The KG should support the generation of relevant quizzes and questions to engage students at optimal times.
- Support for Improving Learning: The KG should enable reasoning with pedagogical strategies to enhance teaching effectiveness and support student learning.
An ontology is developed to provide a structured framework and common understanding of the domain. The lecture ontology encompasses primary concepts such as course, course module, schedule, and assessment, along with their subclasses, including section, lecture, topics, assignment, quiz, test, and project.
Lecture Voice Sentiment Analysis
The problem of lecture voice sentiment analysis is to classify whether the lecture speech in a certain period is engaging or not engaging. The model is trained using a set of 1-minute voice clips, allowing for more focused and relevant acoustic feature extraction. The final classification of the entire lecture speech is determined based on whether the average score exceeds a threshold of 0.5.
Experimental Design
Building Data Sets for Lecture Voice Sentiment Analysis
Training Data
Given the absence of an existing dataset specifically tailored for lecture voice sentiment analysis, a new dataset was created from scratch. Researchers manually labeled lecture videos as ‘engaging’ or ‘boring’ based on specific criteria. The voice from labeled lectures was segmented into 1-minute clips, and each clip was independently reviewed and labeled. Additionally, researchers recorded their own voice clips to enhance the dataset. A total of 3,025 lecture voice clips were collected.
Independent Validation Data
To ensure the model’s generalizability, a separate set of validation data was collected, independent of the training set. This independent validation set contains 804 voice clips.
Extracting Features from Raw Voice Signals
Meaningful features were extracted from the raw audio signals using the librosa Python package. These features include Zero Crossing Rate (ZCR), Chroma STFT, Mel Spectrogram, Mel Frequency Cepstral Coefficients (MFCC), Root-Mean-Square (RMS) Value, Chroma CQT, Chroma CENS, Chroma VQT, Spectral Centroid, Spectral Bandwidth, Spectral Contrast, Spectral Flatness, and Spectral Rolloff.
Building Models for Lecture Voice Sentiment Analysis
Various classification models were evaluated, including logistic regression, random forest, XGBoost, fully connected neural networks, and convolutional neural networks. Feature combinations were exhaustively evaluated to identify the most informative subset for classification.
Results and Analysis
Experimental Results
The logistic regression model with a regularization parameter of C = 10 outperformed others across all evaluation metrics. The optimal feature set for this model includes zcrate_mean, chroma_vqt_mean, spcent_mean, and spband_mean. The results indicate that spectral features are particularly effective for distinguishing between engaging and non-engaging lecture voices.
Discussion
The current study is limited by its focus on a binary classification of engagement, which may not fully capture all aspects of lecturer delivery. Future research could benefit from expanding the dataset and incorporating more complex features to enhance the model’s accuracy and robustness. The knowledge graph functions as the “brain” of the intelligent lecturing assistant system, offering the contextual information and reasoning capabilities necessary to act on insights derived from sentiment analysis.
Overall Conclusion
The study explores the development of an intelligent lecturing assistant (ILA) system that utilizes a knowledge graph to represent course content and pedagogical strategies. The system is designed to support instructors in enhancing student engagement by analyzing lecture voice sentiment. The results demonstrate promising performance with an F1-score of 90% for boring lectures on an independent set of test voice clips. This research lays the groundwork for developing additional components of the ILA system, which integrates knowledge representation, reasoning, speech recognition, machine learning, and intelligent intervention. Future research will address ethical considerations related to the use of AI in education and evaluate the system’s effectiveness in real classrooms.
Sources and Data Sets
For reproducibility, the source notebooks utilized for data analysis and model evaluation are accessible in the public GitHub repository: GitHub Repository. The training data and the independent validation sets can be made available for download upon request.
