The Accuracy of ELSA Speak’s Speech Analyzer in assessing students’ speaking performance

Abstract

The study examined whether the Speech Analyzer function of ELSA Speak can provide accurate scores and feedback to help students improve their speaking skills. The research questions focused on the accuracy of ELSA Speak in identifying segmental errors, assessing suprasegmental features, and providing feedback to help students improve their speaking skills.

The research utilized a mixed-method design, collecting quantitative data to validate the scores and qualitative data to present external raters' evaluations. Five university students completed a speaking test using ELSA Speak, and their performances were assessed by both the Speech Analyzer and external raters. The analysis compared the scores and feedback provided by the machine and the external raters to evaluate the accuracy of ELSA Speak.

The results show that ELSA Speak's Speech Analyzer assesses speaking performance at an adequately accurate level; however, human raters offer valuable insights in areas that the machine overlooks. The machine demonstrates accuracy in identifying pronunciation and intonation errors but detects more grammatical mistakes than are necessary. It also provides relatively accurate feedback on lexical knowledge but lacks context sensitivity. While the Speech Analyzer offers comprehensive and immediate feedback which helps students understand their areas of improvement, the machine sometimes overcorrects and fails to evaluate test takers’ content delivery skills. The study acknowledges limitations, including a small sample size, the need for improved training for external raters.

In conclusion, while ELSA Speak's Speech Analyzer can provide detailed feedback, it requires collaboration with human raters to ensure appropriate feedback. Teachers should play a role in guiding students and providing explanations to maximize the efficacy of automatic speech recognition technology use in education.

Keywords

Computer-assisted language learning, corrective feedback, speech analyzer, automated speech recognition