Abstract
English pronunciation remains a major challenge for Indonesian learners, largely due to phonetic differences and limited exposure to native-like speech. To address this issue, this study proposes an interactive learning system that integrates automatic speech recognition (ASR) technology to assess and support learners’ pronunciation skills. This study introduces a speech recognition-based learning technology algorithm to support pronunciation recognition for English. This study used the Transformer-based Wav2Vec 2.0 model to recognize speech and convert into narrative text and conduct a reading assessment automatically. The fine-tuned Wav2Vec2 model demonstrated excellent ASR performance with a final Word Error Rate (WER) of 0.042778 (approximately 4.28%) on validation data, demonstrating high accuracy in speech transcription. The model also demonstrated a significant and consistent reduction in training loss and validation loss across 24,000 training steps, indicating the model consistently shows reductions in both training and validation loss, reflecting effective learning, model stability, and strong generalization capability across unseen data. Furthermore, a gamification mechanism, including learning progress tracking and a feature to "unlock" stories based on a minimum score of 80%, was successfully implemented, effectively motivating users to practice. The accuracy across real-world user interactions in the system indicates the sensitivity to audio quality and user pronunciation. Overall, the system achieves its goal of providing an interactive and innovative learning tool for English pronunciation utilizing natural language processing technology.