Training Students to Listen Better: Advancing Deepfake Audio Discernment
Pilot 2 spring 2023, Fall 2023
We Are currently running additional Pilot studies across Spring 2023 and Fall 2023 in a large multi section study.
We study the impact of training students to listen for distinguishing phonetic and phonological features – which we call Expert Defined Linguistic Features (EDLFs), as they were selected based on the knowledge of sociolinguistics experts – as a means of improving their ability to discern real versus fake English audio.
Pilot 1 Fall 2022
The study focuses on identifying audio deepfakes, which can spread deception and misinformation. The team has combined insights from sociolinguistics and machine learning to enhance detection methods. A pilot training program was conducted in fall 2022, where graduate research assistants trained four undergraduate UMBC students to improve their ability to discern audio deepfakes. Pre- and post-evaluations showed that the students’ skills in identifying authentic audio clips improved. The students then shared their knowledge in an undergraduate Data Science course, emphasizing the linguistic aspects of audio deepfakes.
Sociolinguistics experts (Mallinson and Davis) listened to more than 200 real and fake audio files. Their objective was to identify human-perceivable audio cues that can assist in distinguishing fake audios. Through their analysis of phonetic and phonological characteristics in the audio, they identified five specific features. These features are referred to as Expert Defined Linguistic Features (EDLFs). The sociolinguistics experts discovered that any deviation or abnormality observed in these five EDLFs serves as a reliable indication of a fake audio. These EDLF features also improved the performance of AI algorithms for deepfake detection.
The study highlights the importance of expanding training to increase awareness and detection abilities among college students, given their exposure to social media and the risk of deception and misinformation online.
Preliminary findings indicate that the training process appears to effectively develop, enhance, and deepen students’ linguistic understanding and their ability to discern audio deepfakes. Our ongoing work aims to further develop tools and trainings to guide listeners to more accurately discern audio deepfakes. As of March 2023, we have begun expanding our training by launching a pilot study with 40 undergraduate students and will complete the analysis of the pilot data by Summer 2023.
Provost Poster Board Project-Final
