Team Members – Audio Deep Fake Discernment

Dr. Vandana Janeja (PI) is Professor of Information Systems and Associate Dean for Research and Faculty Development in the
College of Engineering and Information Technology at the University of Maryland Baltimore County (UMBC). She also holds the position of director of the NSF HDR Institute iHARP, which is devoted to employing data and model developments in the Polar Regions. She directs the Multi Data lab at UMBC and combines her data science research with important societal topics like climate change, data science ethics, disinformation detection, and security.

Along with Dr. Mallinson, She is leading the research team to develop novel AI algorithms and design training sessions to help listeners improve the ability to recognize audio deepfakes with varying degrees of linguistic complexity.

Dr. Mallinson (Co-PI) is Professor of Language, Literacy, and Culture, Director of the Center for Social Science Scholarship, and also leads this project.

Dr. Mallinson’s interdisciplinary and collaborative research comprehensively examines language as a socially and culturally contextualized practice in order to address linguistic bias and misinformation and expand educational equity. Her work has provided key, concise information to educators about language variation in English and has contributed to theoretical models of how language is used by students

She leads the identification of the expert defined linguistic features and embedding them in the AI algorithms and human discernment modules.

A second year PhD student at UMBC Information Systems department, Sara implemented some of the state-of-the-artAI-based audio deepfake detectors. Then, using linguistic data augmentation obtained from the linguists in the team, Sara achieved remarkable improvements in catching fake audios. Sara is currently working on automation of the linguistic data augmentation method for robust spoofed audio detection. She is also working on causal-based feature selection for the purpose of spoofed audio detection.

Noshaba is a PhD student at UMBC, Information Systems Department having research interests in data science and machine learning. Currently she is working on Deepfake project by auto annotating the fake audio files with human discernible features using machine learning algorithms.

Lavon Davis is a PhD student in the Language, Literacy, and Culture program at the University of Maryland Baltimore County and has been working professionally in higher education for 10 years. He currently serves as Graduate Research Assistant and an Adjunct Professor.

Lavon’s research interests and background are rooted in sociolinguistics and include gatekeeping/access, educational linguistics, linguistic profiling, raciolinguistics, interpersonal communication, and conversational interaction/style.

Within this project, Lavon has been working on scholarship, training, and research to deepen the perception of listeners for audio deepfakes

Nehal Naqvi is a Statistics and Political Science undergraduate student at UMBC. He is also a Sondheim and Blaustein Public Affairs Scholar and is a part of the Honors College. He is a Data Science Scholar who is assisting as a part of the project.

He is performing the statistical/data analysis of the difference in deepfake recognition between pre and post survey groups.

Kiffy (REU Summer 2023) is an incoming freshman at Rochester Institute of Technology, New York where she will be majoring in computer science. She started working at the lab as a high school intern and has transitioned to an undergraduate researcher under Dr. Janeja and Sara. As a high school intern, her work focused more on data cleaning and collection, trimming audio, sorting audio, creating and sorting paths, and visualizing data sets.

Currently, She works on the machine learning aspect of audio deep fakes. Specifically developing and testing out algorithms that visualize audio and motifs amongst real and fake audio. She is also involved in annotating and comparing audio outputs.

Chloe (REU Summer 2023) is a rising senior at Georgetown University, originally from Houston, Texas, studying linguistics with a minor in Chinese. For the summer of 2023, She joined the project as an undergraduate researcher on the linguistics side with Dr. Christine Mallinson and Lavon Davis. She listens to recordings of both fake and authentic audio to identify linguistic cues that distinguish deepfakes from real speech.

She is now developing a training program based on those cues that were identified with the aim of teaching people what to listen for when they encounter potentially fake audio online. She assists with evaluating the efficacy of this training program and further refining it and also with literature reviews and provides support to the rest of the team in writing about the project for publication in academic journals.