Code-Switching and Health Literacy: A Multilingual ASR-Powered Learning Platform

Authors: Alexa Rivera, Renu Balyan, Nawal Sarwar, Dillon Alex

SUNY Campus: SUNY Old Westbury

Presentation Type: Poster

Location: Old Union Hall

Presentation #: 22

Timeslot: Session C 1:45-2:45 PM

Abstract: Breast cancer awareness and education are critical for early detection and effective treatment. However, low-literacy Hispanic populations encounter significant barriers in accessing reliable health information, particularly due to linguistic and literacy challenges. Existing educational platforms predominantly cater to monolingual English or Spanish speakers, often neglecting bilingual individuals who engage in code-switching, a linguistic phenomenon where multilingual speakers fluidly alternate between different languages within a conversation. To bridge this gap, we propose to build a web-based intelligent tutoring system (ITS) that leverages multilingual Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) technologies within an interactive video learning environment. It will integrate Deepgram Nova-2 ASR API, transcribing code-switched speech. ASR technology enables real-time processing of spoken responses, transcribing user input in English, Spanish, or a combination of both, ensuring seamless linguistic accommodation. Complementing this, NLP algorithms analyze users' responses, evaluate comprehension, and provide adaptive feedback, fostering an engaging and personalized learning experience. The platform, developed on WordPress, will feature educational videos with integrated quizzes in both English and Spanish, enabling users to switch between languages effortlessly. This ITS enhances accessibility, mitigates literacy barriers, and empowers individuals with culturally and linguistically tailored breast cancer education, ultimately improving awareness and health outcomes.