A Clinical Trial to Validate an Automated Online Language Interpreting Tool With Hispanic Patients Who Have Limited English Proficiency

Individuals with limited English proficiency (LEP) are medically underserved in the United States and have significant barriers to accessing healthcare and mental health services. Lack of available translation services contributes to poor healthcare access and utilization, leading to higher rates of hospital admissions, misdiagnosis, improper treatment, and poorer health comprehension and outcomes. In psychiatry, diagnosis, assessment, and treatment relies more heavily on speech and communication than in other medical specialties. LEP hampers a patient’s understanding of healthcare providers’ instructions, explanations, and history-taking questions, even with an interpreter present.

The investigators had previously created and demonstrated that the use of asynchronous telepsychiatry (ATP)—a virtual mental health visit—is a more rapid and efficient way for primary care providers to gain access to psychiatric expertise. This research expanded this work by building an automated language translation process into the ATP consultations to allow multi-language clinical evaluations to occur without human interpreters.

The specific aims were as follows:

• Build and evaluate an automated asynchronous interpretation tool.
• Compare the interview and language interpretation quality and accuracy of automated translation versus interpreted translation.
• Compare patient satisfaction of automated translation versus interpreted translation.
• Compare the diagnostic accuracy of automated translation versus interpreted translation.

Aims one and four were completed within the grant period, while aims two and three are ongoing. In the first phase, the research team iteratively refined and tested the ATP application (app). In the second phase, the team conducted a randomized crossover study, recruiting 114 Spanish-speaking patients who each underwent two psychiatric interviews. The first was a conventional in-person, virtual interview with a psychiatrist and a human interpreter. The second was an asynchronous video-recorded telepsychiatry interview that was subsequently processed with two different automated speech recognition (ASR) and machine translation (MT) systems (Google Translate and Microsoft). Findings indicate one of the systems was consistently more accurate in translation and with fewer word errors than the other MT and ASR system, but both were similar in accuracy to in-person translation accuracy rates reported in the literature. However, interpreters were slightly more accurate at figurative language translation than the automated systems. Analysis demonstrated that patient speech becomes simplified and truncated when using a human interpreter. This simplification aligns with many published guidelines detailing best practices for use of human interpreting services, which often encourage a reduction in the use of idiomatic speech, as well as a simplification of sentence structure. Within the specialty of psychiatry, diagnosis and treatment decisions are heavily reliant on the verbal history conveyed to the provider - results suggest that the history provided through use of a human interpreter will likely differ and could represent a less comprehensive picture of the patient’s psychopathology, which is important.

Ongoing analysis includes comparing the two methods on the identification of anxiety, depression, and substance abuse, as well as patient satisfaction. Early findings indicate patients prefer ATP consultations in their own language than the use of interpreters. Further studies of the accuracy of interpretation over videoconferencing compared with in-person interpreting are required.