Addressing Pitfalls in Auditing Practices of Automatic Speech Recognition Technologies: A Case Study of People with Aphasia

A recent study highlights significant pitfalls in the auditing practices of Automatic Speech Recognition (ASR) technologies, particularly affecting individuals with aphasia. The research identifies three main issues: reliance on a single text standardization method, failure to account for performance disparities among intersectional subgroups, and the exclusive use of Word Error Rate as a performance metric. These oversights can lead to inadequate support for marginalized communities who depend on ASR systems for communication.

This development is crucial as it underscores the need for more inclusive and equitable auditing practices in ASR technologies. By addressing these pitfalls, stakeholders can improve the reliability and accessibility of ASR systems for individuals with speech disorders, ensuring that their unique needs are met and that they receive fair treatment in technology use.

The findings resonate with ongoing discussions about fairness and bias in ASR evaluations, emphasizing the importance of diverse benchmarking practices. The study aligns with broader concerns regarding the impact of demographic factors on ASR performance, advocating for a shift towards more comprehensive metrics that reflect the varied experiences of users. This highlights a critical need for the ASR field to evolve in its approach to inclusivity and user experience.