<img src="https://spectrum.ieee.org/media-library/side-view-of-a-suv-wrapped-with-blue-green-labeling-with-white-lettering-on-the-front-door-panel-that-reads-fraunhofer-idmt.jpg?id=61640995&width=1200&height=800&coordinates=0%2C250%2C0%2C250"/> Autonomous vehicles have eyes—cameras, lidar, radar. But ears? That’s what researchers at Fraunhofer Institute for Digital Media Technology’s <a href="https://www.idmt.fraunhofer.de/en/hsa.html" target="_blank">Oldenburg Branch for Hearing, Speech and Audio Technology</a> in Germany are building with the <a href="https://www.idmt.fraunhofer.de/en/institute/projects-products/projects/the_hearing_car.html" rel="noopener noreferrer" target="_blank">Hearing Car</a>. The idea is to outfit vehicles with external microphones and AI to detect, localize, and classify environmental sounds, with the goal of helping cars react to hazards they can’t see. For now, that means approaching emergency vehicles—and eventually pedestrians, a punctured tire, or failing brakes.“It’s about giving the car another sense, so it can understand the acoustic world around it,” says <a href="https://de.linkedin.com/in/moritz-brandes-861751162" rel="noopener noreferrer" target="_blank">Moritz Brandes</a>, a project manager for the Hearing Car.<h2></h2>In March 2025, Fraunhofer IDMT researchers drove a prototype Hearing Car 1,500 kilometers from Oldenburg to a proving ground in northern Sweden. Brandes says the trip tested the system in dirt, snow, slush, road salt, and freezing temperatures.<h2>How to Build a Car That Listens</h2>The team had a few key questions to answer: What if the microphone housings get dirty or frosted over? How does that affect localization and classification? Testing showed performance degraded less than expected once modules were cleaned and dried. The team also confirmed the microphones can survive a car wash.Each external microphone module (EMM) contains three microphones in a 15-centimeter-wide package. Mounted at the rear of the car—where wind noise is lowest—they capture sound, digitize it, convert it into spectrograms, and pass it to a <a href="https://spectrum.ieee.org/tag/convolutional-neural-networks" target="_blank">region-based convolutional neural network </a>(RCNN) trained for audio event detection.If the RCNN classifies an audio signal as a siren, the result is cross-checked with the vehicle’s cameras: Is there a blue flashing light in view? Combining “senses” like this boosts the vehicle’s reliability by lowering the odds of false positives. Audio signals are localized through <a href="https://spectrum.ieee.org/tag/beamforming" target="_self">beamforming</a>, though Fraunhofer declined to provide specifics on the technique.All processing happens onboard to minimize latency. That also “eliminates concerns about what would happen in an area with poor Internet connectivity or a lot of interference from [radio-frequency] noise,” Brandes says. The workload, he adds, can be handled by a modern <a href="https://spectrum.ieee.org/tag/raspberry-pi" target="_self">Raspberry Pi</a>.According to Brandes, early benchmarks for the Hearing Car system include detecting sirens up to 400 meters away in quiet, low-speed conditions. That figure, he says, shrinks to under 100 meters at highway speeds due to wind and road noise. Alerts are triggered in about 2 seconds—enough time for drivers or autonomous systems to react. <img alt="tablet-size display indicates weather conditions ( sunny, rain, or snow), road wetness level (dry, slightly moist, and wet), and whether a siren has been detected." class="rm-shortcode" data-rm-shortcode-id="ec849c7102db48a1805ca20e0bc61926" data-rm-shortcode-name="rebelmouse-image" id="f645b" loading="lazy" src="https://spectrum.ieee.org/media-library/tablet-size-display-indicates-weather-conditions-sunny-rain-or-snow-road-wetness-level-dry-slightly-moist-and-wet-and.jpg?id=61641000&width=980"/> This display doubles as a control panel and dashboard letting the driver activate the vehicle’s “hearing.”Fraunhofer IDMT<h2>The History of Listening Cars</h2>The Hearing Car’s roots stretch back more than a decade. “We’ve been working on making cars hear since 2014,” says Brandes. Early experiments were modest: detecting a nail in a tire by its rhythmic tapping on the pavement or opening the trunk via voice command.A few years later, support from a tier 1 supplier (a company that provides complete systems or major components such as transmissions, braking systems, batteries, or advanced driver assistance systems (ADASs) directly to automobile manufacturers) pushed the work into automotive-grade development, soon joined by a major automaker. With EV adoption rising, automakers began to see why ears mattered as much as eyes.“A human hears a siren and reacts—even before seeing where the sound is coming from. An autonomous vehicle has to do the same if it’s going to coexist with us safely.” —Eoin King, University of Galway Sound LabBrandes recalls one telling moment: Sitting on a test track, inside an electric vehicle that was well <a href="https://b-quiet.com/blogs/resources/sound-deadening-solutions-for-electric-vehicles#:~:text=Acoustic%20Foam%20Panels,cabin%20optimally%20quiet%20and%20comfortable." target="_blank">insulated against road noise</a>, he failed to hear an emergency siren until the vehicle was nearly upon him. “That was a big ‘ah-ha!’ moment that showed how important the Hearing Car would become as EV adoption increased,” he says.<h2></h2><a href="https://ie.linkedin.com/in/eoin-king-b4b29627" target="_blank">Eoin King</a>, a mechanical engineering professor at the <a href="https://www.universityofgalway.ie/" target="_blank">University of Galway</a> in Ireland, sees the leap from physics to AI as transformative.“My team took a very physics-based approach,” he says, recalling his <a href="https://www.sciencedirect.com/science/article/abs/pii/S0003682X20307787" target="_blank">2020 work in this research area</a> at the <a href="https://www.hartford.edu/" target="_blank">University of Hartford</a> in Connecticut. “We looked at direction of arrival—measuring delays between microphones to triangulate where a sound is. That demonstrated feasibility. But today, <a href="https://spectrum.ieee.org/tag/artificial-intelligence" target="_self">AI</a> can take this much further. Machine listening is really the game changer.”Physics still matters, King adds: “It’s almost like physics-informed AI. The traditional approaches show what’s possible. Now, machine learning systems can generalize far better across environments.”<h2>The Future of Audio in Autonomous Vehicles</h2><h2></h2>Despite progress, King, who directs the <a href="https://www.thegalwaysoundlab.com/" target="_blank">Galway Sound Lab</a>’s research in acoustics, noise, and vibration, is cautious.“In five years, I see it being niche,” he says. “It takes time for technologies to become standard. <a href="https://mycardoeswhat.org/deeper-learning/lane-departure-warning/" target="_blank">Lane-departure warnings</a> were niche once too—but now they’re everywhere. Hearing technology will get there, but step by step.” Near-term deployment will likely appear in premium vehicles or autonomous fleets, with mass adoption further off.King doesn’t mince words about why audio perception matters: Autonomous vehicles must coexist with humans. “A human hears a siren and reacts—even before seeing where the sound is coming from. An autonomous vehicle has to do the same if it’s going to coexist with us safely,” he says.King’s vision is vehicles with multisensory awareness—cameras and lidar for sight, microphones for hearing, perhaps even vibration sensors for <a href="https://www.niradynamics.com/road-maintenance?utm_term=road%20condition%20monitoring&utm_campaign=USA&utm_source=adwords&utm_medium=ppc&hsa_acc=1326993342&hsa_cam=21940756935&hsa_grp=171650616432&hsa_ad=722940006874&hsa_src=g&hsa_tgt=kwd-2391777516395&hsa_kw=road%20condition%20monitoring&hsa_mt=b&hsa_net=adwords&hsa_ver=3&gad_source=1&gad_campaignid=21940756935&gbraid=0AAAAAC3cv-svwc_gVe6OoVq47bjuoRLeo&gclid=Cj0KCQjw_rPGBhCbARIsABjq9ceF6bhsDuu1huOEAjghom--D-YlESja2vGa2-YcisEymIZiYRx0BKUaAmgfEALw_wcB" target="_blank">road-surface monitoring</a>. “Smell,” he jokes, “might be a step too far.”<h2></h2>Fraunhofer’s Swedish road test showed that durability is not a big hurdle. King points to another area of concern: false alarms.“If you train a car to stop when it hears someone yelling ‘help,’ what happens when kids do it as a prank?” he asks. “We have to test these systems thoroughly before putting them on the road. This isn’t consumer electronics, where, <a href="https://spectrum.ieee.org/chatgpt-reliability" target="_self">if ChatGPT gives you the wrong answer</a>, you can just rephrase the question—people’s lives are at stake.”Cost is less of an issue: microphones are cheap and rugged. The real challenge is ensuring algorithms can make sense of noisy city soundscapes filled with horns, garbage trucks, and construction.Fraunhofer is now refining algorithms with broader datasets, including sirens from the United States, Germany, and Denmark. Meanwhile, King’s lab is improving sound detection in indoor contexts, which could be repurposed for cars.Some scenarios—like a Hearing Car detecting a red-light-runner’s engine revving before it’s visible—may be many years away, but King insists the principle holds: “With the right data, in theory it’s possible. The challenge is getting that data and training for it.”Both Brandes and King agree no single sense is enough. Cameras, radar, lidar—and now microphones—must work together. “Autonomous vehicles that rely only on vision are limited to line of sight,” King says. “Adding acoustics adds another degree of safety.”

طور الباحثون في معهد فراunhofer 'سيارة سمعية' مبتكرة تكشف الأصوات لتعزيز السلامة في المركبات الذاتية القيادة. تهدف هذه التكنولوجيا الرائدة إلى منح السيارات ذاتية القيادة القدرة على 'سماع' محيطها، تمامًا كما يفعل البشر، مما قد يقلل بشكل كبير من الحوادث ويحسن السلامة على الطرق بشكل عام. مع تزايد انتشار المركبات الذاتية القيادة، قد يكون دمج الوعي السمعي نقطة تحول، مما يجعل طرقنا أكثر أمانًا للجميع.

Investigadores del Instituto Fraunhofer han desarrollado un innovador 'Coche Auditivo' que detecta sonidos para mejorar la seguridad en vehículos autónomos. Esta tecnología busca otorgar a los coches autónomos la capacidad de 'escuchar' su entorno, al igual que los humanos, lo que podría reducir significativamente los accidentes y mejorar la seguridad vial en general. A medida que los vehículos autónomos se vuelven más comunes, integrar la conciencia auditiva podría ser un cambio radical, haciendo nuestras carreteras más seguras para todos.

Des chercheurs de l'Institut Fraunhofer ont développé une 'Voiture Auditive' révolutionnaire qui détecte les sons pour améliorer la sécurité des véhicules autonomes. Cette technologie innovante vise à donner aux voitures autonomes la capacité d' 'entendre' leur environnement, tout comme les humains, ce qui pourrait réduire considérablement les accidents et améliorer la sécurité routière globale. Alors que les véhicules autonomes deviennent de plus en plus courants, l'intégration de la conscience auditive pourrait changer la donne, rendant nos routes plus sûres pour tous.

Researchers at the Fraunhofer Institute have developed a groundbreaking 'Hearing Car' that detects sounds to enhance safety in autonomous vehicles. This innovative technology aims to give self-driving cars the ability to 'hear' their surroundings, much like humans do, which could significantly reduce accidents and improve overall road safety. As autonomous vehicles become more prevalent, integrating auditory awareness could be a game-changer, making our roads safer for everyone.