Simple Printed Signs Can Trick a Self-Driving Car Into Driving Directly Into Pedestrians: Study
A simple printed sign can hijack a self-driving car and make it drive directly toward pedestrians, a new study shows. This is very concerning news considering the recent track record of autonomous Waymo and Tesla robotaxis.
Researchers found that the AI powering self-driving cars can mistakenly read text on billboards or posters as directives that take precedence over the system’s internal safety protocols. They also carried out successful hijacking attacks in Chinese, Spanish, and even Spanglish, an English-Spanish hybrid, as well as in varying lighting conditions.
This simple hack could have dire consequences
This means attackers may be able to easily affect an autonomous system’s behavior using this hack. Autonomous robots and drones, and other AI systems that use cameras may also be vulnerable to these types of attacks.
“Every new technology brings new vulnerabilities,” said one of the authors of the study, Alvaro Cardenas, a cybersecurity expert at the Baskin School of Engineering at UC Santa Cruz. “Our role as researchers is to anticipate how these systems can fail or be misused – and to design defenses before those weaknesses are exploited.”
Large visual-language models (LVLMs), a class of AI algorithm that can handle both text and visual input, are increasingly powering embodied AI systems. These models assist the robots in navigating the unpredictable situations that arise in the real world.
“I expect vision-language models to play a major role in future embodied AI systems,” Cardenas said. “Robots designed to interact naturally with people will rely on them, and as these systems move into real-world deployment, security has to be a core consideration.”
Researchers call the hack ‘command hijacking against embodied AI’
The researchers call their sets of attacks CHAI: command hijacking against embodied AI. When testing CHAI, they found that it achieves up to 95.5% attack success rates for aerial object tracking, 81.8% success on driverless cars, and 68.1% success on drone landing. They successfully tricked the AI into making dangerous choices in each situation, such as landing in an improper location or colliding with another car.
“We found that we can actually create an attack that works in the physical world, so it could be a real threat to embodied AI,” Burbano said. “We need new defenses against these attacks.”
