In a bid to prove that its robot drivers are safer than humans, Waymo simulated dozens of real-world fatal crashes that took place in Arizona over nearly a decade. The Google spinoff discovered that replacing either vehicle in a two-car crash with its robot-guided minivans would nearly eliminate all deaths, according to data it publicized today.
The results are meant to bolster Waymo’s case that autonomous vehicles operate more safely than human-driven ones. With millions of people dying in auto crashes globally every year, AV operators are increasingly leaning on this safety case to spur regulators to pass legislation allowing more fully autonomous vehicles on the road.
But that case has been difficult to prove out, thanks to the very limited number of autonomous vehicles operating on public roads today. To provide more statistical support for its argument, Waymo has turned to counterfactuals, or “what if?” scenarios, meant to showcase how its robot vehicles would react in real-world situations.
Last year, the company published 6.1 million miles of driving data in 2019 and 2020, including 18 crashes and 29 near-miss collisions. In those incidents where its safety operators took control of the vehicle to avoid a crash, Waymo’s engineers simulated what would have happened had the driver not disengaged the vehicle’s self-driving system to generate a counterfactual. The company has also made some of its data available to academic researchers.
That work in counterfactuals continues in this most recent data release. Through a third party, Waymo collected information on every fatal crash that took place in Chandler, Arizona, a suburban community outside Phoenix, between 2008 and 2017. Focusing just on the crashes that took place within its operational design domain, or the approximately 100-square-mile area in which the company permits its cars to drive, Waymo identified 72 crashes to reconstruct in simulation in order to determine how its autonomous system would respond in similar situations.
The results show that Waymo’s autonomous vehicles would have “avoided or mitigated” 88 out of 91 total simulations, said Trent Victor, director of safety research and best practices at Waymo. Moreover, for the crashes that were mitigated, Waymo’s vehicles would have reduced the likelihood of serious injury by a factor of 1.3 to 15 times, Victor said.
OK, it’s a good idea, but surely they could have modelled Waymo response on hundreds of thousands of crash scenarios instead of this very tightly controlled tiny subset?