Enlarge /. Joel Johnson with a Waymo driverless vehicle.
Waymo has long kept details about its industry leading self-driving technology under wraps. The company has tested millions of miles in Arizona and California – including thousands of driverless miles where no one was behind the wheel. But until last month, almost everyone who witnessed these driverless trips was bound by a strict nondisclosure agreement.
Waymo finally pulled back the curtain on its driverless technology in October. Today customers can call a completely driverless taxi near the Chandler suburb of Phoenix. They can record rides, post videos and talk to reporters about their experiences.
A young Arizona citizen in particular took the opportunity to document the real-world performance of Waymo's driverless taxis. Joel Johnson is an Arizona State University student on hiatus from college during the pandemic. He lives near Waymo's service area and has used some of his free time to put Waymo's driverless taxis through their paces. He says he has made more than 60 driverless trips in the two months since Waymo opened driverless service to the public. He's posted more than a dozen videos.
The most striking thing about these videos is how boring they are. In almost five hours of driving, I haven't seen Waymo's vehicles make a single significant mistake. This is in contrast to the "fully self-driving" software that Tesla released in beta back in October. I spent three hours watching videos from customers testing Tesla technology. The drivers intervened more than a dozen times – including two cases where a crash was imminent.
Johnson's experience wasn't like that at all. "It was absolutely solid," he told Ars in a telephone interview.
"You forget that nobody drives."
Enlarge /. Johnson without a mask on another type of vehicle that drives itself.
The Waymo One service still has a waiting list, so Johnson has offered rides for a number of others who don't yet have access. In addition to friends and family, Johnson has hosted industry insiders and YouTubers who have taken special trips to the Phoenix area to see Waymo cars in action.
"Everyone I've taken on private trips trusts it," said Johnson. "They forget that nobody drives but the computer because the experience is so smooth."
"I love the way you brake and accelerate – you don't even notice it," said one of Johnson's companions during a ride. "It's going to be really very smooth."
Johnson has been driving Waymo vehicles since mid-2019 when he joined Waymo's closed early rider program. He says he has seen significant progress.
"They really ironed out things like unprotected leftists," Johnson said in a video. "It has definitely improved over time."
"That was great," one passenger said to Johnson at the end of a ride. "It's getting smarter. That was a lot better than March."
Johnson also said Waymo's vehicles can handle pedestrians better.
In an October video, a Waymo car drove through a Costco parking lot crowded with pedestrians. It waited patiently for them to get out of the way and then moved forward confidently.
"That number of pedestrians would have resulted in whiplash in March 2020," Johnson wrote in an on-screen note. "And it would have given up completely in July 2019. Not anymore!"
The vehicles are still a little too cautious with pedestrians. In a recent video, Johnson called a Waymo vehicle into a crowded retail parking lot, hit "Start Ride", and then had to wait nearly 3 minutes for the vehicle to move a significant distance. There were apparently so many pedestrians and other vehicles nearby that the Waymo car did not feel safe driving forward.
A human driver would almost certainly have moved sooner. But it's hard to blame Waymo – much better to be a little slow than take the risk of running someone over.
Proving security is not easy
Of course, four hours of perfect driving – or 40 or 400 hours – wouldn't be enough to prove that Waymo's cars are safe. A lot of data is required to properly assess the safety of Waymos vehicles. And Waymo has more than 32 million kilometers of real driving data. Almost all of the mileage is covered on public roads with a safety driver behind the wheel. A small portion – 105,000 km (65,000 miles) by September 2020 – was completely driverless.
Until recently, Waymo kept this data secret, making it difficult for the public to evaluate the technology. In October, Waymo took a giant step towards greater transparency by releasing data on the actual performance of its vehicles. It comprised 10 million km (6.1 million miles) that the company covered with a safety driver at the wheel in the greater Phoenix area in 2019 – plus 105,000 km (65,000 miles) of driverless operation from early 2019 to September 2020.
Waymo's vehicles were involved in 18 accidents in six million miles. Of course, for most of those 6 million miles, the vehicles had safety drivers to intervene in the event of an impending accident. To gauge how well the cars would have done without a safety driver, Waymo ran simulations for each situation where a safety driver took control. These simulations predicted that 29 more accidents would have occurred had the safety drivers not intervened.
While 47 crashes seem like a lot, it's important to remember the denominator. Waymo's vehicles have been involved in an accident – or likely would have been in an accident without human intervention – approximately every 210,000 km. That's equivalent to more than 10 years of driving for a typical person who drives 1,000 miles a month.
It is surprisingly difficult to figure out what the comparable rate would be for a typical human driver. Some of the 47 collisions reported by Waymo were extremely minor. For example, a pedestrian stepped into the side of a stationary Waymo vehicle at 4.3 km / h. In two simulated accidents, a bicycle and a skateboarder rolled into the sides of stationary Waymo vehicles at speeds of 2.2 and 5.9 mph, respectively.
Such minor, low-speed collisions would never be reported to the police or other authorities, so we do not know how many "accidents" like this one would go to a typical human driver.
More importantly, most of these 47 incidents were the fault of another driver. For example, a third of the real and simulated crashes were incidents. All but one – 14 actual collisions and one simulated accident – involved a different vehicle, leaving a Waymo car behind. The final tail was a simulated accident in which the Waymo car would have left another vehicle behind at a speed of 1 km / h.
In the majority of side-wiping accidents (eight out of ten), the other vehicle changed lanes into the lane of the Waymo vehicle. In one of the instances where the Waymo changed lanes, Waymo said the other car was 30 mph over the speed limit.
Over a six million mile drive, there were only three collisions (and five simulated accidents) severe enough to cause an airbag to deploy. Waymo says none of these resulted (or could have caused) serious or life-threatening injuries.
To sum up, Waymo had a low accident rate, had no life threatening accidents and most of the accidents were the fault of the other driver when driving more than four million miles. These results make it plausible that Waymo's vehicles are safer than the average human driver in the vast majority of situations.
Waymo's extreme gradualism
Enlarge /. A backrest display informs customers about the status of the journey.
However, there is still one big open question: Is Waymo's driver less likely to cause a fatal crash. The difficulty with this is that the US only has one freeway fatality for every 100 million miles driven. Even six million miles of near-flawless driving does not remotely prove that Waymo's driver is less likely to kill someone than a human driver.
There's a tough chicken-and-egg problem here, as even a company with Waymo's deep pockets probably can't afford to test its technology hundreds of millions of kilometers before bringing a commercial product to market. However, it is risky to bring a driverless car to market before you have proven it is safer than human.
So Waymo is approaching the problem very gradually. His cars have a top speed of 45 miles per hour. This makes sense because high-speed highway accidents are the most likely to kill someone. Waymo is rolling out its fully driverless service at a rapid pace. Two years after the company was hoping to launch a driverless commercial service, the company still only makes about 100 trips a week – about two or three full-time taxi drivers.
I would suspect that behind the scenes, an army of people is monitoring and analyzing every trip to make sure it goes smoothly. If those reviews are positive, the company is likely to increase the number of cars on the road. At some point, Waymo will be confident enough to expand into a larger area – first across the Phoenix area and then in other metropolitan areas.
It's a very different approach to Tesla, where new software updates are released with relatively little testing and customers monitor the system for errors. At least three Tesla customers in the US have died since 2016 after failing to correct bad decisions made by the autopilot software.
Waymo seems more determined to keep his near-flawless driving record. The question is whether this cautious strategy allows it to scale fast enough to be a viable business.