In the days since the release of my colleague Lex Fridman’s interview of Elon Musk, attention has focused on Musk’s comment that Teslas with Full Self-Driving capability may become an appreciating asset. Given cars are (except in the case of special relics) depreciating assets, this is more likely than not closer to science fiction if Tesla continues increasing production.
Throughout the interview, Musk appears focused on the capabilities empowered by Tesla’s new “Full Self-Driving” feature, a three-year-old quest that integrates a new in-vehicle computer, fleet-level data learning, and large scale neural network training. While Tesla is developing a technology that may take a major step towards self-driving, experts (including myself) do not expect that any system produced and marketed for consumers in the next few years will be truly self-driving, and others feel strongly that a primarily vision-based system like Tesla is pursuing for self-driving will have limitations. What many do expect is an evolution of features that utilize drivers as a backup to the automation in situations requiring intervention.
The quest and associated publicity may only serve to accelerate consumer confusion around what is self-driving and expand a documented history of deceptive marketing. If the system requires any element of human supervision or oversight, it is simply not self-driving, but rather assisted driving. Nonetheless, Tesla’s technology prowess in system architecture and data capture (near real-time targeted capture of images from a large fleet, etc.) has perhaps accomplished what others have not yet found feasible and could revolutionize edge case discovery.
One can bemoan regulatory hurdles for slowing progress, but there is a public expectation that technology, sensors and algorithms have been proven to ensure a level of safety. As an earlier post discusses, we have yet to clearly define what success looks like in our race to autonomy. Musk himself states, “it might need to be like two or three hundred percent safer than a person.” He also discusses that the incidents needed to prove safety are rare, requiring a lot of data.
What makes things even more difficult is that it’s easy to count crashes, but it is hard to count near misses and incidences avoided (nothing happened), making the statistical interpretations of where and when a system provides advantages over humans a complex science. Counting incidents per mile as a simple aggregate ignores aspects of risk such as vehicle type, model year, owner profiles, driving location and a host of other factors. A further complication is that a system may be technically off at the moment of an event, having disengaged seconds or even a minute earlier, but still be functionally linked to its use (e.g., the now “driver” did not have the full awareness needed to start driving).
A well-validated estimate of risk associated with Tesla’s Autopilot, now a three and a half-year-old assisted driving system, remains elusive. Although the system has been used for over one-billion miles, the telemetry behind whether Autopilot is enabled or disabled during fender benders, when the car unexpectedly leaves the lane of travel or traverses an intersection without stopping (all indicators of risk), are largely publicly unknown with only gross estimates of safety published by Tesla. Perhaps the data needed to aid our understanding is buried somewhere in the vehicles or the connected Tesla ecosystem.
I am certain that there are situations in which Autopilot may be safer than a human driver; humans do have a long history of falling asleep at the wheel. But, we also know from multiple crashes and fatalities that technology is not perfect. In the context of an undertreated health crisis on our roads, data transparency is required – as innocent until proven guilty, with consumers used as test subjects, is not an acceptable standard. Automakers have traditionally relentlessly invested in proving the functional safety of systems prior to release, but with connected vehicles and over-the-air or dealer-based updates, the standard for proving safety needs to evolve in order to provide clearer granularity of the risks and benefits of each and every software update, to ensure drivers (and regulators) know what they are consenting to.
Turning back to the topic of drivers supervising automation, camera-based driver monitoring has been a growing area of global focus. When asked about this topic, Mr. Musk responded, “if you have a system that’s at or below human level reliability then driver monitoring makes sense, but if your system is dramatically better, more reliable than a human, then driver monitoring does not help much.” Tesla might argue, as detailed in its vehicle safety report, that Autopilot improves safety; however, other data seems to suggest differently. If Tesla’s Autopilot feature has not been clearly established to be dramatically better than a human, should one ask, how was Tesla allowed to release (and perhaps continue to sell) Autopilot without a camera-based driver monitor?
Many experts feel that camera-based driver monitoring is essential in both manual and automated vehicles; however, monitoring may only be the foundation. In an early article, colleagues and I explored the need for Monitoring, Managing and Motivating Driver Safety and Well-being. Driver attention management and the need to develop ways of motivating driver behavior (e.g., Cadillac CT6 with Super Cruise as one example) remain critical elements in the pursuit of safety that effectively combines the capabilities of humans and machines.
