Yes, autonomous cars will radically change our environment

Arnaud de La Fortelle / Professor and Director of the Center for Robotics, Mines ParisTech PSL Research University / 2017-09-15

Some automakers such as Renault and PSA are announcing the marketing of autonomous cars with a level 4 of driving automation (the penultimate stage, according to a classification established by SAE International) for 2020 – literally the day after tomorrow. This may be too optimistic in view of the progress that still needs to be made, particularly in terms of driving contextualization. But there is little doubt that in the near future, the autonomous car will be generalized, with a major impact on the entire automotive value chain and our environment.

Read this article in Chinese | Français

Paris Innovation Review — You have been working on autonomous cars for 15 years now. What progress has been made over this period?

Arnaud de La Fortelle — Fifteen years before I started, there were already prototypes of smart cars: the first units dated back from the late 1980s. I am talking of autonomous vehicles driving at 130 km/h on French highways. Thirty years later, we are still at the same point! The greatest advances relate to computational power and sensors. The first vehicles operated with radars and cameras, vision algorithms were still in their infancy. Today, they rely on a triptych of sensors (cameras, radars and lasers), even if there are in fact four – like the three musketeers! Lasers are the real novelty. They were already available fifteen years ago, but the technology was very expensive and far less accurate. We now have very precise rotating laser sensors that exist in 2D or in 3D: for example, the Valeo Scala laser scanners. The fourth component, ultrasound, is used for low speed maneuvers (such as parking). This technology is already widely used in our current vehicles.

Thanks to these advances in sensors, machines have a much richer perception of reality. Fifteen years ago, a simple operation such as recognizing a car on a road was rather complicated. Road recognition itself was based on white stripes: with a white line to the right and a white line to the left, the machine deduced that the road was in the middle. But as soon as there were several white lines – or indeed, none at all – things became a lot more complicated. Today, deep learning algorithms allow to categorize the image provided by the camera in real time: roads, vehicles, pedestrians, etc. There is a total of approximately 20 categories. I am currently funding a thesis on this subject in China.

What are the remaining obstacles?

We are still unable to contextualize driving i.e. to identify the driving context: am I driving on a very quiet road at 2 am? Am I driving on a busy highway with fast traffic? Am I in town inside a traffic jam? All these situations may seem obvious to us, but their recognition by machines is very complicated and clearly tests the limits of artificial intelligence.

As human beings, there is one thing we do intuitively and that computers are unable to mimic: we share driving with other users. This is what I call our “social behavior.” Driving is not a solitary practice. When we get to a road intersection, a sort of game starts with the other drivers. There are rules of course, but also borderline cases. When I write rules of mathematical calculations, I have to ask myself an awful lot of questions. For example, what is the distance for priority-to-the-right? For a vehicle at half a mile, the priority never applies but at 30 feet, it does. When does the priority-to-the-right rule apply? No human reasons like this. To start with, we never measure distances, we simply don’t know how to. Hence, the mathematical formalism we develop for inter-machine behaviors is very different from the rules of inter-human cooperation.

There are many ways to improve the contextualization of driving such as the semantic classification of scenes i.e. basing the meaning of driving on the meaning of a scene. For now, it’s more about labeling with several targets than building a true semantics, but we have good hopes. Thanks to machine learning, after autonomous vehicles have driven over hundreds of millions of miles — something that will happen very soon — we will get a clearer idea of certain contexts and ultimately, provide a meaning to the driving of autonomous vehicles. Another possibility: the Internet of Things. The exchange of information between connected objects and autonomous vehicles inside cities will help better understand these different contexts and improve machine interaction.

What are the most challenging environments for autonomous cars?

Autonomous cars operate relatively well on motorways but have a harder time in town with many road users, especially pedestrians. The latter are extremely difficult to “read” in terms of predictions. In addition, how can the vehicle interact with pedestrians? Since there is no driver, the pedestrian has no one to look at, which could prove quite disorienting. It is essential to avoid the vicious circle: a vehicle doesn’t know how to communicate with pedestrians, the latter has an erratic behavior and therefore, the vehicle too… One possible solution is to program defensive trajectories that are not conservative i.e. being able to react and defend oneself against hazards without making the hypothesis that everyone does whatever they like. An autonomous vehicle must be able to avoid all accidents but it will not be able to do so in the event that someone wants to cause an accident. Mathematically, being able to protect oneself against any reaction from a driver is protecting oneself against the worst case scenario i.e. that all road users seek to cause an accident. In this case, is it even reasonable to start the car?

Another difficulty lies in the differences between countries. For example, the Swiss stop 6 feet before a pedestrian crossing. In Paris, consider yourself lucky if a car stops one foot inside the crossing… This is why autonomous cars will never be global. The social factor must be adapted to each country.

There is much talk about the moral choice that an independent car should make in the event of an accident, killing its driver vs a group of pedestrians, hitting a single pedestrian vs a group of pedestrians, etc. Are these real problems?

From my point of view, this type of moral reasoning is absurd, because the question should never be put in these terms. A judge who rules on a case of road death will blame the driver for not having controlled their car and causing an accident. I do not know of any judgment where a driver is convicted for killing three people when, in case of having steered, they would have killed only one. Today, algorithms at work in autonomous cars are simply unable to make this kind of choice. They try to avoid accidents, nothing more. In the event of an accident, if driverless car knowingly decided to kill a person rather than hitting a group of people, I am sure that premeditation would be held against it! There will be no piece of code saying “I will kill.” As far as I know, Tesla has been involved in two fatal accidents but never inserted such a piece of code.

I must say that I was very surprised by these accidents. Everyone expected that the first accidents involving autonomous vehicles would dent the chassis – but would never prove mortal. This is both surprising and disturbing. My hypothesis is the following: maybe we have gone so far in avoiding crashes that the only ones remaining are the fatal ones? If so, this could prove very dangerous because autonomous vehicles may never get a chance to learn the difference between normal driving and risky driving if they never have the opportunity to experience near miss situations. This being said, we need to keep things in perspective: the first autonomous cars are much safer than traditional vehicles. They represent a real leap forward in terms of security.

Will it be possible to eliminate accidents completely?

It certainly is our goal, but even if we make enormous progress, there will still be deaths on the roads. According to the Tesla autopilot and driver assistance statistics, the number of accidents will certainly be halved quickly and reduced even more on the long term.

Some manufacturers are talking about selling autonomous cars as early as 2020. Is this realistic?

I could show you a 2005 press release from General Motors announcing automatic driving on freeways by 2008! Nobody agrees on the date. Some car manufacturers, German ones especially, do not believe in a widespread implementation before 2025-2030. Learning algorithms still need to make progress even if this can be achieved very quickly: we have already gone from tens to hundreds of test vehicles in the last years and these numbers will increase to approximately one thousand in 2017-2018. My personal prediction is therefore around 2030 for level 5 autonomy. I am talking of complete autonomy i.e. putting your child alone in a vehicle and sending him off to school, to their grandmother’s house… a vehicle without steering wheel or pedals that will drive them everywhere.

It should be noted that there are already many automated vehicles all over the world. In Rouen, for example, the TEOR bus has been partially automated for ten years: the driver releases the steering wheel upon arrival at the bus stop for docking. This works so well that drivers don’t want to go back to the old version. For drivers, maneuvering an articulated bus of 65 feet and parking it at one inch of the sidewalk was a true challenge.

Will the generalization of the autonomous car be socially accepted?

An overwhelming majority of people accept very well the idea of leaving driving to robots, especially if it is safer. Cities full of autonomous vehicles would be much safer for children, for example. From this point of view, there should be no problem in terms of social acceptance. However, it could prove more difficult at other levels. Some people, like taxi drivers, will lose their jobs. There will be collisions caused by autonomous cars. When a driver causes an accident, one can search for an explanation: they were tired, they didn’t see, etc. With an autonomous car, we are left with the plain reality: a vehicle crushed a child. It is brutal and much more difficult to accept.

A massive deployment of autonomous driving involves profound changes, for example, in terms of legislation or insurance…

Legislation is already changing. An autonomous car is a machine: it will therefore be subject to the same laws as any other machine. In the event of an accident, eyes will turn to the person responsible, as with any other machine. This is already what we do in the event of an elevator accident, for example.

The question of insurance is double-edged: this risk needs to be covered but the cost will probably be divided by ten in the medium term. I have spoken with insurers and this is a serious problem for them because vehicle premiums account for 70% of their turnover. 90% of that part of their turnover could simply disappear… The autonomous car will not only impact the world of insurance but all trades related to the automotive sector. In the future, there will probably be fleets of vehicles rather than single cars purchased by private individuals. Purchase and maintenance channels will also need to change. Insurances will work in B-to-B and no longer in B-to-C. I think we still cannot fathom the radical changes for our entire environment.

Is cybercrime a real risk for autonomous cars?

In the latest installment of Fast and Furious, there is a scene where autonomous vehicles are hijacked. It certainly is a fun idea but it also seems completely surrealistic. Critical systems such as transports will never be fully connected unless they are not completely out of the reach of hackers. Network functions will be prohibited, as they makes them too vulnerable. This does not mean that there will not be any cybercrime. But hacking will become increasingly sophisticated. Hacking a smart city will be considered a true act of war.

Your laboratory is also working on urban logistics. What consequences with autonomous cars?

One of the main factors for the improvement of urban logistics is pooling, both between actors and especially, for each actor. For example, a 35 tons truck that delivers pallets is a good solution, from a logistical standpoint. But since cities do really like 35 tons trucks driving on their streets, they chose to fragment their logistics. This is a serious miscalculation in terms of pollution and traffic because it takes 35 light commercial vehicles to deliver the same amount… The other factor of fragmentation depends on customers, who wish to receive their packages one by one. Today, the trend is to optimize revenue and not logistics: if it is more profitable for Amazon to deliver three parcels the same day to the same person, they will do so. For this reason, among others, the automation of e-commerce logistics flows will take some time: cooperation between competing players is far from being natural! Other urban logistics flows, such as garbage collection, will probably be automated much earlier.