Software for cars has been increasing in complexity over the past few years. The amount of code in vehicles can reach a hundred million lines, and for the new and fast-rising self-driving vehicles, it could mean even more hundreds of million lines of code. These cars are not programmed in an “if-then” computer algorithm; instead, they rely on machine learning and pattern recognition. For them to drive continuously and without human assistance, experts said it will take 10, maybe even 20 years, and the end of driving for humans is not quite in view.
Regulators’ top priority is to make sure that these cars are safe to be on the roads, which could explain the stall in certain states like California, where some companies are already pushing for self-driving cars to hit the road by 2016. According to the state’s DMV, the regulations for post-testing deployment of autonomous vehicles are still being developed, and the DMV wants to make sure the self-driving vehicles are as safe as human drivers before giving the public access to these automobiles. Automobile fatalities decreased nearly 25% since 2004, according to the National Highway Traffic Administration (NHTSA). In 2013, 32,719 people died in car crashes, and in 2012, 33,782 people died. The NHTSA reports that this is a historic low for automobile fatalities.
(Related: Even Apple is making a self-driving car)
Yet the goal is to come up with cars that drive themselves more safely than humans, which is why the Department of Transportation and NHTSA support the initiatives, according to a statement for a symposium by Mark R. Rosekind, administrator of NHTSA.
He said that the potential to overcome human driving flaws (such sleepiness, inattention or recklessness) makes the technology worth pursuing. As much as driver safety, though, the NHTSA is also concerned with cybersecurity, which he said could threaten the technology.
Concerns about safety
Since the software in self-driving cars controls important components like the steering wheel, gas pedal and brake pedal, this means they are at risk for being hacked, according to Raj Rajkumar, professor and co-director of the General Motors-Carnegie Mellon Autonomous Driving Collaborative Research Lab. There have been a few incidents already where hackers went through the WiFi, 3G and 4G connections into the car’s software, such as when two hackers made a Chrysler Jeep stop dead in the middle of a highway. Rajkumar said that companies like Tesla, GM and anyone else working on self-driving cars need to be “sensitive to security vulnerabilities.”
GM is making sure it covers potential cyber threats by using a layered approach to in-vehicle security, which means they will create a series of defenses to cover gaps in protection, possibly including intrusion-detection systems, malware scanners and other tools. They are also designing many vehicle systems, which can be updated with security measures as potential threats evolve.
“If there is a security loophole that becomes an entry point for outsiders to get into the car, remotely and wirelessly, then that poses a big problem,” said Rajkumar.
Cem Saraydar, director of the Electrical and Controls Systems Research Lab at General Motors, said the company is encouraging safety and regulation be addressed at the federal level because GM believes that legislative efforts at the state level will hurt the technology, and the government can help speed up the process. GM is participating in collaborative research to find out what safety aspects of automation systems may be important.
“If it turns out that safety regulations are required to ensure safe deployment of automation systems, then we think it would make more sense for such a safety regulation to be set by a single, federal, entity versus individual states,” said Saraydar.
According to the National Conference of State Legislatures (NCSL), 16 states have introduced legislation related to autonomous vehicles in 2015, 12 states in 2014, nine states (and D.C.) in 2013, and six states in 2012. Nevada was the first state to authorize the operation of autonomous vehicles in 2011, with California, Florida, Michigan, North Dakota and Tennessee and D.C. following.
California’s Department of Motor Vehicles made progress late last year when it issued a draft of potential regulations for putting humans behind the wheel of autonomous vehicles, such as Google’s self-driving car. Its proposal said that a licensed driver would need to take over the car if necessary, and manufacturers would have to report accidents, create security measures to keep the car’s controls safe from hackers, and tell the car’s operator how to use the vehicle safely. While this is a step in the right direction, this proposal hurt Google’s chances of getting their cars on the road in the next year.
Another concern of both regulators and developers is making sure the cars can understand how to handle an issue it otherwise hadn’t encountered before. Some ways to test this would be to create a realistic urban environment, much like Mcity, where Ford is testing their self-driving vehicles for issues that they otherwise wouldn’t be able to test safely in the real world.
The DMV suggests that an alert driver can react to a situation in 1.75 seconds. Because of this, Rajkumar said the self-driving vehicle must be capable of having the “ability to slow down and come to a safe stop,” just as a human driver would. The “vehicle must be capable of dealing with tricky conditions by itself.”
AdasWorks’ CEO Laszlo Kishonti said that the current thinking is that the cars will give back the controls to the driver if it is in a situation it hasn’t seen before. (AdasWorks is a company that develops artificial intelligence for cars.)
“If these cars become popular, then people will lose their ability to react, not just because they are not focused, but if you lose your experience with driving, then human drivers will become more and more dangerous,” he said.
Even if the industry manages to work out the kinks of safety and security, it’s another thing to get people to want to ride in self-driving cars. Since there is a younger generation growing up surrounded by new technology, it’s possible that they will be more willing to embrace this new paradigm of using cars, according to Jim McBride, head of research and innovation at Ford Motors. However, he said that, in his experience, once he has put someone who was reluctant about self-driving vehicles behind the wheel, their opinion changed.
“It’s hard for a lot of people to imagine what it’s like because those products exist for them to try,” he said. “They’ve never had that experience, so imagining what that experience is going to be, they have no paradigm in their mind.”
Rajkumar said that the general public might have an unclear understanding about the software and technology that currently exists and is ready for the road because of the “misinformation, hype and confusion.” He said that the cars on the market are not fully autonomous and that the technologies being introduced are to assist the driver that is still actively paying attention to the road. The cars that are being tested have driver-assistance software, and drivers are still expected to pay attention at all times.
“It will take many, many years for that technology to evolve and mature,” said Rajkumar. “Meanwhile, the technology has to be reliable; it’s going to take time.”
Companies agree about the different degrees of vehicle automation, but there is disagreement about what those degrees are, according to Rajkumar. The Society for Automotive Engineers (SAE) has defined six degrees of automation, from Level 0 of no automation to Level 5 of full automation. Rajkumar added that the NHTSA has defined four levels of automation.
GM uses SAE definitions of driving automation, saying that they provide a way to describe differences between low-automation and high-automation systems. GM’s Saraydar said that commercially and widely available fully autonomous vehicles are “likely a ways out because the technical challenges to handle all driving conditions as good as, or better than, a human driver is a difficult task and it will take time to develop the necessary technical solutions.”
Despite the difficulties, Saraydar said that there are demonstrations of concepts of autonomous driving capabilities that are already happening, and that “This trend will intensify across the industry.”
The exact year of when self-driving cars will be completely road ready are all over the place since automobile companies are making and changing their predications of when their cars will be on the market. Glimpsing at their predictions, anywhere from 2025 to 2040 is fair game for auto-driving cars on the market.
Current self-driving vehicles and the ones that are in the making rely on the sensors and image data to understand objects, and they include driver-assistance software, which means the vehicles are just assisting the drivers. Although it seems the industry is close to having driving control solely in the hands of the vehicle, it could be decades before the software and learning capabilities of the car are completely road ready.
What’s already on the road
While it might take years for cars to drive us to and from places without us paying attention, there is still a lot of software moving out that is getting close to it.
Tesla: Tesla Motors has been dominating the news the past few months, starting when the company rolled out Tesla Version 7.0, which allows Model S tools to deliver new active safety and convenience features, including a fully integrated autopilot system with four different feedback modules: a camera, radar, ultrasonics and GPS.
Tesla’s Autopilot allows the vehicle to steer within a lane, change lanes with the tap of a turn signal, and manage speed by using active, traffic-aware cruise control. The car can also scan for a parking space, alert when one is available, and parallel-park on command.
YouTubers and other car enthusiasts have posted videos of these Tesla updates in action, showing hands-free driving, the car changing lanes, and even the car appearing to dive into traffic when it’s supposed to stay in the lane. Despite these videos, Tesla released its autopilot as a beta. Drivers are still supposed to be actively paying attention to the road, which is probably why the new feature also comes a small warning located at the bottom of the dashboard: “Always keep your hands on the wheel. Be prepared to take over at any time.”
General Motors: Although they’ve been a little quieter than Tesla, General Motors is gearing up for its own self-driving vehicles. By the end of 2016, General Motors will have a fleet of self-driving Chevrolet Volts that will cruise the GM Technical Center out of suburban Detroit.
The fleet will operate within the roads of the campus, and the object is to gain an understanding of how the car operates, how the sensors work and how the computer controls work, according to Saraydar. The roads on the tech center are not public, but there are still vehicles on the roads. The fleet will be able to interact with other vehicles and pedestrians, and GM will be able to learn much about self-driving from this “learning laboratory” of vehicles.
Saraydar said that GM is anticipating the complexity of the software to grow in the near future. GM develops their own software for some of its embedded controllers as well as for active safety features. They will continue to do this in their own engineering groups as they move into automated driving.
Google: Google’s driverless fleet of modified Lexus SUVs are cruising around Mountain View, Calif. and Austin. According to Google’s Self Driving Car Project, its software helps classify an object, predict what the car will do next, and maneuver accordingly. While Google’s fleet is on the road, their goal is to use the cars to get feedback on how they perform and all of the real-world scenarios a car might encounter.
They’ve capped their cars at 25 mph for “safety reasons,” according to a Google blog post. But some might argue that this is too slow, such as the officer who pulled over a driverless Google car recently. Lucky for the developers, the car did not receive a ticket.
Mitsubishi: Last year, it was reported that Mitsubishi Electric is using machine-learning algorithms to analyze vehicle data and driver behavior to keep drivers alert. Katsunobu Muroi, a spokesperson for Mitsubishi, said that this is different from self-driving cars because the “goal of self-driving system is to travel without human intervention.” They expect it to be installed on driver-sensing commercial vehicles around 2019, but Muroi said they do not know when the units will be installed into cars.
Mitsubishi isn’t the only company trying to go above and beyond when it comes to autonomous vehicles. A big announcement by Toyota in November said they invested US$1 billion to establish a new company, Toyota Research Institute Inc., where the focus will be on artificial intelligence and robotics. Its headquarters are in Silicon Valley, and they plan to develop technology that will make driving safer for humans instead of replacing it.
Ford: Ford recently declared that they will be the first to test an autonomous vehicle at Mcity, which is part of the University of Michigan’s full-scale simulated urban environment that’s designed for testing these types of vehicles. “We can construct some limited number of scenarios that we prefer to do in a closed environment,” said McBride. “We can do them in a high density fashion [in Mcity].”
He said that some of the testing is done in the real world, but they can set up other possible scenarios that a car might run into in Mcity, including pedestrians jumping out into traffic, bicycles diving in front of cars, or having an oncoming car run a red light. They will use a virtual environment, complete with virtual people, which will test the cars’ ability to expect the unexpected.
A car driven by a single embedded processor
New technology comes with a price, and until these cars are able to be mass-produced, the cost is not affordable for the average person who might spend $30,000 to $40,000 on a new car. Google’s self-driving car is running at $75,000 because of the pricey lasers and radars that sit on top of the vehicle, and the 2015 Tesla Model S goes for about $70,000.
A small team of engineers wants to show the world that it is possible to create a vehicle that doesn’t cost thousands of dollars to implement software or hardware, and they too are joining the race, even if they are across the country.
Located in Budapest, Hungary, the group is called AdasWorks, a spinoff of the company Kishonti, which provides high-performance graphics, compute, automotive and computer-vision solutions. AdasWorks brings in high-performance embedded programming, computer vision, parallel computing, visualization, cloud computing and more to the development of semi-autonomous systems. Currently, they are working on an autonomous control system for an automobile, which only cost them a few hundred dollars.
AdasWorks said they created the first autonomous car driven by a single embedded processor. It uses vision-based information and a front-facing camera. It also uses a safety system developed by ThyssenKrupp Presta Hungary, a steering system company. Unlike other self-driving features on other vehicles, it does not need pre-recorded 3D laser scan maps of the road or surroundings to detect lanes and turns.
“[Other systems] are good for detecting lanes, but they don’t really understand the environment; they just detect,” said AdasWorks’ Kishonti. “They don’t make very complex decisions; they just mainly brake the car or keep the car in the lane.”
He added that they are looking to develop artificial intelligence or deep learning for the car, so that the single unified artificial intelligence detects and understands the objects around it. They want the car to be able to predict what will happen in the next few seconds. An example he used was how a human driver would react differently when they see a regular pedestrian on the sidewalk, versus a drunk pedestrian. They want the car to be able to react in those same ways, he said.
Driving toward the future
As the systems become more intelligent, automobile companies and software companies are going to need more individuals with the skills and knowledge to take this innovation above and beyond the road.
This advancement isn’t one that will take away jobs from individuals, and according to GM’s Saraydar, it will open up endless possibilities for software developers, IT experts and other software-savvy individuals.
As software capabilities continue to increase, there will be a need for IT skills for both in-vehicle technology and also simulation capabilities. Saraydar said that with these advancements, automobile companies are going to need more people in the area of embedded software.
“As these technologies progress and increase in acceleration to the marketplace, we will need more skilled people that understand various aspects of automated driving, but are also skilled in the art of software,” he said.
GM’s Rajkumar said that when maintaining these cars, two services will be required. Hardware will need to be updated (the sensors, actuators and computing platforms) when they fail or need re-calibration due to accidents, weather or fender-benders. Software will also need to be updated, and he said ideally these updates will be over the air.
Ford’s McBride said that it’s important to realize there is not a “one size fits all” for autonomous vehicle. He means that there are a variety of possibilities for autonomous vehicles, like an urban taxi fleet, Amazon or FedEx delivery vehicles, personally owned and operated vehicles, or commercial vehicles.
“You can imagine there is a broad spectrum of use cases, and in each of the use cases there might be different customers or partners that would want to use [the vehicles],” said McBride. “For each of those, there would be different software applications to go along.”
Sports car fanatics talk self-driving
Picture the classic auto show. There are rows of customized cars with their hoods popped and doors open, ready for a wandering guest to ogle at them. The owners are standing by their prized possessions, waiting for someone to ask them to rev the engine of their souped-up car.
It’s easy to picture one of these hot rod enthusiasts cruising down the highway at high speeds, but is it easy to picture them owning a car with upgraded self-driving features that will most likely require them to stay in lanes, drive at a safe (and slow) speed, and perhaps even leave some of the driving up to the car itself?
At the New York Autofest, these fast car lovers had mixed opinions on the future of this technology.
An accident on black ice left his 2003 yellow corvette totaled with thousands in damage, and Grey Cherveny of Bay Shore, N.Y., said that he would have loved to have something in his car that could let him know of the road conditions or to help him stay in the lane. His car was fixed up at the show, with new Lambo doors and a custom paint job, including a special exhaust, rims, spoilers, and a dress up kit in the engine. His car can get up to 200 mph, and he was able to get up to 175 mph driving it himself.
Even though he loves to go fast, he wants to keep this car for life, and keeping it (and him) safe is crucial.
“Safety is mandatory,” said Cherveny. “If these features [of self-driving] are cost effective, everyone will have it.”
He said he still would like to have control of his car, but if he had something that was there just to assist him (much like Tesla’s autopilot features), he would have no problem driving such a car.
Scott Becktold of Glen Cove, N.Y., was hesitant, saying he wouldn’t mind a car with self-driving features as long as it is going to take him fast because, “I’m a New Yorker, and we like to go fast.”
He had his own GT500 Shelby, which he said can get up to 200 mph, but he has only gone up to 160 mph with it. He said he could see other countries adding separate lanes for self-driving vehicles, but in “New York, it’s not going to happen,” he said.
Dee Budd of Hauppauge, N.Y., was polishing her Stingray C7 Corvette, and she said she is definitely against self-driving vehicles and vehicles with additional software. She said she’s more old-fashioned, and doesn’t want a car driving her because, “If someone is going to make a mistake, it’s going to be me [not the computer].”
Her son Vinny Budd agreed because recently he had his Jeep recalled due to hacking issues. He said they were shutting down the brakes and hacking the radio, and that if people could do that, what is stopping them from driving a person into a wall or trying to kill them?
“I don’t want a car driving me,” said Vinny. “I like to have control.”
Dee said sports cars are a passion of hers, and those are the cars she prefers. She said that because hacking and security is a big problem, she is wary of all of the software and computer components being added to cars.
“I don’t think we are safe anymore,” she said.
The start of something new
The idea of autonomous vehicles began as early as the 1930s, when aerodynamics industrial designer Norman Bel Geddes created General Motors’ “Futurama” exhibit at the 1939 World’s Fair. His exhibition displayed the future of highways, where cars would eventually drive themselves, according to the Museum of the City of New York.
Stanford University also had a first attempt at an autonomous road vehicle in the 1960s when researchers were trying to create a platform for controlling a Moon Rover from Earth, according to Stanford research. They named it the Stanford Cart, where a magnetic tape loop made it possible to vary the time delay of steering commands. It went through several upgrades over the years, and by 1979 it was able to get through a crowded room.
In the 1970s, a professor named Sadayuki Tsugawa and his colleagues at Japan’s Tsukuba Mechanical Engineering Laboratory created what might be the first autonomous car. It was a route guidance system named “Comprehensive Automobile Traffic Control System,” as well as a vision-based intelligent vehicle. In the 1980s, “He conducted research on autonomous navigation of the vision-based intelligent vehicle with a dead-reckoning function, as well as control of indoor mobile robots linked with infrared-based vehicle-to-vehicle communications,” according to his bio found on the ITS Laboratory website.
Carnegie Mellon University would say that they triggered the biggest leap in autonomous driving. Raj Rajkumar, professor and co-director of the General Motors-Carnegie Mellon Autonomous Driving Collaborative Research Lab, believes that they did—mainly in the DARPA Urban Challenge of 2007.
“It was the turning point in the history of autonomous vehicles, proving that vehicles that drive themselves are no longer just science fiction,” said Rajkumar. “It was a watershed moment which made the singular point that vehicles that drive themselves are no longer a question of if but when.”
There were several memorable challenges that Carnegie Mellon participated in, with the first being in 1995, where roboticists drove NavLab 5, a 1990 Pontiac Trans Sport, from Pittsburgh to Los Angeles on a trip called “No Hands Across America.” The vehicle used a portable computer, a windshield-mounted camera and a GPS receiver, and it almost completed the drive fully autonomously.
Here in 2016, headlines claim there will be fully autonomous cars on the road in the next year or two. But experts argue that it’s going to take much longer than consumers think.
