Léo RamosOn the USP campus in São Carlos, students take a backseat test drive in an autonomous carLéo Ramos
Drivers in their seats free to read, doze off, access their smart phones or even play a video game? All the while inside a moving vehicle? That is the promise of the autonomous car, which is expected to go on the market in the next 10 or 20 years. Meanwhile, research efforts are underway in this area, especially at universities, at some companies in the automotive industry, and at Google, which also maintains an experimental project. In Brazil, a Palio Weekend Adventure vehicle owned by the University of São Paulo (USP) was the first autonomous car authorized to travel on the streets of a city — São Carlos in São Paulo State — going a distance of 5.5 kilometers (km) in early October.
The car, bought at a Fiat dealership, was outfitted with special equipment by a group of researchers from the Institute of Mathematical Sciences and Computation (ICMC) and the School of Engineering at the USP São Carlos campus. “One of today’s major problems is that people are pressed for time and spend too many hours stuck in traffic especially in large cities,” says Professor Denis Wolf of ICMC, coordinator of the Intelligent Robotic Car for Autonomous Navigation (Carina) project, which has funding from FAPESP and the National Council for Scientific and Technological Development (CNPq), mainly through the National Institute of Science and Technology for Critical Embedded Systems (INCT-SEC).
“There are scientific studies showing that cars lined up in a row and traveling at the proper speed, automatically exchanging information with each other and with sensors on poles and in the ground which, in turn, communicate with the transit company, are able to increase the vehicular flow rate by up to three times on urban roads and non-urban highways. And this is only possible with autonomous cars,” says Wolf. In a potential future of smart cities we might see traffic lights and road signs exchanging information with cars. Signals from Wi-Fi and satellite-based GPS (Global Positioning Systems) and from vehicles, mobile phones and other devices carried by the passengers of all the vehicles, including motorcycles, trucks and buses, would form a network of information that would create ideal conditions for everyone to travel more safely and faster.
The technologies for this new world are being shaped now, as demonstrated by experiments such as those taking place at São Carlos and two other Brazilian universities: the Federal University of Espírito Santo (UFES) and the Federal University of Minas Gerais (UFMG). The São Paulo team is made up of two professors, plus Wolf, and 15 master’s and doctoral students, in addition to a mechanic who is responsible for adapting the car. They developed the control system with command software, and created an innovation that is likely to be patented as soon as development is finished. It consists of an electric motor on the steering column that engages and magnetizes a mechanism that causes the car to move the steering wheel according to commands from the computers, of which there are two. One computer, which is installed in the trunk, has high processing capacity and an industrial design capable of withstanding vibrations and high temperatures. It receives information from external sensors such as lasers, cameras and a very advanced GPS system that indicates the vehicle’s position on a map on a screen next to the driver. This processor tells the car what to do based on the information it receives and forwards it to another computer, a smaller one installed in the glove compartment.
“The smaller one controls the vehicle’s hardware, for example, turning the steering wheel 10 degrees, or accelerating and braking,” says Wolf. The brake has a mechanism called a linear actuator that activates a piston based on messages from the onboard computers. This system was designed by UFMG’s autonomous vehicles research group. The gearshift is factory automated and changes gears as needed by the car’s engine. “We also developed a system, which has not yet been implemented, that shifts into reverse based on the software controlling the vehicle.”
Ana Paula CamposAs safety is vital, the driver can at any time take over control of the car by pressing a button on the dashboard. This action switches off all the electronic systems that are not found in a conventional vehicle. With this, the driver is now in command. In the autonomous mode, the driver’s eyes are replaced by lasers and cameras that provide a 360° view, similar to the system used by Google Street View, which makes it possible to identify a location on a map along with everything around it at locations that were previously filmed with a car owned by Google. There are two lasers, one in the front and one on the roof. Each one, also with a 360° view, emits 700,000 beams of light per second to map everything within a radius of 50 meters, measuring the proximity of other cars, poles, people, dogs, signposts and any other obstacles. Each laser feeds back information on the angle and height of the obstacle in relation to the vehicle. “The lasers are invisible to the naked eye and are not harmful to human health,” says Wolf. A stereo camera with two lenses operates with a laser installed in the front of the car and estimates the depth of anything around the car, in addition to interpreting and sending back information on the traffic lanes. “At this point we are in the midst of development, and for the time being the car stays within its own lane of the road, but soon we think the camera will be able to determine whether the lanes are single continuous or double lanes, plus recognize crosswalks.”
The route of the car is preset on the map shown on the monitor, but along the route it is possible for the driver to change it at will. In the future, information on traffic conditions may order route changes autonomously based on reports received by the vehicle from the transit company or highway administrator. Images on the monitor also display obstacles and people around the vehicle.
The car, called the Carina II—Carina I was an electric golf cart that ran only on the campus—first traveled on USP campus roads in São Carlos in 2012, but in October of this year it moved onto city streets with the support of the city’s department of transportation and transit, which removed other vehicles from the route and escorted it with two motorcycles. “To our knowledge this is the first test of an autonomous car on public roads in Latin America that involved obtaining all the necessary permits from the authorities,” says Wolf, who has no funding or partnership with any automobile or auto parts company. “With funding from FAPESP and CNPq, we bought the car for R$50,000 in 2010. All the major equipment such as lasers and cameras was imported. The lasers cost R$110,000 and the 360° camera over R$40,000, but import duties did not have to be paid because they were devices purchased via CNPq for research,” says Wolf.
“But we still have a long way to go because our car still cannot overtake another car, and the speed is limited to 40 kilometers per hour (km/h) for safety reasons,” he says. The studies will become more advanced as we work with a group from Ohio State University (OSU) in the United States, through a project made possible by a cooperation agreement signed between FAPESP and the American university. “The Ohio group is more focused on electrical engineering and less on computers than we are. They have more than 15 years of experience and are more advanced in controls and instrumentation, but we are further ahead in the interpretation and processing of the images captured by cameras and sensors,” he says. Even while working on the evolution of the system, the USP group has been recruited as a consultant to companies in the agricultural and heavy vehicle sectors, on projects about which entrepreneurs prefer not to comment.
Léo RamosDenis Wolf, USP: autonomous cars could increase traffic flow in cities up to 300%.Léo Ramos
Even with so many projects, the São Carlos group is not the most famous in Brazil. The team led by Professor Alberto Ferreira de Souza of the High Performance Computing Laboratory (LCAD), of the UFES Information Technology Department, became well-known in Brazil by chance when they were involved in a minor accident with Ana Maria Braga, a Globo TV Network host, during a live program on the morning of April 22, 2013. After following the intended route in autonomous mode, the car unintentionally went into manual mode due to human error. With that, the vehicle moved—the hand brake had not been set and the car was on a small incline—so that an open door knocked Braga to the ground; she suffered minor injuries to her mouth and hands. “We were nervous because the program was live, and so we failed to follow the safety rules when we switched off the automatic system and went to manual operation,” says De Souza. “But she handled it very well and was even worried about a potential negative impact on our project.” But what happened was just the opposite, a lot more people became interested in learning about “the car that can drive itself.”
The UFES group, composed of three professors and nine master’s and doctoral students, developed the software that controls the car, but the focus of their long-term research is to understand how the human brain works in relation to visual cognition, in order that they can improve the car’s vision. “The brain is able to create internal representations, such as, for example, measuring the volume of something. Some people can look at a piece of furniture and estimate its size. And tennis players must accurately calculate the speed of the ball and the position of the racket,” says De Souza. “In the car, the universe is quite sophisticated because there are traffic lights, for example, and specific rules for everyone. We want to understand the neural models, not just the mathematics, of situations that indicate where the person is, in what place. But perhaps cars do not need to have as much information as we do.”
To carry this experiment forward, De Souza acquired, with funding from CNPq and the Espírito Santo Research Foundation (FAPES), a hybrid Ford Escape vehicle from the United States. He hired an American company to modify the car, equipping it with mechanisms to control the steering wheel, brake, accelerator, gearshift and other instruments via computer, in addition to installing the 360° view cameras and lasers. De Souza’s team then developed the artificial intelligence software to control and navigate the autonomous car. So far, the vehicle has cost a total of R$500,000. In order to get it out and onto the streets, the car is waiting for bureaucratic problems such as license plates and other procedural matters to be resolved. Meanwhile, tests are being carried out on the UFES campus in Vitória. The ambitious goal of the group is, in 2014, to make the car go as far as Guarapari, 50 kilometers away, and advance their research on neural networks, especially in terms of understanding traffic rules.
Unlike De Souza, who purchased a car in 2012 with the computer activated mechanisms already built in, the group led by Professor Guilherme Augusto Silva Pereira, who is with UFMG’s Department of Electrical Engineering, began in 2007 to build an autonomous car from scratch. “With the help of the Minas Gerais Research Foundation (Fapemig), we bought the equipment such as lasers, stereo cameras and GPS, and we designed the installation of all the internal and external automation in such a way that the system can be installed in any vehicle,” says Pereira. The car we used is a GM 2003 Astra, which was won as a prize from the manufacturer by another university group in a competition. Over the years, Pereira and his students have rebuilt the car’s entire control architecture and developed a speed controller ranging from zero to 40 km/h. “There already exist controllers that start at 30 km/h. We made a new one because this equipment will be essential in autonomous cars of the future, when it will be necessary, in traffic, to go slower than 40 km/h behind another vehicle with the aid of sensors.” The UFMG Astra is also restricted to campus streets in Belo Horizonte. “Because the car was donated, it has no chassis number and cannot be registered,” says Pereira.
There is much that we still need to accomplish, and other groups are also trying to develop navigation that avoids obstacles, by identifying a person, a pole, or another vehicle, knowing how to figure out what’s ahead,” says Pereira. “We need semantic mapping for obstacles when naming and identifying the type of problem to be addressed.” He believes that autonomous vehicles are leaving the realm of research, at least in the area of car assembly. “Companies in the automobile industry are already building advanced models. For example, Nissan has announced that it will have an autonomous electric car by 2020,” says Pereira.
“We have examples to prove that the technology is feasible, for example, in the US, where Google adapted a Toyota Prius for use as an autonomous car; it has traveled thousands of kilometers. In Germany, Mercedes, VW and the Fraunhofer Institute all have prototypes,” says Ricardo Takahira, who is an engineer and the manager of New Business and Innovation in Electric Vehicles for Magneti Marelli, and a member of the committees on hybrids, electric vehicles, telematics and infotainment (information and entertainment) of the Society of Mobility Engineers (SAE Brazil). “But we still need a lot more, including specific legislation for autonomous cars. Who would be liable if this vehicle accidentally ran over someone? Would it be the owner, or the manufacturer? Could this type of car be insured? These are open questions,” says Takahira. “Technology has to go hand-in-hand with the law.” De Souza also worries about the future when it comes to hackers who could get into the car’s operating system and even cause accidents.
DENIS WOLF/USPThe car on the streets of São Carlos DENIS WOLF/USP
Takahira says we still need high-definition electronic maps of all locations before electronic navigation can occur, as well as electronic road signs. It would also be necessary to find basic solutions for problems such as dirt on the lens of the cameras, for example, which can lead to interpreting errors by the imaging software. These are details that will gradually be worked out in the same way that cars are already becoming autonomous. He recalls vehicles such as the Volkswagen Touareg, which is sold in Brazil, that with the aid of sensors and cameras, can park itself in a vacant spot.
Some difficulties in the way of autonomous cars are beginning to be resolved, as occurred in the US to benefit Google’s experimental vehicles. To make driving around on the streets easier, California, Nevada and Florida passed laws allowing tests with these types of vehicles. So these cars have already traveled thousands of kilometers without a driver, accompanied only by engineers sitting inside the vehicles. Google has five cars, a Prius and three Lexus RX models, a Toyota, and an Audi TT. The cars control system is called Google Chauffeur and is coordinated by Sebastian Thrun, an engineer from Stanford University’s Artificial Intelligence Laboratory and one of the inventors of Google Street View. Thrun ‘s team won a prize of $2 million in 2005 from the US Defense Department’s Advanced Research Projects Agency (DARPA) for their design of an autonomous car. They were then hired by Google. Many experiments are going on elsewhere in the world, mainly in Germany, Italy, Japan and Israel. Beyond that, Wolf says that even if the technology evolves and is adopted, we are still left with the question: “Who is going to buy such a car?” Besides being more expensive, the car would be monitored by a multitude of sensors in the streets and on the roads. “You could not quickly overtake another car as you do today, drive faster than other cars or drive more slowly than the flow.” Wolf can look even further into a potential future. “Could there come a time when you no one needs to buy a car? Instead, you would phone in a request for a driverless car, which picks you up and drives you to your destination and charges you only for the trip?”
FOCA LISBOA/UFMGUFMG’s pioneer Astra, assembled in 2007FOCA LISBOA/UFMG
Frequency of stars and cars
Vehicle automation increasingly uses radar that can interfere with other sectors
Modern cars are being given sophisticated electronic equipment such as adaptive cruise control (ACC). Using sensors located in the front of the car, the system, without active participation by the driver, controls the accelerator or brake, maintains a safe distance from the vehicle ahead, and is able to avoid a collision in the event of a sudden stop. In such cases, one of the problems encountered by autonomous vehicles is the need to use a radar frequency of 79 GHz, which is the same frequency reserved in Brazil and throughout the Americas for use in radio astronomy. The Itapetininga radio observatory in Atibaia (São Paulo State), run by the National Institute for Space Research (INPE), uses this band to capture data from space. “Brazil’s National Telecommunications Agency (Anatel) and the International Union of telecommunications (ITU) have formed committees to study the matter, because the United States and Europe already have agreements to use this frequency in automotive radar systems,” says Takahira. A single vehicle located at a distance of 30 km and equipped with one of these radar systems is capable of interfering with the radar of a radio observatory. “A group of researchers from Anatel has been involved in discussions with the ITU and the Inter-American Telecommunication Commission (CITEL) trying to encourage and establish compatibility criteria among these services so that radar can be used in frequency bands that do not hinder ongoing scientific work,” says José Willians Vilas Boas, an INPE researcher.
Projects
1. National Institute of Science and Technology for Critical Embedded Systems (INCT – SEC) (nº 2008/57870-9); Grant Mechanism Thematic Project – INCT; Coordinator José Carlos Maldonado – USP; Investment R$ 2,639,677.06 (for all INCT – SEC) FAPESP.
2. A Collaborative Effort for Safer and More Efficient Transportation with Intelligent Vehicles. (FAPESP-OSU/2013) (nº 2013/50332-0); Grant Mechanism Regular Line of Research Project Award; Coordinator Denis Fernando Wolf – USP; Investment R$ 21,660.00 (FAPESP).
3. Carina Project – Intelligent Robotic Car for Autonomous Navigation (nº 2011/10660-2); Grant Mechanism Regular Line of Research Project Award; Coordinator Denis Fernando Wolf – USP; Investment R$ 55,753.20 (FAPESP)
