What it takes to build a self-driving car

Hola!!!

This is the first of the many upcoming posts in the series of autonomous driving. Its a basic introduction to autonomous driving and the challenges that any engineer faces while working on a self driving car. So lets decode autonomous driving and the complex challenges it pertains.

Meet FOMO: Introduction

Lets name our new autonomous car as FOMO - Future of Mobility (not Fear of Missing Out). Our aim is to give life to FOMO so that it could drive us safely from home to work place without killing or hurting us. Is our goal too ambitious? Lets find out!!!

In order to develop FOMO, we need to design it in such a way that it could drive itself without the need of a human driver. Although, you might argue about several fools you might have encountered in your life, human's in general are good with their perception and decision making skills. These human skills are what separates FOMO from its friends (normal cars) which require human assistance. In order for FOMO to drive safer and better than a human driver, it needs to develop these skills without relying on a human driver.

Image source

Giving Eyes to FOMO : Perception

First challenge in developing FOMO is to provide it with a vision, so that it can perceive its environment. Although, for a normal car with a human driver, a single pair of human eyes usually suffice, it is rather challenging for an autonomous car. We need to provide FOMO with multiple sets of eyes (sensors) so that it could perceive its environment full 360⁰ without any blind spots. Furthermore, we want its eyes to work in any kind of environmental conditions, be it clear or raining, day or night. On top of everything, even if one pair of eyes gets tired or damaged (sensor failure), we still want FOMO to keep everyone safe.

This is a problem which is formally known as Perception. Typically, for perception, multiple sensors such as Lidars, Cameras and Radars are used which we will discuss in future posts. However, we are not done yet. Installing multiple sensors doesn't mean that FOMO can perceive its environment. In order to make any sense of the information from these sensors, it needs to know how to process this sensor information and how to perceive its environment, the way that a human driver does. Specifically, it needs to learn how to process the data from different sensors and be able to identify street signs, lanes, other cars, moving pedestrians, traffic lights and much more (the list goes on and on). This is where fields like computer vision, machine learning, deep learning comes to a rescue. In the future posts, we will look into some of these and learn try to solve the problem of perception.

Finding FOMO: Localization

Localization is a major engineering challenge whenever it comes to autonomous driving or any autonomous robot. How does FOMO know where is it currently located? When driving with a human driver, humans are typically aware of their current location (city, street etc), but how does FOMO locate itself in this huge world. Engineers solve this problem by installing more sensors, typically, Global Positioning System (GPS), Inertial Measurement Unit (IMU) and wheel encoders and trying to localize the vehicle by comparing it with HD maps (High definition Maps). We will look into these sensors and how localization works in the future posts.

Localization is a critical problem while designing an autonomous car. FOMO can't take us from Point A to Point B safely without accurately (up to few centimeters) knowing its current location and position. However, all the sensors used for localization has some noise and some flaws e.g. GPS is fairly accurate but its update rate is slow (also it doesn't work in tunnels and extreme environments), IMU can provide updates more frequently, but its errors accumulate over time. Hence, FOMO needs to somehow estimate its location using noisy measurements from these different sensors and somehow fuse all this information to estimate its location. This is where fields such as Stochastic Robotics comes to a rescue. In particular, we will see in the future posts, how different filters such as Kalman Filter, Extended Kalman Filter, Unscented Kalman Filter and Particle Filters are used for localization using these noisy sensor readings.

Giving Brain to FOMO: Behavior Planning

FOMO needs to drive in the complex world, a world full of crowd, moving cars, and complex driving rules. Hence, to make sure FOMO is capable of signalling other cars about its next turn (by switching on the turn signal), waiting for the pedestrians to cross the road, and not drive in the wrong lane, it needs behavior planning. Typical behavior planning involves designing state machines such as lane following, lane change, intersection scenario etc. so that FOMO can drive safely without endangering itself or the other cars/pedestrians (also don't forget the traffic rules). More on Behavior Planning in the future posts.

Showing Path to FOMO: Path Planning

Image source

Path planning should come as no surprise to any of the readers. To take us from Point A to Point B, FOMO needs to plan its path. The path it plans should be safe, short, and should follow the driving rules. Although global path planning can be done using Navigation systems such as Google Maps, it is very important to do local path planning. Local path planning involves using the knowledge from perception and localization to plan the local trajectories between way points generated by the navigation system e.g. planning lanes, exact location, velocity etc for next few seconds between the way points. We will discuss different path planning algorithms in the future posts.

Manipulating FOMO: Control

In order to execute the trajectories generated by the path planning algorithm, FOMO needs control systems. Specifically, it needs controllers, which can control it gas pedal, brakes and steering angle to achieve the desired trajectory. The controller should be able to control the car with high accuracy and should try to minimize the error between the desired and the actual trajectory. Furthermore, it needs to be a stable controller such that it doesn't do something silly to put FOMO or other cars at risk. We will also look into control designing in the future posts.

Building FOMO: Conclusions

In this article, we saw different engineering challenges of Perception, Localization, Behavior Planning, Path Planning and Controls that are faced while building an autonomous car. It is by no means all the challenges faced in autonomous driving, but still covers the major modules that needs to be designed to truly build a future of mobility- FOMO.

In my future posts, we will discuss each of these challenges and try to decode the engineering behind it. I hope you found some value in this post.

Until next time, Hasta la Vista. Make sure to follow and vote.