Build Self Driving Car Model Using Deep Learning

Sanjeeva Rao Palla
4 min readSep 19, 2020

Objective : Predict the steering angle in Self Driving Car model using front camera view

Self Driving Car :

A self-driving car(Conditional Driving Automation) can drive itself in some or even all situations, but a human passenger must always be present and ready to take control.

Levels Of Driving Automation :

Fig 1 : Levels of driving automation

Challenges with Self Driving Cars :

Lidar and Radar

Lidar is expensive and is still trying to strike the right balance between range and resolution. If multiple autonomous cars were to drive on the same road, would their lidar signals interfere with one another? And if multiple radio frequencies are available, will the frequency range be enough to support mass production of autonomous cars?

Weather Conditions

What happens when an autonomous car drives in heavy precipitation? If there’s a layer of snow on the road, lane dividers disappear. How will the cameras and sensors track lane markings if the markings are obscured by water, oil, ice, or debris?

Traffic Conditions and Laws

Will autonomous cars have trouble in tunnels or on bridges? How will they do in bumper-to-bumper traffic? Will autonomous cars be relegated to a specific lane? Will they be granted carpool lane access? And what about the fleet of legacy cars still sharing the roadways for the next 20 or 30 years?

State vs. Federal Regulation

The regulatory process in the U.S. has recently shifted from federal guidance to state-by-state mandates for autonomous cars. Some states have even proposed a per-mile tax on autonomous vehicles to prevent the rise of “zombie cars” driving around without passengers. Lawmakers have also written bills proposing that all autonomous cars must be zero-emission vehicles and have a panic button installed. But are the laws going to be different from state to state? Will you be able to cross state lines with an autonomous car?

Accident Liability

Who is liable for accidents caused by an autonomous car? The manufacturer? The human passenger? The latest blueprints suggest that a fully autonomous Level 5 car will not have a dashboard or a steering wheel, so a human passenger would not even have the option to take control of the vehicle in an emergency.

Fig 2 : How a self driving car works

Problem Statement:

We are here building a minimal version of self driving car. Here, we have a front camera view. This will transfer input to the computer. Then Deep Learning algorithm in computer predicts the steering angle to avoid all sorts of collisions. Predicting steering angle can be thought of as a regression problem. We will feed images to Convolution Neural Network(CNN) and the label will be the steering angle in that image. Model will learn the steering angle from the as per the turns in the image and will finally predicts steering angle for unknown images.

Data set :

Refer this : https://github.com/sanjeevpalla/Self_Driving_Car/tree/master/Dataset

There are total 45406 images in the data set along with their steering angles. We will split the data set into train and test in a ratio of 70:30 sequentially.

Model Architecture :

Fig 3 : Self Driving Car Deep Learning Model

Procedure :

Our goal in this model is to predict steering angle and it is real value. So we can classify this problem as regression problem. Since it is regression problem we used Mean Squared Error(MSE) as our performance metric. First, we trained this model on 70% of data with 30 epochs and calculated MSE for each epoch. We observed that for each epoch MSE has decreased and we saved final model in disk. Later, we loaded the saved model and tested on 30% of data.

Mean Squared Error(MSE) :

MSE or Mean Squared Error is one of the most preferred metrics for regression tasks. It is simply the average of the squared difference between the target value and the value predicted by the regression model. As it squares the differences, it penalizes even a small error which leads to over-estimation of how bad the model is. It is preferred more than other metrics because it is differentiable and hence can be optimized better.

Fig 4 : Mean Squared Error Formula

Results :

https://youtu.be/ksGjMIpEaU8

Source Code :

Refer this : https://github.com/sanjeevpalla/Self_Driving_Car

References :

--

--

Sanjeeva Rao Palla

Artificial Intelligence & Machine Learning Engineer Aspirant | Learner