Back in 2016, Tesla announced that it would be shipping all of its cars with the necessary hardware to support the future advancements in autonomous tech. And, as promised, Tesla not only managed to live up to their ambitious goal but also ended up manufacturing the world’s most advanced chip. So far, Tesla’s cars were outfitted with NVIDIA’s hardware 2.0 computer that was installed in all Teslas produced from October 2016, a news report indicated which was based on NVIDIA’s DRIVE PX 2 platform, that is the open AI car computing platform that enables automakers to provide Level 4 autonomy.

At the recently concluded Tesla autonomy event for investors, Pete Bannon, VP of Hardware Engineering and Andrej Karpathy, Director of Artificial Intelligence and Autopilot Vision along with Elon Musk unveiled the company’s in-built Full Self Driving Compute (FSD) platform the new brain behind Tesla’s cars which rivals NVIDIA’s self-driving hardware.

In this article, we dive into the underlying architecture behind FSD.

The above image is of the processing unit developed by Tesla called Tesla FSD(full self-driving) computer, which will take care of everything an autonomous vehicle requires. The board consists of two chips.

These chips can handle video from the on-board cameras and computation for a full self-driving car. The two computers on the board receive power supply and video independently. This is to keep the car running in the event of a malfunction.

Each chip consists  of a die, of area equal to 260 mm^2, which houses 6 billion transistors(!!)

The image signal processor has a 24-bit pipeline which enables the chip to take full advantage of HDR sensors around the car. Its advance tone mapping refines details from hazy attributes like shadows of the objects. These images would be fed into the neural networks which will their job easier.

Tesla also took care of the authenticity of the software used by cryptographically signing the chip to use only a certain kind of software.

The full computer assembly is compact and can fit between the glove box and firewall in the car. And, all these upgrades without any implications on the range or cost of the car.

Why Go For A Neural Network Accelerator?

The chip design team at Tesla figured out that the post processing on GPU would decline with improvement in neural networks.

So, neither CPU nor GPU but a dedicated neural network accelerator(NNA) was deployed. The main reason being the insignificance of GPU and CPU when pitted against NNA.

There are two neural network accelerators on the chip. Activation and pooling are two important operations in a neural network and each neural network processor here, has a dedicated ReLU hardware and pooling hardware delivering 36 trillion operations per second(TOPS).

Convolution occupies major chunk of the operation in the neural network followed by deconvolution and activations. At the fundamental level, these operations are nothing but a bunch of multiplications and additions (dot product) happening on a large scale. So, the idea behind the accelerator is to be smart about the way energy is used to carry out arithmetic operations; multiplication consumes 3 pJ(pico Joule) of energy whereas addition only 0.10 pJ.

Fleeting Approach For Training Neural Nets

The real world is as weird as it can get and more so for driving amongst a bunch of other drivers with varying moods and motivations. A human annotated dataset cannot capture all the complexity of the real world scenario. The neural nets have to be automated for training as the data gets streamed through the 8 cameras.

Cut ins are one of the potential candidates for accidents on highways. When a neighbour’s car switches lanes suddenly, a normal driver would have to sway smartly or  crush the brake pedal in desperate situations. It gets tricky when self driving cars are presented with these kind of problems.

Since the time available for taking action is very less, the AI team at Tesla took the fleeting approach. Whenever a vehicle switches lanes, the data is taken, neural networks are trained and are made to predict a potential cut in and warn the system well in advance, avoiding a probable catastrophe.

Tesla enjoys the advantage having started their journey very early. They possess the world’s largest customer base for semi-autonomous vehicle. Whenever a Tesla driver takes an action be it steering left or right or pressing the pedal, what they are doing is annotating the data and generating more refined data related to driver’s behaviour.

As the time passes, more data will be generated and every possible driver reaction will be captured. This will give the computers more control over the steering mechanism and perhaps drop the need for having the steering wheel altogether in the future.

Musk and his team have time and again amazed the world with their ability to pursue their ambitious goals in a successful yet honest way.  It has been only one day since the Tesla event and one might expect the news portals to overflow with the technical details and the potential use cases of the technology used. But, unfortunately, today there is more news about Tesla’s quarterly losses than the achievement that was more than a decade in the making. The inexplicable urge of the armchair perfectionists to overshadow the triumph of a man’s passion has been depriving others of a chance to yearn for the future as a species; something which has been missing since the lunar landings.

Know more about how the heuristics are designed for path predictions using shadow mode, how false positives are tackled, data annotation using RADAR and how self supervision comes into play by watching the video below: