Elon Musk doesn’t need Tesla to be simply an automaker. He needs Tesla to be an AI firm, one which’s discovered find out how to make automobiles drive themselves.
Essential to that mission is Dojo, Tesla’s custom-built supercomputer designed to coach its Full Self-Driving (FSD) neural networks. FSD isn’t really totally self-driving; it may possibly carry out some automated driving duties, however nonetheless requires an attentive human behind the wheel. However Tesla thinks with extra knowledge, extra compute energy and extra coaching, it may possibly cross the brink from virtually self-driving to full self-driving.
And that’s the place Dojo is available in.
Musk has been teasing Dojo for a while, however the government has been ramping up discussions concerning the supercomputer all through 2024. Dojo’s significance to Tesla is perhaps existential – with EV gross sales slumping, buyers need assurances that Tesla can obtain autonomy. Under is a timeline of Dojo mentions and guarantees.
2019
First mentions of Dojo
April 22 – At Tesla’s Autonomy Day, the automaker had its AI group up on stage to speak about Autopilot and Full Self-Driving, and the AI powering them each. The corporate shares details about Tesla’s custom-built chips which might be designed particularly for neural networks and self-driving automobiles.
Through the occasion, Musk teases Dojo, revealing that it’s a supercomputer for coaching AI. He additionally notes that each one Tesla automobiles being produced on the time would have all {hardware} essential for full self-driving and solely wanted a software program replace.
2020
Musk begins the Dojo roadshow
Feb 2 – Musk says Tesla will quickly have greater than one million linked automobiles worldwide with sensors and compute wanted for full self-driving — and touts Dojo’s capabilities.
“Dojo, our coaching supercomputer, will be capable to course of huge quantities of video coaching knowledge & effectively run hyperspace arrays with an enormous variety of parameters, loads of reminiscence & ultra-high bandwidth between cores. Extra on this later.”
August 14 – Musk reiterates Tesla’s plan to develop a neural community coaching laptop referred to as Dojo “to course of actually huge quantities of video knowledge,” calling it “a beast.” He additionally says the primary model of Dojo is “a couple of yr away,” which might put its launch date someplace round August 2021.
December 31 — Elon says Dojo isn’t wanted, however it’s going to make self-driving higher. “It isn’t sufficient to be safer than human drivers, Autopilot in the end must be greater than 10 instances safer than human drivers.”
2021
Tesla makes Dojo official
August 19 – The automaker formally proclaims Dojo at Tesla’s first AI Day, an occasion meant to draw engineers to Tesla’s AI group. Tesla additionally introduces its D1 chip, which the automaker says it’s going to use — alongside Nvidia’s GPU — to energy the Dojo supercomputer. Tesla notes its AI cluster will home 3,000 D1 chips.
October 12 – Tesla releases a Dojo Know-how whitepaper, “a information to Tesla’s configurable floating level codecs & arithmetic.” The whitepaper outlines a technical customary for a brand new kind of binary floating-point arithmetic that’s utilized in deep studying neural networks and could be applied “totally in software program, totally in {hardware}, or in any mixture of software program and {hardware}.”
2022
Tesla reveals Dojo progress
August 12 – Musk says Tesla will “section in Dojo. Received’t want to purchase as many incremental GPUs subsequent yr.”
September 30 – At Tesla’s second AI Day, the corporate reveals that it has put in the primary Dojo cupboard, testing 2.2 megawatts of load testing. Tesla says it was constructing one tile per day (which is made up of 25 D1 chips). Tesla demos Dojo onstage operating a Secure Diffusion mannequin to create an AI-generated picture of a “Cybertruck on Mars.”
Importantly, the corporate units a goal date of a full Exapod cluster to be accomplished by Q1 2023, and says it plans to construct a complete of seven Exapods in Palo Alto.
2023
A ‘long-shot wager‘
April 19 – Musk tells buyers throughout Tesla’s first-quarter earnings that Dojo “has the potential for an order of magnitude enchancment in the price of coaching,” and likewise “has the potential to turn out to be a sellable service that we might provide to different firms in the identical method that Amazon Internet Providers affords net providers.”
Musk additionally notes that he’d “take a look at Dojo as form of a long-shot wager,” however a “wager price making.”
June 21 — The Tesla AI X account posts that the corporate’s neural networks are already in buyer automobiles. The thread features a graph with a timeline of Tesla’s present and projected compute energy, which locations the beginning of Dojo manufacturing at July 2023, though it’s not clear if this refers back to the D1 chips or the supercomputer itself. Musk says that very same day that Dojo was already on-line and operating duties at Tesla knowledge facilities.
The corporate additionally initiatives that Tesla’s compute would be the prime 5 in all the world by round February 2024 (there aren’t any indications this was profitable) and that Tesla would attain 100 exaflops by October 2024.
July 19 – Tesla notes in its second-quarter earnings report it has began manufacturing of Dojo. Musk additionally says Tesla plans to spend greater than $1 billion on Dojo via 2024.
September 6 – Musk posts on X that Tesla is proscribed by AI coaching compute, however that Nvidia and Dojo will repair that. He says managing the info from the roughly 160 billion frames of video Tesla will get from its automobiles per day is extraordinarily troublesome.
2024
Plans to scale
January 24 – Throughout Tesla’s fourth-quarter and full-year earnings name, Musk acknowledges once more that Dojo is a high-risk, high-reward undertaking. He additionally says that Tesla was pursuing “the twin path of Nvidia and Dojo,” that “Dojo is working,” and is “doing coaching jobs.” He notes Tesla is scaling it up and has “plans for Dojo 1.5, Dojo 2, Dojo 3 and whatnot.”
January 26 – Tesla introduced plans to spend $500 million to construct a Dojo supercomputer in Buffalo. Musk then downplays the funding considerably, posting on X that whereas $500 million is a big sum, it’s “solely equal to a 10k H100 system from Nvidia. Tesla will spend greater than that on Nvidia {hardware} this yr. The desk stakes for being aggressive in AI are at the least a number of billion {dollars} per yr at this level.”
April 30 – At TSMC’s North American Know-how Symposium, the corporate says Dojo’s next-generation coaching tile — the D2, which places all the Dojo tile onto a single silicon wafer, relatively than connecting 25 chips to make one tile — is already in manufacturing, in accordance with IEEE Spectrum.
Could 20 – Musk notes that the rear portion of the Giga Texas manufacturing facility extension will embrace the development of “a brilliant dense, water-cooled supercomputer cluster.”
June 4 – A CNBC report reveals Musk diverted 1000’s of Nvidia chips reserved for Tesla to X and xAI. After initially saying the report was false, Musk posts on X that Tesla didn’t have a location to ship the Nvidia chips to show them on, as a result of continued development on the south extension of Giga Texas, “so they might have simply sat in a warehouse.” He famous the extension will “home 50k H100s for FSD coaching.”
He additionally posts:
“Of the roughly $10B in AI-related expenditures I stated Tesla would make this yr, about half is inner, primarily the Tesla-designed AI inference laptop and sensors current in all of our automobiles, plus Dojo. For constructing the AI coaching superclusters, NVidia {hardware} is about 2/3 of the associated fee. My present greatest guess for Nvidia purchases by Tesla are $3B to $4B this yr.”
July 1 – Musk reveals on X that present Tesla automobiles could not have the best {hardware} for the corporate’s next-gen AI mannequin. He says that the roughly 5x improve in parameter rely with the next-gen AI “could be very troublesome to attain with out upgrading the automobile inference laptop.”
Nvidia provide challenges
July 23 – Throughout Tesla’s second-quarter earnings name, Musk says demand for Nvidia {hardware} is “so excessive that it’s typically troublesome to get the GPUs.”
“I believe this due to this fact requires that we put much more effort on Dojo so as to be sure that we’ve received the coaching functionality that we’d like,” Musk says. “And we do see a path to being aggressive with Nvidia with Dojo.”
A graph in Tesla’s investor deck predicts that Tesla AI coaching capability will ramp to roughly 90,000 H100 equal GPUs by the tip of 2024, up from round 40,000 in June. Later that day on X, Musk posts that Dojo 1 can have “roughly 8k H100-equivalent of coaching on-line by finish of yr.” He additionally posts pictures of the supercomputer, which seems to make use of the identical fridge-like chrome steel exterior as Tesla’s Cybertrucks.
XXX
July 30 – AI5 is ~18 months away from high-volume manufacturing, Musk says in a reply to a submit from somebody claiming to start out a membership of “Tesla HW4/AI4 homeowners offended about getting left behind when AI5 comes out.”
August 3 – Musk posts on X that he did a walkthrough of “the Tesla supercompute cluster at Giga Texas (aka Cortex).” He notes that it will be made roughly of 100,000 H100/H200 Nvidia GPUs with “large storage for video coaching of FSD & Optimus.”
