Elon Musk’s AI start-up xAI has vowed to expand its Colossus supercomputer tenfold to incorporate more than 1mn graphics processing units (GPUs), in a concerted effort to leap ahead of rivals such as OpenAI, Google and Anthropic. Colossus, built in merely three months earlier this year, is said to be the world’s largest supercomputer, operating a cluster of over 100,000 interconnected Nvidia GPUs. The chips are employed to train Musk’s chatbot Grok, which is less advanced and has far fewer users than market-leader ChatGPT or Google’s Gemini.

Work has already started to increase the size of the facility in Memphis, Tennessee, according to a Wednesday statement from the Greater Memphis Chamber. Nvidia, Supermicro Computer and Dell would also establish operations in Memphis to support the expansion, the chamber of commerce informed. The chamber of commerce also said it would establish an “xAI special operations team” to “provide round-the-clock concierge service to the company”.

The cost of obtaining so many GPUs would be substantial. The latest generation of Nvidia GPUs typically cost tens of thousands of dollars, though older versions of the chips can be less pricy. Musk’s planned expansion of Colossus would need an investment likely of tens of billions of dollars — plus the high cost of building, powering and cooling the massive servers in which they would sit. xAI has raised roughly $11bn in capital from investors this year.

AI companies are scrambling to acquire GPUs and access to data centres to source the computing power required to train and run their frontier large-language models. OpenAI, the creator of ChatGPT, has an almost $14bn partnership with Microsoft that includes credits for computing power. Anthropic, the creator of the Claude chatbot, has received $8bn in investment from Amazon and will soon get access to a new cluster of more than 100,000 of its specialised AI chips.

Instead of forming partnerships, Musk, the world’s wealthiest man, has used his power and influence in the tech sector to build his own supercomputing capacity, though he is playing catch-up after founding xAI merely more than a year ago. The trajectory has been quite steep — the start-up is valued at $45bn and recently raised another $5bn.

Musk is fiercely competing with OpenAI, which he had helped co-found with Sam Altman among others in the year 2015. The pair later on fell out and Musk is now suing OpenAI, seeking to block its transition from a non-profit to a more traditional business. A xAI investor said that the speed with which Musk had created Colossus was the “feather in the cap” of the AI company, despite it having very limited commercial product offerings. “He has built the most powerful supercomputer in the world in three months.”