Deep dive: Can Non-GPU Chips Really Serve Open Models Cheaper?

Why today

On October 19, Tenstorrent said it had closed a $1.5B Series E. The post-money valuation is $6.2B, more than double the $2.6B mark from its late 2024 Series D. Two new US investors led the round, Trace Capital and Innovation X. BetaKit and TechCrunch both report the pitch. The money goes into chips for open-weight models and video generation, and the company claims its approach is cheaper, more scalable and easier to customize than GPUs. The timing matters for two reasons. First, in May Bloomberg reported takeover interest from Intel and Qualcomm. TechCrunch puts the deal values discussed at up to $5B. Tenstorrent chose to stay independent and raise money instead. Second, the past few years have been hard on GPU challengers. So this episode asks a narrow, practical question. Once you count software, utilization and model support, what does a token actually cost on non-GPU silicon?

Plainly

Start with what Tenstorrent sells. It designs its own AI processors around small cores it calls Tensix, alongside RISC-V CPU designs. The first generation was called Wormhole and the current one is Blackhole. As CEO Jim Keller put it, "We built one architecture and sell it at every size." The range runs from developer cards to desktop boxes to racks. Tenstorrent also licenses its chip designs to companies like LG and Samsung. Now the key idea. A chip's sticker price is not its cost per token. Cost per token is everything you spend to run the machine, divided by the useful tokens it actually serves. That includes the hardware spread over its life, power, data center space, and the engineers who keep it running. Think of a delivery van. A van that costs half as much is no bargain if it fits only some of your parcels and spends half its day parked. Chips work the same way. Suppose a challenger box costs half as much as a GPU server but runs at half the utilization. Then your cost per token is about the same. Utilization depends on software. On Nvidia, the serving software is mature. Tools like vLLM batch many users' requests together and manage memory, so the chip stays busy. Model support is the other half. When a lab releases a new open-weight model, the low-level code that runs it on Nvidia usually exists on launch day, because almost everyone writes for CUDA first. On a challenger, the vendor's own engineers often have to write or adapt that code. If your customers want this week's model, waiting weeks costs you revenue. Tenstorrent's answer is an open-source software stack, including a compiler and a low-level kit. A company spokesperson told BetaKit that "AI models keep changing, so hardware has to run whatever comes next." That is the right goal. The question is how close the stack is to meeting it.

What the evidence shows

Some things are established by reporting. The round size, valuation and lead investors come from the company and are reported by both outlets. BetaKit reports that Tenstorrent has begun shipping Galaxy servers commercially and has about 1,000 employees. Named customers and partners include Virtu Financial, Equinix and Smallest.ai. Revenue is not stated. The company declined to give its total funding or say whether the round included any secondary sales. Last year it cut 7.5 percent of staff while shifting from enterprise to developer sales. Now the performance numbers. Almost all of them are self-reported. It served Llama 3.1 70B at 15 tokens per second per user, with 32 users at once. EE Times noted that H100-based API services delivered 20 to 50 tokens per second. That was a journalist watching a company demo, not an independent test. As Spheron summarizes it, Tenstorrent's own figures for a Wormhole Galaxy show 4,000 to 5,000 tokens per second on Llama 70B. The comparison is 2,500 to 3,500 on eight H100s running vLLM. Spheron flags the problem. The Tenstorrent runs were controlled, single-model tests without a production serving layer. The H100 figures came from a full serving stack. That comparison does not show what it appears to show. The strongest cost claim is for DeepSeek R1, a very large open-weight model. Which Nvidia setup, at what utilization, is not stated. Spheron also describes real gaps in the software. In its account, the models that work cleanly are mainly the Llama, Falcon and Mistral families. Support for mixture-of-experts models is partial, and most Hugging Face models need hand-written code. Spheron and GPU.NET both run GPU businesses, which is worth knowing when they assess a rival. The plain summary is that the evidence is lopsided. We found no independent, production-style benchmark of cost per token on this hardware.

How people read it

The same facts support different readings. The AGI-forward reading says inference demand will keep climbing steeply as models get used more. In that world, buyers want a second source of supply, and open-weight and video models are growing workloads. A credible non-Nvidia stack is valuable even before it fully matches CUDA, and the step-up prices that scarcity. The AI-realist reading points to history. Groq's biggest result was Nvidia paying $20B, as CNBC reported, to license its technology and hire its leaders. On this reading, challengers' best ideas end up flowing to Nvidia, because the software ecosystem is the moat. The market reading sits in between. The valuation is partly anchored by Intel and Qualcomm showing interest. Licensing chip designs to LG and Samsung gives Tenstorrent revenue that does not depend on beating Nvidia rack for rack. The policy reading matters for sovereign buyers. An open stack that a country or a cloud provider can own and modify appeals to governments wary of depending on one supplier. Tenstorrent's Canadian roots are part of that story. Toronto is still its largest office, and backers include Export Development Canada. These readings disagree mainly about how fast software catches up.

What it means for builders and investors

Our read is that this step-up was priced on the open-weight inference thesis and on strategic interest from buyers. It was not priced on disclosed revenue or independent cost data. That may still prove right. For builders, non-GPU silicon looks most plausible for narrow, stable, high-volume work. Think one open model, steady traffic and a team willing to tune it. It looks weakest for anyone who needs the newest model the week it ships. For seed and Series A companies, the pattern to watch is inference startups whose margin story depends on cheaper chips. One question worth exploring might be this. What is your cost per million tokens on your own traffic, at your real utilization? Does that figure include the engineering hours spent porting models? A close second is how many days it takes to serve a major new open-weight model after release. Go-to-market matters here too. Tenstorrent's move from enterprise to developer sales suggests the sales motion is still being worked out, so customer references are worth as much as benchmarks. Two things would change our mind. One is an independent, production-style benchmark showing a lasting cost-per-token gap. The other is a paying inference provider publishing prices backed by this hardware.

What to watch

Three things would settle most of this. First, independent measurement. That could be a third-party benchmark run, or a cloud provider publishing a price per million tokens on Tenstorrent hardware at real load. Second, software maturity. We'd want to see support in mainstream serving tools like vLLM, plus launch-week support for the next major open-weight and mixture-of-experts releases. Third, revenue. The company has said an IPO is its target, and a filing would finally show sales, margins and how much comes from licensing versus hardware. None of this has a fixed date. The next big open-weight model release will be the quickest test, and an IPO filing would be the most complete one.