Tenstorrent Engineers Talk Open-Sourced Bare-Metal Stack

By Sally Ward-Foxton, EETimes (February 2, 2024)

Tenstorrent, the Jim Keller–led AI chip and IP startup, is making available and open-sourcing its bare-metal software stack, which allows access to the hardware at the lowest level, Tenstorrent senior fellow Jasmina Vasiljevic told EE Times. The startup also recently showed EE Times a large language model (LLM)—Falcon-40B—up and running on its 32-chip Galaxy system, and it has made hardware available to purchase for the first time in the form of evaluation kits for its first-generation Grayskull chips.

One often-quoted reason for Nvidia’s dominance in AI and HPC applications is its mature software stack, CUDA, which allows users to write their own custom kernels for the most efficient possible execution of algorithms on Nvidia GPUs. Several startups have opened up bare-metal programming capabilities for their own hardware to date (including Untether, Esperanto and now Tenstorrent) to try to chip away at Nvidia’s lead in the data center. Doing so requires a significant software effort over and above the bare-essentials software stack many startups launch with, wherein typical initial functionality may be limited to inference for a subset of popular models.

Click here to read more ...