TPUs are general purpose accelerators for any neural network. They still have to fetch instructions, manage a programmable memory hierarchy because it has to provide a general purpose platform for anything from a CNN to a Transformer.

What Taalas is doing amounts to hard-wiring the actual weights and structure of a specific model directly into the silicon. They call these Hardcore Models because the model is expressed directly in the topology of the chip. Their approach eliminates the overhead of fetching instructions or moving data between a separate processor and memory. They bypass the memory wall that plagues GPUs and TPUs by merging storage and computation at the transistor level. And that's why they see such massive leap in tokens per second and power efficiency. The trade off here is that you sacrifice the ability to run any model in exchange for running one specific model at near instantaneous speeds at a fraction of the cost.