Kneron takes intention at GPU scarcity with its neural processing unit (NPU) replace

With issues a few world scarcity of GPUs for AI, edge AI startup Kneron sees a chance for its neural processing unit (NPU) expertise as a aggressive various.

Kneron at the moment is saying its newest KL730 NPU, with the corporate claiming that it provides as much as 4 instances extra power effectivity than its prior fashions. The new chip can also be function constructed to assist speed up GPT, transformer-based AI fashions.

Kneron’s silicon is basically focused at edge purposes, akin to autonomous automobiles and medical and industrial purposes, though the corporate additionally sees potential for enterprise deployments. Kneron advantages from the backing of Qualcomm and Foxconn and has deployments with Quanta in edge servers.

“An NPU has more cores compared with a GPU,” Kneron founder and CEO Albert Liu informed VentureBeat. “The cores are extra environment friendly and they’re extra targeted with nuanced connectivity.

The expertise inside Kneron’s NPUs

Liu argued {that a} GPU is just not a purpose-built machine for AI. 

“GPU hardware was specifically designed for gaming, and right now it’s just Nvidia trying to brainwash all of us trying to say that only a GPU can do AI,” mentioned Liu.

Nvidia’s GPU expertise is, in fact, market main and is the idea on which fashionable giant language fashions (LLMs) and generative AI are constructed. Liu doesn’t suppose it should at all times be that manner, he mentioned, and he’s hopeful his firm will carve out an expanded market footprint as organizations more and more search for methods to fulfill AI calls for.

Kneron’s chips use a reconfigurable AI structure to speed up AI, which is a unique structure than what’s utilized in a GPU. With the KL730, the structure has additionally been particularly optimized for GPT’s transformer-based AI fashions.

Kneron well-established within the NPU market

The KL730 isn’t Kneron’s first chip optimized for transformers — the corporate introduced the KL530 silicon two years in the past, which had that functionality. The unique use case for the transformer mannequin in Kneron’s silicon was to assist autonomous automobile producers. Liu mentioned that transformer fashions could be very useful with actual time temporal correlation detection use circumstances. 

What wasn’t clear in 2020, not less than to Liu, was that transformers would turn out to be extensively used for enabling LLMs and generative AI. To assist meet the wants of LLMs, Liu mentioned that his firm has made its AI chip bigger for GPT model purposes.

“The reconfigurable AI architecture can dynamically change the structure inside the chip to support almost any kind of new model,” Liu mentioned.

The cascading energy of the KL730

With the brand new KL730, Kneron has made some dramatic efficiency enhancements to its NPU silicon.

Liu mentioned that the KL703 has higher efficiency than prior generations and will also be clustered. As such, if a single chip isn’t sufficient for a selected use case, a number of KL703s could be clustered collectively in a bigger deployment.

While Kneron’s silicon is basically used for inference use circumstances at the moment, Liu is hopeful that the flexibility to mix a number of KL730s collectively will allow broader use of the expertise for machine studying (ML) coaching as effectively.

“For server applications, Kneron already has customers like Naver, Chunghwa Telecom and Quanta,” mentioned Liu. “Foxconn is one of our strategic investors and they are closely working with us for AI servers.”