AMD Simplifies NPU Application in Software with the Release of Open Compiler LLVM Peano for Ryzen with XDNA and XDNA2

In an AI acceleration milestone, AMD has integrated artificial intelligence (AI) engines into its processors for the first time, starting with the Ryzen 7000 (Phoenix) having a Xilinx Neural Processing Units (NPU or XDNA) able to perform up to 10 trillion operations per second (TOPS). Advancing further with its Ryzen 8000 (Hawk Point), AMD increased the NPU’s performance to 16 TOPS. However, it was with the unveiling of Ryzen AI 300 Strix Point (XDNA2), boasting an impressive 50 TOPS, that AMD launched its specially designed open-source compiler called Peano.

Expected to boost the development of AMD NPU applications, the Peano project allows for the acceleration of large language models deployed on new AMD processors. AMD demonstrated how systems running Ryzen and Radeon processors, using tools such as LM Studio, can be utilised for this purpose. The company also highlighted multiple instances of Ryzen AI processors being employed in software development, even though these processors are not designed for end-users.

The extensive use of built-in NPUs from AMD, Intel, and Qualcomm indicates a competitive race for the fastest AI accelerators. AMD has been playing catch-up, given Intel’s earlier release of NPU software. Linux has had instructions for the Meteor Lake NPU even before its new architecture was unveiled. Plus, the open-source code for the NPU plug-in is available on Intel’s OpenVINO platform. With both companies providing open-source compilers to end-users, developers should find it easier to offer cross-vendor solutions. No such developments for Qualcomm’s Snapdragon series have been reported.

“On behalf of AMD, I’m pleased to announce open-sourcing the server-side of the LLVM for AMD/Xilinx AI Engine processors. These processors are present in several devices, including the Ryzen AI SoC. The current repository targets the AIE2 architecture, deployed in Phoenix and Hawk Point devices’ XDNA accelerators. Note that these accelerators consist of an array of processors, while the server-side of the LLVM supports only a single processor. Open-source tool support using MLIR is available for entire devices.” – said Stephen Neuendorffer, the chief engineer at AMD/Xilinx, while commenting on the release of the compiler.