Hardware performance gains only matter when developers can access those capabilities through familiar frameworks and tools, according to Arm.
The relationship between hardware and software is shifting as artificial intelligence workloads demand new optimizations, AI Magazine reported. Some industry observers argue that hardware now dominates software economics, but Arm maintains that both layers remain deeply connected.
“Advances in hardware only translate into better experiences when developers can access those capabilities through familiar frameworks and tools,” Chris said.
To bridge hardware and software, Arm is investing across its entire platform architecture. This includes SME2 acceleration in CPU designs and Arm Neural Technology integrated into Mali GPUs.
The company also develops software tools designed to help engineers run machine learning models efficiently. KleidiAI assists developers in optimizing AI workloads specifically for Arm-based hardware, alongside resources provided through the Arm AI Portal.
“That’s why Arm is investing across the platform, from SME2 acceleration in our CPUs and Arm Neural Technology in our latest Mali GPUs to software such as KleidiAI, which helps developers optimise AI workloads for Arm-based devices, and our AI Portal,” Chris said.
Traditional metrics centered purely on peak compute capability are giving way to broader efficiency standards. Sustained responsiveness and power efficiency now dictate how continuous artificial intelligence experiences run on everyday client devices.
“Peak compute capability matters, but sustained responsiveness, power efficiency and the ability to support continuous AI experiences will matter even more,” Chris explained.
The evolution of edge inference centers on widespread utility rather than raw speed for individual models. Bringing intelligence to all devices requires balancing thermal limits with continuous workload demands.
“The future of inference is about making intelligence useful everywhere, not simply making individual models run faster,” Chris said.
KleidiAI is a software tool developed by Arm to help software developers optimize artificial intelligence workloads specifically for Arm-based devices.
Arm is deploying SME2 acceleration in its CPUs and integrating Arm Neural Technology into its latest Mali GPUs to handle heavy machine learning tasks.
Performance metrics are shifting away from raw peak compute speed toward sustained responsiveness, power efficiency, and the ability to support continuous AI experiences everywhere.
Keep reading