Arm Kleidi Arrives in Automotive Markets to Accelerate Performance for AI-based Applications

AI in the automotive industry is often described as the heart of self-driving vehicles – futuristic in tone and years before we’ll see it in mass production on the road. But the reality is that many of the vehicle applications and features available in new cars today, from adaptive cruise control to personalized infotainment systems to driver and passenger monitoring, are already leveraging AI, with these application-specific models and workloads running on Arm. This is due to the flexibility, performance, power-efficiency and scalability of the Arm compute platform, making it the platform of choice for developers across the automotive industry.

However, as vehicle applications and features become more sophisticated with growing demands for more AI and workloads that are getting increasingly complex, the industry faces a unique set of challenges, including:

• A vast web of software systems in automotive, more so than in other markets, which makes the developer environment hugely complex;
• Needing a robust cloud to car software development and validation infrastructure to enable applications developed in the cloud to be seamlessly deployed to the car;
• Software must be compliant with strict safety and security industry standards and regulations; and,
• The longer ownership cycle of a car means the software must scale to accommodate new features and updates over many years (akin to updating your smartphone when the latest software update becomes available, but without the short product cycle).

Arm has been addressing these unique challenges through leading automotive enhanced (AE) technologies and collaborations across the automotive ecosystem on virtual platforms and standards-based software solutions via SOAFEE (Scalable Open Architecture for Embedded Edge) to accelerate automotive development cycles. All these solutions are equipping developers with the tools to seamlessly access improved performance and speed time-to-market for their applications via the ubiquitous Arm compute platform.

Today, I am pleased to announce the availability of another technology for the automotive market that will unlock the benefits of next-generation application-specific AI models in the car and enable their deployment at a quicker rate.

Automatic performance optimizations with Arm Kleidi

Arm Kleidi was introduced last year to deliver software performance optimizations for AI inference workloads running on Arm CPUs in key markets like mobile, cloud, and datacenter, with zero additional work needed from developers. Kleidi is already integrated into the latest versions of popular AI frameworks like ExecuTorch, llama.cpp, MediaPipe and PyTorch, so developers can simply start building their applications and automatically see performance improvements on Arm-based platforms.

These performance optimizations are now available to the automotive market and leading technology companies are already seeing improvements across their automotive software for a range of new applications. From vehicle chatbots, personalized driver recommendations, to image and motion enhancements for user assistance, diagnosis, and problem-solving, the integration of Kleidi libraries into key developer frameworks is accelerating AI applications to deliver faster, more efficient in-vehicle experiences.

AWS Automotive improves vehicle chatbot response time by 10x

Using a virtual platform from Arm and Corellium, AWS created a new proof-of-concept AI chatbot for software-defined vehicles (SDVs) allowing drivers to interact with the vehicle directly to ask questions, or inquire about dashboard alerts, rather than flipping through the pages of a physical car user manual. By leveraging the KleidiAI integration in the latest version of llama.cpp to develop the chatbot, AWS has seen chatbot response times improve by 10x, enabling chatbot responses of just one to three seconds. The use of KleidiAI has also resulted in time savings of six weeks for developing the application, as the developer does not need to focus on low-level software optimizations. For more information, check out this blog.

Video: Srini Raghavan, Senior Partner Solutions Architect, AWS, describes AWS’ new proof-of-concept AI chatbot demo

VicOne speeds response time to automotive cybersecurity threats

VicOne’s xCarbon on-board solution for SDVs enables vehicles to learn and recognize cybersecurity threats, defending the vehicle with reduced cloud dependency to lower costs and keep data more secure. Using the TinyLlama-1 1B model, VicOne has achieved significant performance uplifts, doubling the prefill prompt speed and improving token generation by more than 60%, reducing response time to any cybersecurity threats detected within the vehicle. The performance optimizations enabled by Kleidi is empowering VicOne to deliver a safer, more secure driving experience without any additional engineering work required.

Sonatus seamlessly brings AI Technician Builder for OEMs to run on Arm-based AWS Graviton chips in the cloud

Sonatus’ AI Technician Builder is a cloud-to-edge platform enabling OEMs to create customer service agents that improve driver experiences by providing rapid responses to user queries about the vehicle. Initially the platform used cloud-based GPUs; however, using a Kleidi-optimized framework, the AI Technician Builder was easily ported to run on Arm-based AWS Graviton Amazon EC2 instances in the cloud. This maintains the price performance and response times required by the user, while paving the way for running in-vehicle AI workloads at the edge on Arm-based automotive compute platforms with no network connectivity required.

Driving AI innovation in the car and beyond, on Arm

Today, 94% of global automakers are using Arm technology in their latest vehicle models. The ubiquity of the Arm platform across automotive means we’re uniquely positioned to enable impactful performance optimizations like what AWS, VicOne and Sonatus are already seeing for key in-vehicle AI applications. The integration of Kleidi into existing software stacks propels the possibilities for more efficient, advanced AI use cases and faster response times for critical automotive applications where every second counts. This gives developers the superpower to optimize and run new application-specific AI models and workloads on the Arm CPU with no additional overhead or developer effort.

With the extension of Kleidi to automotive, Arm is delivering on its promise to provide seamless performance acceleration for AI developers in all Arm markets – from cloud to edge – enabling them to create new, compelling user experiences that continue to push the envelope of AI’s transformative potential.