Google Deploys LiteRT Tensor API and Play Services Upgrade for on-Device Machine Learning
Google has introduced the LiteRT Tensor API alongside a major upgrade to LiteRT in Google Play services, giving developers C++ control over machine learning models and delivering faster GPU inference. The LiteRT Tensor API allows developers to construct computation graphs operation by operation in plain C++. Beyond controlling model architecture, the API supports direct weight ingestion from checkpoints including Hugging Face .safetensors files, and provides zero-copy buffer interoperability to prevent redundant copying across pre-processing, inference, and post-processing stages. Google demonstrated the framework using an AI coding agent to build a multi-object segmentation and tracking application for the SAM 2 model. Running on MacBook Pro GPUs, the implementation executed roughly 1.5 times faster than Apple MLX using fp16 precision for the image encoder and mask decoder. Complementing the developer tools, Google Developers reported that LiteRT in Google Play services received its largest upgrade to date. The runtime now delivers GPU inference approximately 1.4 times faster than the legacy TensorFlow Lite GPU delegate. By shipping the runtime and GPU accelerator directly through Play services rather than bundling them inside individual APKs, apps that switch to the new CompiledModel API shrink by about 8 megabytes. Arm director of AI Laurence Moroney noted on LinkedIn that developers should start with standard model conversion and reserve the C++ API for scenarios requiring strict graph-level control or accelerator-resident pre- and post-processing.
AI Coding Agent Optimizes SAM 2 Model Performance
The dual release targets different development workflows by pairing low-level C++ graph authoring with a managed runtime for Android. According to InfoQ, the LiteRT Tensor API enabled an AI coding agent to optimize the SAM 2 model by moving per-frame visibility decisions out of CPU code and directly into the computation graph. The agent also rewrote operations that triggered CPU fallback on WebGPU into GPU-compatible arithmetics, pushing performance from under 10 frames per second to more than 30 frames per second at a 512×512 resolution on WebGPU.
At the same time, the Google Developers blog detailed the footprint and deployment advantages of integrating LiteRT into Google Play services. The shared runtime infrastructure reduces app size and accelerates downloads while delivering automatic performance improvements, bug fixes, and security patches through staged Play services rollouts. The runtime supports Android devices running Android 7 and newer, alongside Wear OS 4 and higher and Android Automotive OS version 11 and up.
🧠 Google dévoile LiteRT Tensor API, une couche C++ pour construire des modèles d'IA opération par opération et les exécuter sur l'appareil, sans conversion préalable. Google recommande toujours la conversion de modèles existants, ici 👇https://t.co/tWkGIfpDog#LiteRT #Google pic.twitter.com/KgwqtNrHEg
| Feature | LiteRT Tensor API | LiteRT in Google Play Services |
|---|---|---|
| Primary Interface | Manual C++ Graph Authoring | CompiledModel API (Kotlin/Java) |
| App Size Impact | Varies by bundled dependencies | Reduces app size by roughly 8 MB |
| Hardware Targets | CPU, GPU, and NPU via custom graphs | CPU and GPU acceleration (NPU in development) |
| Target Use Case | Advanced model optimization and agent iteration | Standard runtime deployment across billions of Android devices |
Google Expands LiteRT Ecosystems with Upcoming Feature Releases
Google is expanding the capabilities of both LiteRT ecosystems through upcoming releases. InfoQ noted that the engineering team is actively building direct ML Drift backend support, incorporating additional NPU-friendly operations, developing automatic full-graph quantization, and widening dynamic-shape support for varied image sizes, batch sizes, and sequence lengths. Simultaneously, Google Developers confirmed that NPU support inside Play services is currently under active development to broaden hardware acceleration coverage across supported Android endpoints.
Keep reading