Google ML Drift Accelerates On-Device AI Inference Across Diverse GPUs - infoq.com

Google Releases ML Drift for On-Device AI Inference Across Diverse GPUs

Google has released ML Drift, an open-source, cross-platform GPU compute engine designed for on-device AI and machine learning inference. The engine abstracts the complexities of different GPU hardware and low-level APIs, supporting OpenGL ES, OpenCL, Metal, and WebGPU. ML Drift is the core GPU acceleration engine that powers LiteRT, and its main goal is to simplify access to diverse GPU hardware for developers creating applications for edge devices.

ML Drift introduces several technical improvements over the legacy TFLite GPU delegate, including unified shaders through tensor virtualization, extensible custom operations, and five-dimensional tensor support. It also incorporates hardware-specific optimizations for GPUs from Arm, Intel, and Qualcomm Technologies to maximize hardware utilization across modern GPUs. The engine extends desktop support for inference on Windows and Linux through Dawn, Chromium's WebGPU implementation, while retaining native Metal backend support on macOS.

Google has benchmarked ML Drift using Gemma models on developer-grade workstation hardware, reporting a 12% lower memory overhead compared to other frameworks. The company also emphasized that ML Drift is already running on millions of devices through apps like Chrome, YouTube Shorts, Photos, and Meet. In YouTube Shorts, ML Drift achieves up to a 40% reduction in average frame latency for segmentation-based effects on Android and iOS.