In the latest update for Android, a new feature called “Screen automation” is being introduced in the Gemini app, specifically for AI Ultra subscribers. This capability is currently being tested in the 16 QPR3 Beta 2 version and allows applications to interact with other apps’ screen content, enabling users to complete tasks more efficiently.
This functionality is visible in the Settings under Special app access, though it is presently limited to Pixel 10 devices, with no support for the Pixel 9 series. The Google app is the only application that appears to support this feature, offering options to always allow, ask every time, or not allow access to screen automation.
Furthermore, the feature is part of Google's broader initiative to enhance computer use by allowing agents to navigate the web similarly to human users. The company previously showcased Project Astra, which tests future capabilities for Gemini, demonstrating potential advancements in how tasks can be managed on mobile devices.