I tried Google's Pixel Screenshots and it's everything I'd want an AI feature to be

According to the gallery app on my phone, I have 361 screenshots saved, including billing statements, important dates, memes, and web articles I’ve always wanted to revisit but didn’t trust my browser to bookmark. If you asked me to find a specific image in the album, you’d have to give me a minute. Maybe longer.

Also: Google tops the Index with Gemini Live and Pixel’s AI features

In theory, Google’s new Pixel Screenshots feature should expedite that process. Now, I can simply type keywords or questions like “Con Edison bill for September” or “Which stadium is next week’s soccer game taking place?” and the phone will populate the most relevant screenshots. It works like magic. Or, should I say, it works like AI. 

The AI part of Pixel Screenshots fits what I’ll call the “old definition” better than the new one, meaning it has more to do with automated backend processing than content generation. You’re not redefining a picture here, creating new emojis, or generating captions for images; instead, the Pixel 9‘s Gemini Nano extracts as much information as possible from screenshots, stores it, and then retrieves it when prompted.

The whole Pixel Screenshots process happens on-device, so the internet and all its dangers are not involved when using the feature. Google tells me it plans to keep things that way for security and privacy reasons (read: the company would like to avoid a situation similar to Microsoft’s Recall debacle), and I’m glad it is. 

Also: I went hands-on with every Google Pixel 9 model, and this is the one you should buy

Besides screenshotting more things than I probably should — because it’s much easier to press two buttons simultaneously than to download a full webpage or save a URL in a notepad — I often don’t think about how much personal information ends up being captured in the frame. A skim through my phone’s screenshots album reveals home addresses, usernames of close friends and family, contact numbers, and other trinkets of information that I’d rather not have fall into the wrong hands. I digress.

From my brief demos of Pixel Screenshots, I’m most impressed by three things: the speed with which the phone pulls out image results (because this all works locally), the ability to upload and capture more images for the sake of future retrieval, and how seamlessly the feature works with natural behavior. To that last point, you don’t have to purposely label or manually transfer screenshots for the feature to work; everything you capture automatically flows into the dedicated app.

Also: Everything announced at Made by Google 2024: Pixel 9 Pro, Fold, Gemini, Watch 3, and more

Some finer details tell me just how much thought was put into Pixel Screenshots, such as the ability to quickly launch the URL that the image was captured in on Chrome or YouTube and the option to set a reminder when you initially screenshot something. This feels bigger than just an archiving feature; it can potentially change the way we interact and bookmark digital content.

At a time when smartphone makers are spending more time pitching AI features than camera hardware, Pixel Screenshots is a rare winner. I’m nearly sold on Google’s AI vision here, and that may be just enough to reel me into the Gemini universe.