Google Search's AI mode goes multimodal

Google has dominated the search engine market for decades. However, the rise of generative AI has presented a new form of competition — AI search engines. With its AI Mode, Google provides users with exactly that: an AI chatbot that understands and outputs conversational search queries. 

As you might have expected, it looks a lot less like a traditional Google search and a lot more like ChatGPT Search.

Also: Microsoft is offering free AI skills training for everyone – how to sign up

Google’s AI Mode was met with such high interest that the feature went from being exclusive to Google One AI Premium users who pay $20 monthly to any Google account user via Google Labs. On Monday, Google updated it further, giving it multimodal capabilities and expanding access to millions of more Lab users. 

Now, leveraging Google Lens, the feature that allows users to upload and learn more about images on Google Search, users can snap or upload a photo into AI Mode. By using Lens in AI Mode, users get to experience a much more in-depth visual search experience than with traditional search because AI mode can understand the entire scene of an image. 

Google explains that this means Lens can understand specifics about an image, such as how objects relate to each other and characteristics of the image, such as the shapes, colors, and materials of the objects. Then, when a user issues a query, the AI Mode is able to asses both the image as a whole and the objects within it, providing users with more information and more contextually accurate responses. 

Google’s blog post gave the example of a user taking a photo of their bookshelf and asking for recommendations based on the books pictured. AI Mode was able to identify each book, then run a Search and output recommended books in a conversational response organized by the theme of the books. 

The feature is still accessible via Google Labs, which is free to access. All you have to do is visit the site, log in with your Google account, and join the waitlist. You’ll be alerted by email when you are taken off. Once you have access, you can try out this new capability in AI Mode in the Google app for Android and iOS. 

Also: Microsoft unveils 9 new Copilot features – you can try some now

This feature differs from Google’s AI Overviews and Gemini chatbot by combining real-time web-based answers with a conversational AI interface, offering the accuracy of a search engine and the interactivity of a chatbot.

Its biggest competitors include OpenAI’s ChatGPT Search or Perplexity AI. There are also many other AI Search engines on the market that suit different needs, and you can find ZDNET’s roundup here. 

Want more stories about AI? Sign up for Innovation, our weekly newsletter.