Microsoft Edge will use AI to translate YouTube videos while you watch

Coming soon, an AI-powered update to Microsoft Edge will let you watch and listen to certain online videos in other languages. Announced by Microsoft at its Build conference on Tuesday, the new feature will translate videos in real-time on YouTube as well as a range of other sites.

On a Microsoft Edge features page, the company revealed more details about the real-time translation. To start, the feature will work with videos on YouTube, Reuters, CNBC News, Bloomberg, Money Control, LinkedIn, and Coursera. Microsoft said that it’s looking to support other sites in the future.

Also: Microsoft’s latest Windows 11 security features aim to make it ‘more secure out of the box’

For now, the number of languages included is limited. Audio sources in Spanish and Korean can be translated into English, while audio in English can be translated into Hindi, German, Italian, Spanish, and Russian. More languages should start to become available once the feature rolls out officially.

To address privacy concerns, Microsoft said that the translation occurs completely on your PC or mobile phone so that no data or content from the video leaves your device or gets processed in the cloud.

And although the translation strives to be accurate, the usual flaws and fallibilities found in AI may pop up. In particular, the translation is likely to be affected by such factors as the source language, the number of speakers, and any background music.

A video clip of Microsoft CEO Satya Nadella on the Edge features page showed how the real-time translation would work. For any supported video, hovering over it will display a small toolbar with a translate icon. Click the icon and you can select the source and target languages and opt to display subtitles as well.

After you click the Translate button, the video pauses as the audio is translated on your device. Once the translation is available, the video resumes using the target language you selected. The subtitles also appear at the bottom if you choose that option.

Also: 3 AI features coming to Copilot+ PCs that I wish were on my MacBook

With the buzz around generative AI, Microsoft has been aggressively infusing its products and services with artificial intelligence — and Edge has been one recipient of this push. Microsoft’s browser offers a Copilot sidebar through which you can access the AI tool to ask questions, find information, and generate content.

Real-time translation is another AI-based capability in Microsoft’s wheelhouse. Promoting the new and upcoming Copilot+ PCs at Build this week, the company touted a caption feature that will display text in English for any audio across several applications and video platforms. Supporting more than 40 languages, the captions can appear in such apps and services as Microsoft Teams, Zoom, Chrome, and Netflix.