Microsoft last week reversed course on some concerning aspects of Recall, but infosec experts say the rollout shows security was an afterthought for the AI-powered feature and could indicate larger issues for the software giant.
On May 20, Microsoft announced CoPilot+ PCs for Windows with AI features such as Recall, a tool that uses a natural language model to take snapshots of the user’s work every five seconds. While the feature is intended to help users search for previously viewed content, the infosec community quickly called out the tech giant for building a tool that poses significant data security and privacy concerns for enterprises and consumers alike.
The outpour of backlash from security researchers and users led Microsoft to address some of the concerns in an update to Recall last week. Now, Microsoft has disabled the feature by default and made it so that Recall snapshots can only be decrypted and accessed with user authentication.
Microsoft said the updates will go into effect before Recall previews are shipped to customers on June 18.
“Even before making Recall available to customers, we have heard a clear signal that we can make it easier for people to choose to enable Recall on their Copilot+ PC and improve privacy and security safeguards,” wrote Pavan Davuluri, corporate vice president of Windows+ Devices, in the update.
Prior to the update, Recall being enabled by default was one of the industry’s biggest concerns. Enterprises are inundated with different services and accounts, and it’s not uncommon for administrators to forget to opt out of an enabled-by-default feature. Another top risk is how vulnerable Recall-stored data is to the average infostealer malware. Infosec experts worry that Microsoft has created an easy target for attackers where a trove of pertinent, sensitive data is stored in one place.
TechTarget Editorial contacted Microsoft, but the company did not comment at press time.
UPDATE: Microsoft published another update Thursday announcing that it had pushed back the preview release date. Now, the preview will be available in the Windows Insider Program (WIP) before it’s ready for Copilot+ PCs. Microsoft describes WIP as “Windows biggest fans.” Microsoft did not provide a specific date but said the preview will become available for WIP in the coming weeks.
“We are adjusting the release model for Recall to leverage the expertise of the Windows Insider community to ensure the experience meets our high standards for quality and security,” Davuluri wrote in the update.
Ongoing concerns
Despite the recent Recall updates that addressed encryption and opt-out concerns, the infosec community still sees more risks than benefits for the new tool. For example, researchers and analysts highlighted the feature’s dangerous keylogging and web tracking capabilities; some even compared it to spyware. In addition, infosec experts don’t see many instances where Recall would even be helpful, particularly for the average enterprise user.
Gabe Knuth, a senior analyst at TechTarget’s Enterprise Strategy Group, said Microsoft, like many other vendors, is trying to find features that drive widespread adoption for AI PCs. While that was the intention with Recall, he emphasized that Microsoft leaped before it looked.
Knuth stressed that many analysts derided Recall as something of a “sanctioned spyware” due to its ability to capture screenshots every few seconds and store detailed analysis of each screenshot. He added that the recent updates don’t address other concerns regarding challenges enterprises might face. If end users adopt Recall, there will now be a record of the data that IT departments have been trying to centralize for years, he said.
The problems are not limited to Microsoft, Knuth said, and market demand will reveal how detrimental they are to users.
“Even though Microsoft touted the security of keeping the data local, the fact that it was turned on by default and all that data was there for the taking by anyone who could compromise a machine was too much to swallow,” Knuth said. “This [update] announcement walks back both those things. Microsoft adding a bare minimum of protection around Recall is something they should’ve done from the start. It’s a misstep in what has otherwise been a pretty good start to the AI era at Microsoft.”
Knuth emphasized the timing of Recall’s release as well. On May 3, Microsoft CEO Satya Nadella penned a blog post following an investigation conducted by the Department of Homeland Security’s Cyber Safety Review Board (CSRB) into Microsoft’s security shortcomings.
Last year, a Chinese nation-state threat actor tracked as Storm-0558 obtained a consumer signing key from Microsoft and used it to breach Outlook email accounts at 22 customer organizations, which included some federal agencies. In response to CSRB’s criticisms, Nadella wrote that Microsoft is “prioritizing security above all else.”
“If you’re faced with the tradeoff between security and another priority, your answer is clear: Do security,” Nadella wrote.
Shortly after the blog post was published, during RSA Conference 2024, Microsoft announced the expansion of its Secure Future Initiative, which it first debuted in November. The vendor promised to address problems with software development and vulnerability mitigation. However, experts said the fumbled Recall rollout showed there’s more work to be done.
Jeff Pollard, vice president and principal analyst at Forrester Research, agreed that while the updates Microsoft made to Recall align better with its proposed security and privacy goals, they are insufficient. Pollard told TechTarget Editorial that no practitioner is comfortable with the idea of a built-in keylogging, screenshotting and web tracking tool in an operating system — even if Microsoft turned off the feature by default and added extra encryption.
“I don’t know anyone that’s tried it outside of trying it for security research purposes, and that’s the other part of this feature that befuddles me. I can’t think of anyone that uses a PC that actually wants the functionality Recall provides,” Pollard said. “The less sophisticated people I know that aren’t power users would be confused by it — or think their computer is hacked — and more sophisticated users I know would find it lacking in terms of features they would need to make it valuable.”
While Recall might not be useful for enterprise users, infosec experts agree that it could benefit attackers. Jeremy Nichols, director of the Global Threat Intelligence Center at NTT Security Holdings, told TechTarget Editorial that Recall will be a prime target for malicious actors because it could take screenshots of potential banking activity, medical systems and private messaging applications, which poses significant risks.
“This makes high-level executives, government officials, service providers dealing with customer data and activists/journalists very attractive targets,” Nichols said.
Recall compared to infostealers
Security professionals who spoke with TechTarget Editorial all cited simple scenarios where Recall could be exploited for malicious purposes, with the most basic threat being password exposure.
Like other infosec experts, Brian Reed, cybersecurity evangelist at Proofpoint, was also disappointed in Microsoft’s lack of security forethought around Recall, and the updates did not quell his concerns. One of his biggest concerns is the potential for misuse and unintended consequences of a tool that indiscriminately captures data. In the initial announcement, Microsoft warned that Recall does not moderate content and will not hide information such as passwords or financial account numbers.
“Effectively, it’s an information stealer disguised as a productivity feature,” Reed said. “The fact that Microsoft even thought that Recall was a compelling feature, in my mind, just perfectly shows how little regard they show for protecting personal privacy, and how stunningly tone-deaf they are with securing their own platform ecosystem.”
I’m conflicted as to whether Microsoft promoting Recall is genuinely malicious, stunningly negligent or some combination of both. Brian ReedCybersecurity evangelist, Proofpoint
Reed spoke with a Proofpoint researcher who discussed how easy Recall makes it for attackers to leverage living-off-the-land techniques. Ransomware groups and nation-state actors increasingly use commercial tools during attacks to stay under the radar. Reed stressed how easy it could be if an attacker accessed Recall, scrolled the screenshots and gained information on the user’s work and personal life. The risk rises if that user has a password manager and Recall captures screenshots while passwords are entered.
“I’m conflicted as to whether Microsoft promoting Recall is genuinely malicious, stunningly negligent or some combination of both,” Reed said.
Another potential risk he anticipates involves future updates to Windows 11. For example, although Microsoft claims that the data is only stored locally, future models could upload screenshots to Azure and be used for training large language models. Reed added that Microsoft could also simply enable Recall by default again in updates to the operating system.
He stressed that disabling Recall by default does not change the significant potential misuse of Recall screenshots.
“The fact remains that it took an enormous level of pressure from the entire security industry for the Microsoft productivity team to see this as a potentially large security [and] privacy risk and to ‘do the right thing,'” Reed said. “Microsoft’s response to the backlash on Recall tells us even more than them thinking that such a feature is in users’ best interests.”
Dan Schiappa, chief product and services officer at Arctic Wolf, previously worked at Microsoft and cited a lack of security professionals working at the company. He told TechTarget Editorial that while Microsoft has made some strides toward improving security, Recall is a big misstep that many people at the company just didn’t think through.
“They don’t have security people. They have people who work at the company, and they have a tour of duty of security at that moment, and then they move off to another job in the company,” Schiappa said.
While he was happy to see that Microsoft disabled Recall by default and he applauded the additional encryption safeguards, Schiappa’s concerns remain, as the feature is a prepackaged target full of sensitive data.
Schiappa said he hopes that Microsoft put substantial effort into tamper protection for Recall’s security settings. On the other hand, he understands how adaptive and persistent well-funded adversaries are and stressed that there’s nothing they can’t do over time.
“I think it’s one of the greatest violations of privacy in the history of computer science,” Schiappa said of Recall, “so it’s frankly shocking that Microsoft didn’t think this one through all the way.”
This article was updated on 6/14/2024.
Arielle Waldman is a news writer for TechTarget Editorial covering enterprise security.
