Smarter Homes & Gardens: Smart Speaker Privacy

So is your smart speaker really listening in on your conversations? 

That’s the crux of a popular privacy topic. Namely, are we giving up some of our privacy in exchange for the convenience of a smart speaker that does our bidding with the sound of our voice? After all, you’re using it to do everything from search for music, order online, and control the lights and temperature in your home. 

What is your smart speaker really hearing—and recording? 

Let’s take a look at what’s going on inside of your smart speaker, how it processes your requests, and what companies do with the recordings and transcripts of your voice. 

So, are smart speakers listening in? 

More or less, smart speakers are listening to all the time. Each smart speaker has its own “wake word” that it listens for, like Alexa, Siri, or Google. When the device hears that wake word or thinks it hears it, it begins recording and awaits your verbal commands. Unless you have the microphone or listening feature turned off, your device indeed actively listens for that wake word all the time. 

Here’s where things get interesting, though. There’s a difference between “listening” and “recording.” The act of listening is passive. Your smart speaker is waiting to hear its name. That’s it. Once it does hear its name, it begins recording for a few seconds to record your command. From there, your spoken command goes into the company’s cloud for processing by way of an encrypted connection.  

There are exceptions to when your command may go to the company’s cloud for processing, like Siri on iPhones, which according to Apple, “You don’t sign in with your Apple ID to use Siri, and the audio of your requests is processed entirely on your iPhone.” Also, Google Assistant may process some requests without going to the cloud, like “When a user triggers a smart home Action that has a local fulfillment path, Assistant sends the EXECUTE intent or QUERY intent to the Google Home or Google Nest device rather than the cloud fulfillment.” 

In the cases where information does go to the cloud, processing entails a few things. First, it makes sure that the wake word was heard. If it’s determined that the wake word was indeed spoken (or something close enough to it—more on that in a minute), the speaker follows through on the request or command. Depending on your settings, that activity may get stored in your account history, whether as a voice recording, transcript, or both. If the wake word was not detected, processing ends at that point. 

Enter the issue of mistaken wake words. While language models and processing technologies used by smart speakers are constantly evolving, there are occasions where a smart speaker acts as if a wake word was heard when it simply wasn’t said. Several studies on the topic have been published in recent years. In the case of research from Northeastern University, it was found that dialogue from popular television shows could be interpreted as wake words that trigger recording. For example, their findings cite: 

“We then looked at other shows with a similarly high dialogue density (such as Gilmore Girls and The Office) and found that they also have a high number of activations, which suggests that the number of activations is at least in part related to the density of dialogue. However, we have also noticed that if we consider just the amount of dialogue (in a number of words), Narcos is the one that triggers the most activations, even if it has the lowest dialogue density.” 

Of interest is not just the volume of dialogue, but the pronunciation of the dialogue: 

“We investigated the actual dialogue that produced Narcos‘ activations and we have seen that it was mostly Spanish dialogue and poorly pronounced English dialogue. This suggests that, in general, words that are not pronounced clearly may lead to more unwanted activations.” 

Research such as this suggests that smart speakers at the time had room for improvement when it comes to properly detect wake words, thus leading to parts of conversation being recorded without the owner intending it. If you own a smart speaker, I wouldn’t be too surprised to hear that you’ve had some issues like that from time to time yourself. 

Is someone on the other end of my smart speaker listening to my recordings? 

As mentioned above, the makers of smart speakers make constant improvements to their devices and services, which may include the review of commands from users to make sure they are interpreted correctly. There are typically two types of review—machine and human. As the names suggest, a machine review is a digital analysis and human reviews entail someone listening to and evaluating a recorded command or reading and evaluating a transcript of a written command. 

However, several manufacturers let you exercise some control over that. In fact, you’ll find that they post a fair share of articles about this collection and review process, along with your choices for opting in or out as you wish: 

Setting up your smart speaker for better privacy 

The quickest way to ensure a more private experience with your smart speaker is to disable listening—or turn it off entirely. Depending on the device, you may be able to do this with the push of a button, a voice command, or some combination of the two. This will keep the device from listening for its wake word. Likewise, this makes your smart speaker unresponsive to voice commands until you enable them again. This approach works well if you decide there are certain stretches of the day where your smart speaker doesn’t need to be on call. 

Yet let’s face it, the whole idea of a smart speaker is to have it on and ready to take your requests. For those stretches where you leave it on, there’s another step you can take to shore up your privacy.  

In addition to making sure you’re opted out of the review process mentioned above, you can also delete your recordings associated with your voice commands. 

Managing your voice history like this gives you yet one more way you can take control of your privacy. In many ways, it’s like deleting your search history from your browser. And when you consider just how much activity and how many queries your smart speaker may see over the course of days, weeks, and months, you can imagine just how much information that captures about you and your family. Some of it is undoubtedly personal. Deleting that history can help protect your privacy in the event that information ever gets breached or somehow ends up in the hands of a bad actor.  

Lastly, above and beyond these privacy tips for your smart speakers, comprehensive online protection will help you look out for your privacy overall. In the case of ours, we provide a full range of privacy and device protection, along with identity theft protection that includes $1M identity theft coverage, identity monitoring, and identity restoration assistance from recovery pros—and antivirus too, of course. Together, they can make your time spent online far more secure. 

You’re the smart one in this relationship 

With privacy becoming an increasingly hot topic (rightfully so!), several companies have been taking steps to make the process of managing yours easier and a more prevalent part of their digital experience. As you can see, there are several ways you can take charge of how your smart speaker uses, and doesn’t use, your voice. 

It used to be that many of these settings were tucked away deep in menus, rather than something companies would tout on web pages dedicated to privacy. So as far as smart speakers go, the information is out there, and I hope this article helps make the experience with yours more private and secure.