Microsoft 365’s new security model: How to use phishing simulations and security mailboxes

We live in a dangerous world. You only have to look at the headlines to see yet another ransomware attack or someone who’s had their savings stolen by phishers. Criminals have found the digital world rich pickings, with email their route into your systems and your bank accounts.

That’s partly our fault as an industry. We’ve got all manner of security tools in our applications and services, but most of the time we don’t use them. Maybe the software ships in a low-security mode, or maybe we remove controls to make things comfortable for our noisiest users. Most of the time it’s the first; it can be complicated to add security to a running system without affecting how everyone does their jobs—in some cases even the security team.

Enhancing security in Microsoft 365

So, what happens when the overrides we’ve chosen suddenly go away? That’s happening to Microsoft 365 as it moves it to a “secure by default” model. It’s a process the initial notification described as Microsoft taking responsibility for its role as a security service and acting “on your behalf to prevent your users from being compromised.” As the process continues to roll out, one of the most obvious effects will be on security teams testing their systems and their staff.

SEE: Security Awareness and Training policy (TechRepublic Premium)

One of the areas where security is being tightened is email delivery. That’s not surprising: Email is a major delivery method for malware and for financial attacks, often using carefully crafted phishing emails to attract targets to malicious sites or to gain access to credentials or financial transactions.

Microsoft is tightening the rules used to block and quarantine malicious emails, extending its malware blocks to phishing emails. It’s an important change, using Microsoft’s security graph to build a model of phishing messages that’s good enough to identify them with high confidence. If you’ve not got any overrides in place, messages marked with “high confidence phish” tags will already automatically move into Exchange Online’s quarantine folder for inspection before delivery or deletion.

As an aside, it’s important to report messages that have been mis-classified using the tools in the quarantine system. Microsoft’s mail security tooling is a massive machine learning project, built using signals from the Microsoft Graph. It’s continuously learning, based on signals from users reporting junk mail, malware payloads and phishing using the tools built into the various versions of Outlook. That includes messages coming out of Outlook’s junk mail folder as well as those being marked as junk.

SEE: Windows 10: Lists of vocal commands for speech recognition and dictation (free PDF) (TechRepublic)

Microsoft 365’s quarantine tools have a similar function, with a somewhat higher weighting in the rules. Messages sent to users from quarantine can be reported to Microsoft as safe, allowing it to use that as an additional signal in its machine learning training. 

The new secure-by-default stance means that any existing mail rule overrides you’ve put in place will be ignored. This will block high-confidence phish messages from allowed sender or domain lists, from allowed IP addresses, and from Outlook-safe senders. Microsoft is now extending this by removing overrides from mail transport rules. Lower-confidence messages, like spam, can still be managed by overrides, but it’s recommended to allow Microsoft’s tools to handle them for you.

Getting in the way of security?

While this approach makes users safer, it understandably causes issues for security teams, as mail transport overrides were a recommended way of doing things and in some cases were used for regulatory compliance. Microsoft has already held the change back from its initial June deadline to the end of August 2021, with rollout due to be completed by the end of September, so it’s time to start making changes if you’ve not looked at this yet.

It’s clear that this final stage of the process could cause some security teams issues, as Exchange’s mail flow system is often used to manage attack simulations and to route suspicious messages to third-party security tools and custom security mailboxes that aren’t part of Exchange’s mail quarantine tool. Microsoft is providing some workarounds, with the launch of a new Advanced Delivery Policy.

How to use Advanced Delivery Policies

Advanced Delivery Policy is a powerful tool, as it stops messages defined by the policy from being filtered, using specific overrides for phishing simulations and for specific security mailboxes. The policy can only be managed by users with the Security Administrator and Organization Management roles. This sensibly limits access to a very limited subset of users, reducing the risk of compromise.

Setting up a phishing simulation allows you to configure how you will run a phishing drill on your organization. This locks down specific details of a message, ensuring that only your simulated phish will get delivered. Any simulation rule needs the sending domain, its IP address, and a list of URLs that will be in your message. You can have up to 10 of each, allowing you to construct a series of different phishes to send to different groups of users from different sources.

The list of protected URLs is important: URLs in detected phishes are normally blocked by SmartScreen, as well as automatically inspected in a sandbox. Configuring this setting stops Exchange Online’s security tools from trapping them.

You can use either the Microsoft 365 portal to set up and manage Advanced Delivery Policies or you can use PowerShell to configure SecOps override policies. These define mailboxes used for security purposes, with tools to check rules and remove invalid ones, for example adding additional security mailboxes. Similarly, you can use PowerShell to configure phishing simulations. There’s no difference between using the web portal and PowerShell, however the PowerShell option gives you the ability to use scripts to create new policies before running a test, removing them after it’s complete, and ensuring that the same features can be used for future tests.

One other option for a complete opt out is to use Microsoft Defender for Office with non-Microsoft MX record. This allows you to pass messages through another relay before delivering them to Exchange, blocking secure-by-default from operating. This allows you to use third-party filtering services, without the confusion of having mail filtered more than once.

Making Microsoft 365 secure by default (or as near as possible) is essential in today’s threat landscape. Whie that can be awkward for some users, the overall benefits are clear, and there are workarounds for certain essential services. It may be more work to configure your Microsoft 365 instances, but protecting users from phishing attacks is worth some extra configuration.

Microsoft Weekly Newsletter

Be your company’s Microsoft insider by reading these Windows and Office tips, tricks, and cheat sheets.
Delivered Mondays and Wednesdays

Sign up today

Also see