Microsoft Purview^™ DLP: how to validate your policies with evidence

A common scene in Purview consoles: DLP policies left in simulation mode for months, endpoint rules set to audit only, and a team convinced that everything is blocked. Nobody lied. Your console shows matches, alerts and charts, without telling you what would have got out.

Validating a Microsoft Purview DLP policy in Microsoft 365 means proving that it detects and blocks data leaving through every location where it is deployed: Exchange, SharePoint, OneDrive, Teams and endpoints. Simulation mode shows what a rule would have triggered, without demonstrating that it stops an exfiltration. For that, you need an active test, replayed with synthetic data, with the result recorded.

What Purview simulation mode tells you, and what it leaves out

Purview lets you run a policy in simulation before switching it on. That is good practice: you measure the volume of matches, spot the noisiest rules and avoid blocking payroll on a Monday morning. Its scope, however, remains limited.

• Simulation works on the activity that exists. If nobody sent a sensitive file outside during the period, the policy had nothing to judge. 0 matches does not mean 0 possible leaks.
• It validates detection. The action itself still has to be proven. Once switched on, a rule can behave differently: a user exception, a block that can be overridden with a simple justification, a priority conflict.
• It ignores paths that are missing from your policy. An unmanaged browser or a device that is not onboarded will never appear in your results.

Put simply, simulation answers “is my rule well calibrated on everyday traffic?”. It does not answer “can a user or an attacker get this file out?”. This is the same trap described in our article on DLP false negatives: a silent rule looks just like an effective one. For the second question, you need a data exfiltration test, aligned with MITRE ATT&CK tactic TA0010.

Blind spots specific to M365

Endpoint DLP: coverage depends on device onboarding

Purview endpoint DLP only protects onboarded devices (Windows 10, Windows 11 and the 3 latest major versions of macOS) that correctly report their status. On a mixed estate, gaps appear quickly: reimaged machines, devices outside the domain, unsupported systems, failed agents. Check 3 things before talking about effectiveness:

1. The list of onboarded devices matches your actual inventory, not just the one in Intune.
2. Monitored activities (copy to USB, printing, upload to a cloud service, clipboard, network share, unauthorised Bluetooth app) are set to block where you think they are, rather than audit only.
3. Restricted browsers are listed consistently. DLP applies natively in Edge, whereas Chrome and Firefox need the Microsoft Purview extension.

Teams: messages, files and guests

In Teams, your DLP acts on chat and channel messages, while shared files live in SharePoint (channels) and OneDrive (chats). Your policy may therefore block a card number pasted into a message and let the same content through in an attached document. Test your conversations with guests or federated users separately: that is where data leaves the organisation.

Sensitive information types and classifiers

Microsoft 365 sensitive information types rely on patterns, proximity keywords and 3 confidence levels (65 for low, 75 for medium, 85 for high). An IBAN, a UK National Insurance number, a US Social Security number or an internal reference will only be detected if the type matches your format and the threshold is right. A “Confidential” label that is not a condition of the rule protects nothing either. Here, the quality of your DLP depends directly on data classification.

A validation matrix by location

Rather than a single global test, build a matrix: 5 rows for the locations, 4 columns for the questions you need to answer, with evidence to back them up. If you are comparing tools to automate it, our criteria for choosing a DLP testing tool will help.

Scroll the table →

A concrete scenario: the HR file that was not supposed to leave

A typical scenario, at a consulting firm that switched on an “HR data” policy in April. In October, the dashboard shows a few dozen matches a month, all on Exchange Online. The team concludes that it works.

The team then replays 4 attempts with a synthetic file that mimics a payroll export: sending through Outlook, OneDrive sharing, a Teams message, copy to USB. The email is blocked. The OneDrive share goes through, because the rule targets a specific SharePoint site and not personal spaces. The Teams message is blocked. The USB copy goes through on part of the estate, the devices that were never onboarded. Result: 2 attempts out of 4 get through. This is exactly what simulation could not show. In ATT&CK, these 2 leaks fall under T1567.002 (cloud storage) and T1052.001 (USB medium).

The verdict is not “DLP does not work”. It comes down to 2 corrections, one of scope, the other of device onboarding. This level of precision then lets you prioritise remediation.

Building usable evidence

Your ISO 27001 auditor, your DPO facing Article 32 of the GDPR or your executive committee will not settle for a screenshot. Usable evidence brings together:

• the policy and rule concerned, with their version and mode (simulation or switched on) on the test date;
• the replayed scenario, described from the defender’s side: location, type of synthetic data, attempted action;
• the observed result (blocked, audited, passed) and the corresponding Purview event, exported, because activity explorer only covers 30 days of data;
• the time it takes for the alert to reach the Microsoft Defender portal, the SOC or the SIEM;
• the date of the next replay.

Checklist before declaring a policy validated

• Every location in your policy has been tested at least once with the policy switched on.
• Exceptions and overrides have been tested as separate paths.
• Teams exchanges with guests have been tested.
• Your tests use only synthetic data, never real data.
• Your gaps are recorded with a corrective action and an owner.
• A replay is scheduled after every configuration change.

FAQ