Turn CISA ScubaGear into a governed Microsoft 365 drift assessment

Source fact: ScubaGear assesses configuration

CISA’s ScubaGear repository describes a three-step process: PowerShell queries Microsoft 365 APIs, Open Policy Agent compares returned settings with policies derived from CISA’s Secure Cloud Business Applications baselines, and the tool produces HTML, JSON, and CSV reports. Current coverage includes Microsoft Entra ID, the Microsoft security suite, Exchange Online, Power BI, Power Platform, SharePoint, and Teams.

CISA developed SCuBA principally for federal cloud security, while also recommending that other organizations review the baselines and apply practices where appropriate. A result is therefore not, by itself, a private-sector certification, a complete Microsoft 365 security assessment, or proof that every control is appropriate to a particular tenant.

Source fact: assessment context changes the result

The official configuration documentation supports a YAML or JSON file for product selection, environment details, exclusions, annotations, and policy omissions. It warns that exclusions and omissions can introduce blind spots and says omissions should be approved through the organization’s risk-management process. Rationales and expiration dates can be recorded. The repository also advises users to review prerequisites, permissions, updates, and product-specific limitations.

These facts matter when comparing two reports. A changed result might reflect tenant drift, a new ScubaGear release, a revised SCuBA baseline, different permissions, a changed configuration file, an API response difference, or a corrected policy implementation. Calling every difference tenant drift would overstate what the evidence proves.

DSE recommendation: authorize and freeze the assessment context

Give each run an owner, approved scope, tenant identifier, purpose, and protected execution location. Use only the documented permissions needed for the selected products. Separate interactive testing from any unattended identity, protect its certificate or other authentication material, and remove unnecessary access when scheduled collection is not required. Treat the reports as sensitive because they can expose security configuration and exceptions.

Preserve a run manifest with UTC time, ScubaGear version, baseline or policy version, dependency versions, selected products, environment, invoking identity, configuration-file hash, command parameters, completion status, and output hashes. Store the original HTML, JSON, CSV, console log, and configuration together under access control. This creates reproducibility without claiming the tool captured every relevant setting.

DSE recommendation: turn findings into governed decisions

1. Validate: confirm collection succeeded and required permissions were available. Review unsupported, error, warning, omitted, and indeterminate results before counting pass or fail.
2. Contextualize: map each material result to the exact SCuBA policy and current tenant design. Confirm whether a third-party control, licensing boundary, emergency account, or service dependency changes the interpretation.
3. Decide: assign a technical owner and risk owner. Record remediate, accept temporarily, not applicable, or tool result incorrect, with evidence and approval. Time-bound exceptions and name their review trigger.
4. Change safely: test Microsoft 365 changes with representative users and dependent applications. Use normal change approval and rollback planning; ScubaGear is an assessment tool, not permission to change production automatically.
5. Prove closure: rerun the affected products, preserve the new evidence, and link the result to the approved change and original finding.

Before remediation, DSE recommends comparing each finding with current official Microsoft documentation for the tenant’s licensed services and recording the setting’s actual enforcement scope. Controls can interact across products or depend on licensing. Capture those dependencies so a passing point result is not mistaken for end-to-end enforcement.

Measure drift without losing the baseline

Run again on an organization-defined schedule and after material tenant, licensing, identity, or baseline changes. Compare normalized policy identifiers and results, but retain both manifests. Distinguish a tenant change from a tool or policy change in the review record. New checks should enter triage rather than being silently treated as old failures; removed checks should not disappear from the risk register until their disposition is understood.

This operating model differs from DSE’s Microsoft Secure Score article. Secure Score is a Microsoft-maintained improvement work queue. ScubaGear compares observed settings with CISA policy logic. Neither substitutes for threat modeling, application testing, licensing review, or accountable risk acceptance.

Official sources

• CISA Secure Cloud Business Applications project
• CISA ScubaGear official repository
• ScubaGear configuration-file documentation
• CISA announcement of Microsoft 365 SCuBA baselines