Microsoft's decision to enable Windows Copilot and Recall as default-on features in consumer and enterprise PCs has triggered immediate pushback from regulators and security experts over continuous screen capture, data retention, and opaque data flows. This creates a concrete governance challenge: boards across sectors must now decide whether and under what conditions to permit always-on, embedded AI assistants in their organizations' core devices, including what data they may capture, how that data is governed, and how to oversee the associated privacy, security, and compliance risks.
The Default-On Dilemma: When Vendor Configurations Become Board-Level Risks
The shift from optional to default-on represents a fundamental change in how organizational boards must approach AI governance. When Microsoft shipped these features as enabled by default across its enterprise PC ecosystem, the company transferred the burden of risk assessment and opt-out decision-making directly to the organizations using those devices.
Previously, boards could reasonably assume that enterprise software configurations fell within the domain of IT departments, which possessed the technical expertise to evaluate and implement vendor offerings. The default-on model disrupts this assumption. An AI feature that continuously captures screen content, stores it locally, and processes it through language models now operates on organizational devices without any deliberate configuration choice by the organization's IT team. The board, as the ultimate governing body for organizational risk, bears fiduciary responsibility for data that flows through these features even when no one in the organization consciously enabled them.
This dynamic forces boards to treat vendor system configurations as governance liabilities. The question is no longer whether the organization has adopted a particular AI tool, but whether the organization's default technology environment includes capabilities that create regulatory exposure, data leakage risk, or compliance violations. Hospital boards must consider patient information captured during clinical workflows. School boards must address student data collected on district devices. Corporate and nonprofit boards must evaluate proprietary communications and donor information that may now reside in vendor-managed data stores.
Data Capture Without Consent: The Governance Gap in Continuous Screen Logging
The core governance problem centers on continuous screen capture operating without meaningful informed consent. Windows Recall specifically is designed to take persistent screenshots of user activity, storing and indexing the content for AI-powered search and retrieval. This means application content, web browsing, email fragments, and communication platform data are captured by default and retained in a searchable format on the local device.
Boards face a significant information gap. The data flows remain opaque. Organizations do not have clear visibility into what information these features capture, how long they retain it, or the precise security of the processing pipeline that analyzes the captured content. The vendors have described technical architectures, but governance requires more than technical descriptions. It requires organizational control over data that the organization did not explicitly direct its IT function to collect.
This gap demands specific governance responses. Boards must establish explicit policies defining which data categories these features may capture within their specific organizational contexts. They must mandate regular audits that verify what data these features actually collect in practice, not merely what the vendor's documentation claims. They must allocate clear responsibility for ongoing monitoring, distinguishing between technical oversight by IT, legal assessment of contractual protections and regulatory exposure, and compliance verification that the organization adheres to its own stated policies.
The challenge crosses organizational functions. A hospital board cannot delegate this solely to its IT committee, because the privacy implications involve patient data protections that the legal and compliance functions must evaluate. A school board cannot delegate solely to administration, because the educational mission and student data protection laws create governance obligations that extend beyond technical implementation.
Procurement as a Governance Lever: Rewriting Contracts for Embedded AI
Boards possess a concrete governance tool in procurement authority. The embedded AI challenge arrives through vendor relationships, and boards can reshape those relationships through updated contract terms and procurement policies.
Traditional IT procurement focuses on functionality, pricing, service levels, and basic data protection commitments. Embedded AI features require a different approach. Contracts must explicitly address continuous AI logging and data use, specifying what data vendor AI features capture, where that data travels, how long it persists, and what rights the organization holds regarding its deletion.
Vendor accountability requires specificity. Boards should require vendors to disclose the full scope of AI-driven data processing, including any third-party services involved in analyzing captured content. Contracts should establish audit rights allowing the organization to verify vendor compliance with stated data handling practices. They should define breach notification obligations specific to AI-captured data, which may present different risk profiles than traditional data breaches.
This procurement approach transforms AI governance from an abstract ethical discussion into a concrete operational requirement. Rather than issuing principles statements about responsible AI use, boards direct their procurement teams to negotiate specific contractual provisions that enforce those principles at the device level.
The governance posture this demands is clear: boards must treat embedded AI features as organizational infrastructure decisions, not as technical defaults to be ignored. They must establish explicit positions on what data capture their organizations will permit, who bears responsibility for monitoring those features over time, and what contractual language will govern vendor AI implementations. The era of assuming that enterprise device configurations fall outside board oversight has ended. The data now flows through devices the organization provides, and the board bears governance responsibility for that flow.