Protecting Data in an AI-Enabled Workplace
In the previous post, I explored what it means to take responsibility for AI-assisted decisions.
That responsibility begins before a professional accepts or rejects an AI recommendation. It begins even earlier, when someone decides what information to provide to the system.
Generative AI tools invite us to share context. The more information we provide, the more useful the response often becomes. We paste in documents, describe situations, upload files, summarize conversations, and provide examples. We may include names, dates, internal processes, financial information, customer concerns, medical details, student records, or unpublished work.
From the user’s perspective, this can feel like entering information into a private workspace. The interaction looks like a conversation. The system responds directly to us. The exchange may appear to disappear when we close the window.
That experience can make it easy to forget that we are providing information to an external system governed by technical configurations, contracts, retention practices, access controls, and organizational policies that may not be visible to us.
This creates one of the most immediate professional risks graduates will face in an AI-enabled workplace: They may expose information before they realize it needs protection.
When we teach students about AI and privacy, we often rely on a simple warning: Do not enter sensitive information.
That is good advice, but it assumes students already know what counts as sensitive. In practice, that judgment can be difficult. Some information is obviously protected. Medical records, Social Security numbers, financial account details, passwords, and confidential personnel records should raise immediate concerns.
But professional information does not always arrive with a label that says “sensitive.” A customer complaint may contain identifying details. Meeting notes may reveal an upcoming organizational decision. A résumé may contain personal information about someone who never agreed to have it entered into an AI system. A draft contract may include confidential terms. A troubleshooting log may contain usernames, network details, IP addresses, or security vulnerabilities. A student email may reveal information about disability, health, family circumstances, or academic performance. An unpublished research document may contain intellectual property or identifiable participant data.
Even information that appears harmless in isolation can become sensitive when combined with other information. Privacy is therefore not simply about identifying a short list of prohibited data types. It requires understanding context, relationships, purpose, and potential consequences.
This is particularly important because workplace AI use often involves information about other people. When I enter my own information into an AI system, I am making a privacy decision that primarily affects me. When I enter an email from a colleague, a record from a customer, a student’s assignment, an applicant’s résumé, or notes from a client meeting, I am making that decision on someone else’s behalf.
That person may not know the information was shared. They may not have consented to the use. They may have provided the information for an entirely different purpose. They may have trusted the organization to protect it.
This is sometimes described as a networked privacy problem. Privacy decisions are not always individual because one person’s use of technology can expose information about other people.
Research examining generative AI use in professional settings found that workplace privacy concerns extend beyond the individual employee using the tool. Users regularly work with information involving colleagues, customers, employers, and other third parties. Their choices can therefore create risks for an entire network of people who may have no knowledge of or control over the disclosure. (Lee et al., 2025)
This changes how we should think about AI literacy. Knowing how to protect our own information is not enough. Students also need to recognize when they are acting as stewards of someone else’s information. That stewardship will be part of professional responsibility across nearly every field. A healthcare worker does not own the patient information they handle. An instructor does not own the personal circumstances students share with them. A human resources employee does not own the applicant information contained in a résumé. A technician does not own the organizational data that appears in a support ticket. Access to information does not automatically include permission to provide it to an AI system.
This distinction may be unfamiliar to students because many academic activities ask them to work primarily with their own writing or publicly available information. When they enter the workplace, they may suddenly gain access to records, communications, systems, and data collected for specific organizational purposes.
The question is no longer only, “Is this information private?” It becomes, “Am I authorized to use this information in this way?” That is a much more professional question.
An organization may provide an enterprise AI tool with contractual protections, restricted data use, administrative controls, and different retention settings. Another tool may be a public consumer service governed by very different terms. Both may have a nearly identical chat interface.
Students may assume that because the tools look and behave similarly, the privacy risks are also similar. But the visible interface tells them very little about how information is managed behind it. Even access to an organization-approved tool does not mean every kind of data can be entered into it. Approval is not universal permission.
The appropriate use may depend on the type of information, the purpose of the task, the system’s configuration, the organization’s policy, contractual commitments, and laws governing the profession. This is why “use only approved tools” is necessary but incomplete guidance.
Graduates must also know how to ask what the tool is approved to do. Can it process personally identifiable information? Can it be used with internal documents? Are prompts or uploaded files retained? Can organizational data be used to improve the underlying model? Who can access interaction histories? What happens when information needs to be corrected or deleted? Does the system connect to other applications or data sources? Can an AI agent take information from one system and transfer it into another?
These questions will become more important as AI moves from isolated chat tools into connected workplace systems.
An employee might understand that pasting a confidential document into a public chatbot is risky. But the privacy implications can become less visible when AI is built into email, document storage, customer management software, learning management systems, or meeting platforms. The user may not feel as though they are sharing information externally. They may simply click a button labeled “summarize,” “draft,” “analyze,” or “recommend.”
AI becomes easier to use at the same time that the movement of data becomes harder to see.
Research with knowledge workers in data-sensitive environments has identified this tension. Employees often recognize the potential value of generative AI but remain uncertain about whether particular tools are safe for classified, proprietary, or personally identifiable information. That uncertainty can lead either to risky experimentation or to avoiding useful tools altogether. (Wagman and Parks, 2025) Neither response is ideal.
The goal should not be to make students afraid of providing any context to AI. Without appropriate context, the systems may be far less useful. The goal is to help students make deliberate decisions about what context is genuinely necessary. One useful principle is data minimization: provide only the information required for the legitimate purpose of the task.
The National Institute of Standards and Technology includes data minimization among its recommended privacy controls for AI systems. It also emphasizes considering whether datasets contain sensitive or confidential information, whether they could expose people to harm, and what measures have been taken to reduce that risk. (NIST AI Risk Management Framework Playbook)
For an individual user, data minimization can begin with a simple question: What is the least information the system needs to help me with this task? A professional may not need to include a customer’s name to improve the wording of a response. They may not need to upload an entire report when a short description of its structure would be sufficient. They may be able to replace specific details with fictional or generalized ones. They may be able to extract a small, non-sensitive portion of a document rather than providing the complete file. They may determine that the information cannot be adequately de-identified and that AI should not be used for the task.
But anonymization also requires care. Removing a name does not automatically remove identity.
A combination of job title, location, department, age, dates, or unusual circumstances may still make someone recognizable. AI systems may also infer information that was not explicitly provided. NIST notes that AI can create new privacy risks by identifying individuals or previously private information through inference. (NIST AI RMF 1.0)
Students therefore need to move beyond asking whether they included a direct identifier. They need to consider whether the information could reasonably be connected back to a person, organization, project, or event. This kind of judgment is difficult to develop through warnings alone. Students need practice.
They could examine realistic professional scenarios and decide what information could be entered into an AI system, what should be removed, what could be generalized, and what should prevent AI use altogether. A business student could evaluate whether customer data is necessary for an AI-assisted analysis. A healthcare student could determine whether a patient scenario has been sufficiently de-identified. An information technology student could review a troubleshooting log for credentials, network details, and security information before requesting AI support. An education student could consider whether student work, feedback, or accommodation information can be processed by a particular tool.
The goal would not be to memorize one universal rule. The legal, contractual, and professional requirements will differ across contexts. The goal would be to develop a habit of pausing before sharing. Whose information is this? Why do I have access to it? What permission do I have to use it? What does the AI system actually need? What could happen if the information were retained, exposed, combined with other data, or used for an unintended purpose? Is there a safer way to complete the task?
These questions position privacy as part of professional judgment rather than as an obstacle to productivity. They also reveal an important tension in AI-enabled work. The easiest way to receive a highly tailored response is often to provide more context. But the easiest path is not always the responsible one. A professional must balance usefulness against exposure.
That balance will become even more complicated with AI agents.
A chatbot receives the information a user deliberately enters. An agent may be given permission to access email, documents, calendars, databases, customer records, or other organizational systems. It may gather and combine information without asking the user to copy and paste each piece.
In that environment, data protection will no longer depend only on what someone types into a prompt. It will depend on what permissions the agent receives, what systems it can access, what actions it can take, and whether its movement of information can be monitored.
Students graduating into this world will need to understand that convenience is often created through access. An AI system can assist more effectively when it can see more of the work. But access creates responsibility.
The ability to retrieve information does not mean the system should retrieve it. The ability to combine information does not mean that combination is appropriate. The ability to act across systems does not mean it should be granted unrestricted authority.
These will not be questions only for cybersecurity or privacy specialists. They will be ordinary workplace questions encountered by people throughout an organization. That is why protecting data must become part of broad career preparation. We are not preparing students simply to keep secrets. We are preparing them to recognize that professional information is entrusted to them for a purpose. AI does not erase the obligations that came with that trust.
Before students ask what AI can do with the information available to them, they need to ask whether that information should be provided at all. And even when information is protected, another risk remains.
If AI performs more of the work, will graduates retain enough professional knowledge to recognize when it is wrong?
That is where the next post will turn.
Continuing the Conversation
Series 1: AI Is Exposing Existing Problems ✓ Completed
Series 2: What We Do About It ✓ Completed
Series 3: Cultivating Human Thinking in an AI World ✓ Completed
Series 4: Learning Alongside AI ✓ Completed
Series 5: Preparing Students for an AI World
Current Post (4 of 8): Protecting Data in an AI-Enabled Workplace
Next Up: Working With AI Without Losing Professional Expertise