The rapid evolution of generative AI tools like OpenAI’s ChatGPT and Microsoft’s Copilot has raised concerns about potential privacy and security issues, especially in the workplace. Privacy campaigners have expressed worries about Microsoft’s Recall tool, labeling it a possible "privacy nightmare" due to its ability to take frequent screenshots of users' laptops. The UK regulator, Information Commissioner’s Office, is now urging Microsoft to provide more information about the safety of this feature in its upcoming Copilot+ PCs. Similar concerns surround OpenAI’s ChatGPT, which has also demonstrated the ability to take screenshots in its upcoming macOS app, leading to worries about capturing sensitive data, according to privacy experts. In response to potential risks, the US House of Representatives has banned the use of Microsoft’s Copilot among its staff members, as the Office of Cybersecurity has deemed it a risk to users, potentially leaking House data to unauthorized cloud services. Gartner, a market analyst, has cautioned against using Copilot for Microsoft 365, citing the risks of exposing sensitive data and content both internally and externally. Furthermore, Google had to make adjustments to its AI Overviews search feature after screenshots of misleading and unusual answers to queries went viral. One major challenge for users of generative AI tools in the workplace is the inadvertent exposure of sensitive data. These AI systems often gather vast amounts of information from the internet to train their language models. Steve Elcock, CEO and founder of software firm Elementsuite, highlights that AI companies are hungry for data and make it behaviorally attractive to collect it.

This accumulation of data poses the risk of sensitive information being shared outside of the intended ecosystem and potentially extracted through clever prompting. Moreover, there is the potential threat of hackers targeting AI systems themselves. If an attacker gains access to the language model powering a company's AI tools, they could extract sensitive data, manipulate outputs, or use the AI to distribute malware, warns Camden Woollven, group head of AI at risk management firm GRC International Group. While consumer-grade AI tools bring obvious risks, proprietary AI offerings like Microsoft Copilot, deemed safe for work, also present concerns. Phil Robinson, principal consultant at security consultancy Prism Infosec, points out that such tools could theoretically be misused to access sensitive data if access privileges are not appropriately secured. Employees might request access to pay scales, M&A activity information, or documents containing credentials, which could then be leaked or sold. Another aspect of concern revolves around AI tools being used to monitor employees, potentially infringing upon their privacy. Microsoft's Recall feature claims to keep snapshots locally on the user's PC, emphasizing user control and privacy.