Monday, December 1st 2025
Microsoft Warns: Windows 11 Agentic Features May Hallucinate
A few days ago, Microsoft announced that Windows 11 is undergoing an agentic overhaul. The company indirectly warned that security vulnerabilities may be exposed, and today they issued an updated notice. "As these capabilities are introduced, AI models still face functional limitations in terms of how they behave and occasionally may hallucinate and produce unexpected outputs." After installing Windows 11 Build 26220.7262, you'll find a new toggle for "Experimental agentic features" in Settings > System under "AI Components." Fortunately, this is an optional feature and must be enabled manually.
Upon turning the feature on, Windows will show a warning that these capabilities are experimental and might affect your device. In practice the security risk is the bigger concern. New attack techniques tied to autonomous agents are already appearing, with cross-prompt injection standing out. In those attacks, malicious instructions are concealed inside ordinary documents or interface elements so an agent follows them instead of its original task. That could allow an agent to install malware, leak payment details, or carry out other harmful actions.Microsoft says these agents run inside an "Agentic Workspace," where each agent gets a scoped, auditable account and its actions are recorded for later review. The company compares this to Windows Sandbox, but agents are designed to persist and can continue to act on files across sessions, which expands the possible attack surface. By default an agent may be granted read and write access to common folders such as Downloads, Desktop, Documents, Pictures, Music and Videos. Even with limited action sets and separate execution accounts, those defaults leave gaps until Microsoft implements stronger protections like finer-grained permissions and robust defenses against prompt injection.
Sources:
via Windows Latest, Microsoft
Upon turning the feature on, Windows will show a warning that these capabilities are experimental and might affect your device. In practice the security risk is the bigger concern. New attack techniques tied to autonomous agents are already appearing, with cross-prompt injection standing out. In those attacks, malicious instructions are concealed inside ordinary documents or interface elements so an agent follows them instead of its original task. That could allow an agent to install malware, leak payment details, or carry out other harmful actions.Microsoft says these agents run inside an "Agentic Workspace," where each agent gets a scoped, auditable account and its actions are recorded for later review. The company compares this to Windows Sandbox, but agents are designed to persist and can continue to act on files across sessions, which expands the possible attack surface. By default an agent may be granted read and write access to common folders such as Downloads, Desktop, Documents, Pictures, Music and Videos. Even with limited action sets and separate execution accounts, those defaults leave gaps until Microsoft implements stronger protections like finer-grained permissions and robust defenses against prompt injection.

68 Comments on Microsoft Warns: Windows 11 Agentic Features May Hallucinate
In yet other news: Microsoft doesn't give a crap about user safety, and can't be asked to properly test updates before rolling them out.
Brilliant!
I am so close to air-gapping my PC. Blacklist firewall everything. Only allow an obscure browser in a VM to punch through on port 80, with a whole range of blacklisted IP addresses. Basically MS ones.
I can only hope that some hackers really bring half the internet down for days incl. multiple government ministries so that MS finally gets public egg on face and receives multiple injunctions from multiple governments to STOP. Governments have to recognise that W and internet is now a basic utility, after water, electric, etc. and requires regulation. A windows lean, without bells and whistles and agentic crap, should become a requirement. Gvts would never allow water utilities to "experiment with the water".
And I can't wait for the AI bubble to burst so there is no longer "share price valuation metrics" driving these stupid decisions.
That's the biggest issue, we can't expect everyone to be competent enough.
With agentic Copilot MS are giving those incompetent users more tools to break their system with the help from the faking "expert" AI.
Giving confidence and no limits to people who have no clue what they're doing is not a great feature.
What have trees ever done for me? The day trees start paying tax and contributing to the NHS, is the day I'll start caring.
/s
But, then I ask the question...... won't the hacker/whatever program can already order 200 pizza with or without Windows recall/AI?
Sorry, I'll see myself out.
Not only AI tools write stupidly inaccurate and misleading summaries, as exposed by Steve Burke from Gamers Nexus yesterday, but it also hallucinates. In common language, it makes up $hit due to training algorithms that have bad data triangulation system built in. In other words, it has serious methodological flaws as to how data are sampled and cross-checked.
Also, there is no equivalent of peer review system in AI world, unlike in real world science, to spot those mistakes and eliminate it before replaying to users.
AI needs a psychotherapy, a deep one...
If all he/she wants to do is just make you foot the bill for his pizza sure.
if they want to turn your life in to a nightmare having local AI gas light you in to suicide or worse is far more interesting.
If you want control of the OS, Windows hasn't been an OS that lets you have control for a long time.