The shift from assistant to operator
For a long time, public discussion treated artificial intelligence as an assistant. You ask, it answers. You upload a document, it summarizes it. You request a draft, it proposes one. That framing is already outdated. The real security issue begins when AI stops waiting for instructions and starts operating as a persistent agent with memory, tools, browser control, external triggers, and the ability to act while no human is watching.
At that point, the system is no longer just a productivity layer. It becomes an operational layer. And when that operational layer is connected to citizen data, government systems, internal policy workflows, procurement files, legal drafts, or politically sensitive material, the risk is not accidental. It is structural. The danger does not arise because someone used the tool badly. It arises because the architecture itself concentrates access, initiative, and decision pathways in one opaque mechanism.
An always-on agent changes the security model of an organization. Traditional software usually acts when a user explicitly invokes it. An autonomous agent can monitor, interpret, remember, prioritize, and execute. It can react to a new email, a changed spreadsheet, a webhook, a browser event, or a connector update. That means the attack surface expands in every direction at once. The system is no longer defending one application or one database. It is defending a live chain of perception, reasoning, tool use, and action.
Why access plus autonomy becomes a governance problem
The common mistake is to think that the main question is whether the agent can see sensitive data. That matters, but it is not the whole issue. The deeper question is what the agent can do with what it sees. If it can combine data across tools, follow instructions embedded in untrusted content, operate a browser, trigger API calls, or move through internal environments, then it can become a vehicle for privilege escalation, silent exfiltration, and policy distortion.
This is why autonomous agents are categorically different from static software or ordinary chat interfaces. They are vulnerable not only to ordinary security flaws, but also to prompt injection, hidden instruction hijacking, tool misuse, context poisoning, and persistent manipulation across sessions. An email can become an exploit path. A document can become a control surface. A web page can become an instruction channel. A connected system can become both data source and action target.
Inside government, the implications are even more serious. A civil servant’s inbox is not just a mailbox. It is a stream of legal questions, constituent problems, internal coordination, draft positions, and unresolved decisions. A ministry folder is not just storage. It is institutional memory. A policy brief is not just text. It is strategic intent in formation. When an always-on proprietary agent sits across all of this, it does not merely assist the state. It intermediates the state.
That is why this is not just a cybersecurity issue. It is a democratic governance issue. A system that accumulates organizational memory, acts across connected tools, and remains operational in the background can deepen vendor lock-in far beyond software licensing. The organization does not only depend on the provider’s model. It begins to depend on the provider’s invisible defaults, permission design, extension model, logging decisions, and hidden roadmap.
The Claude Code leak made the problem visible
The recent Claude Code source code leak mattered for more than competitive gossip. What made it important was that it exposed how much can remain invisible in a closed system until an accident reveals it. The public reporting around the leak pointed to 44 hidden feature flags. That number is not just a detail for developers. It is a governance lesson. It shows that critical capabilities can exist inside a system long before users, auditors, customers, or public institutions understand what is actually there.
That is exactly why open source matters here. Not because open source is magically secure. It is not. Open code can still contain serious flaws, and history proves that. But open source changes the balance of visibility. It allows independent inspection, adversarial review, reproducibility, and community scrutiny. Hidden switches do not remain hidden in the same way. Assumptions can be tested. Risk can be debated in public. Vulnerabilities can be found by people other than the vendor.
That difference matters enormously when the software sits close to citizen records, administrative systems, or national policy workflows. In those environments, opacity is not a premium feature. It is a liability.
What public institutions should demand
Public institutions should not respond by pretending autonomous agents can be banned out of existence. They should respond by refusing opaque autonomy in sensitive domains. Any agent with access to citizen data, internal systems, or policy-sensitive information should operate under strict least-privilege rules, full audit logging, clearly defined approval gates, and meaningful independent oversight. Where possible, the code and extension layers should be open for inspection. Where that is not possible, there must at least be enforceable audit rights, technical transparency, and exit paths.
The core principle is simple. The more autonomy a system has, the more visibility and accountability it must provide. The more systems it can touch, the less acceptable a black box becomes. An always-on AI agent in government is not merely a smarter assistant. It is a new concentration of operational power. In a democracy, power of that kind must be inspectable, contestable, and politically accountable from the start.
Sources
- TestingCatalog, “Exclusive: Anthropic tests its own always-on Conway agent”. Useful for the publicly reported picture of Conway as a standalone environment with webhooks, Chrome linkage, extensions, and event-driven behavior rather than a normal chat interface:
https://www.testingcatalog.com/exclusive-anthropic-tests-its-own-always-on-conway-agent/ - The New Stack, “Inside Claude Code’s leaked source: swarms, daemons, and 44 features Anthropic kept behind flags”. Useful for the public reporting that the Claude Code leak exposed 44 feature flags, which is central to the argument about opacity, hidden capabilities, and governance risk in proprietary agent systems:
https://thenewstack.io/claude-code-source-leak/ - Anthropic official safety materials and the OWASP AI Agent Security Cheat Sheet. Useful for documenting that autonomous agents introduce real risks tied to human oversight, prompt injection, computer control, tool abuse, and data exfiltration, and that these risks are already recognized in both vendor and security-community guidance:
https://www.anthropic.com/news/our-framework-for-developing-safe-and-trustworthy-agents,https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool,https://www.anthropic.com/transparency&https://cheatsheetseries.owasp.org/cheatsheets/AI_Agent_Security_Cheat_Sheet.html.