Your AI privacy flag only binds the first hop
Dragos seeded two million canary prompts into AI routers. Keys, URLs and passwords came back from three breached providers, and Zero Data Retention stopped none of it.
A prompt is not a message. It is a document with an audience, and the audience includes every server that decrypts it on the way to a model. That is the uncomfortable takeaway from a five-month experiment published by Dragos on 19 November: nearly two million booby-trapped prompts, each carrying secrets that existed in exactly one place, and the secrets came back — from home broadband connections, from SSH login pages, from AWS accounts nobody had authorised, and once from a company chat in China.
If your product calls a model through a router, a gateway or a marketplace, that is your supply chain. Most teams have never looked at it.
What the experiment measured
HD Moore, Dragos' chief research officer, sent roughly two million prompts between July and November 2026 through widely used AI services. Each carried unique markers: URLs and DNS names never requested before, working AWS keys tied to empty but fully logged accounts, SSH passwords, private git references. Since every secret appeared in exactly one request, any later use of it proved that that prompt had escaped the boundary it was sold under — a policy question answered with evidence instead of paperwork.
Three providers were compromised. Attackers copied requests on arrival — some stolen keys came from prompts the provider rejected before a model ever ran, putting the attacker at the front door rather than deep in the stack. One key seeded in October was still in use three weeks later, and the tooling drifted from a Chrome-impersonating script to the AWS CLI on Kali and finally ARTEX, an LLM-driven pentest framework. Two of its user agents, artex-agent and artex-judge-r121, are worth adding to your detections.
Marketplace resellers harvested prompts without exploiting anything. List a popular model at the lowest price, enable request logging, wait. Moore's team watched a home broadband user open two secret URLs about 36 hours after the prompts passed through one seller. Another attacker hand-typed root SSH attempts with passwords lifted from two prompts. The same model through a different seller was never touched — the leak follows the seller, not the marketplace.
Privacy flags were not honoured. A Chinese AI lab fetched secret URLs from more than half the prompts sent to its models, including about two-thirds of those marked Zero Data Retention or "no data collection," typically two to three days later. Prompts sent to US resellers of that lab's models triggered the same scraper — a customer who picked a US provider to keep data in the US did not get what they paid for. One large US provider kept crawling a URL two months after a single call. And 24 days after one ZDR-marked prompt, its secret link surfaced in a Feishu chat and was opened from six devices in Beijing and Shanghai within two minutes; Dragos had disclosed only the first eight characters, so someone was reading a stored copy.
Why this is an architecture problem
Prompts are encrypted in transit and decrypted at every hop. A router has to read the plaintext to pick a model, so each layer holds a readable copy — in memory, and usually in a log with a retention policy you have never seen.
The path is also not the path you were sold. Moore's team built something like traceroute for inference: image URLs that report which systems fetch them, plus malformed requests whose error messages nest one router's formatting inside another's. Of about two thousand routes traced, several hundred passed through an unidentified system, and a few reached a company that was not on the advertised route. One router was quietly forwarding traffic to a marketplace; one seller appends around a thousand hidden tokens to every prompt.
Then add agents. A chat prompt is a paragraph. An agent's context is the repository, the .env file it can read, the ticket, the database rows, the retrieved documents. One tool call can carry everything the process has access to. We wrote earlier about agents as an attack surface — tool permissions, injected instructions. This is the quieter half: even an obedient agent ships its whole working context to a chain of companies it cannot name.
A checklist we would run
- Classify the payload, not the app. No secrets or personal data: any router is fine. Client source or internal docs: first-tier provider, explicitly chosen. Credentials or personal data: self-host, or do not send it.
- Pin the upstream. Turn off default pools and reseller marketplaces, name providers yourself, then verify the setting changed something — log the provider and region per response and diff it against your config.
- Treat ZDR as a contract clause. Upstream model host named in the DPA, subprocessor list in writing, retention windows per hop. A checkbox in a router dashboard binds the first hop only.
- Seed your own canaries. One unique URL and one fake key per route, alarm on any access. Thinkst Canarytokens are free; that alarm is the cheapest detection you will buy.
- Redact at context assembly. Strip keys, tokens and identifiers before the prompt is built; provider-side secret filtering is a second layer, not the layer.
- Make leaked credentials worthless. Short-lived, scoped credentials in agent sandboxes turn a leak into a nuisance. Attackers here went from key to Secrets Manager to new IAM users; a 15-minute expiry breaks that chain.
- Keep an egress inventory and log the route. Which service, model and country, per feature, recorded with the provider that answered. For Indian clients that inventory is also the cross-border transfer decision under DPDP — a prompt holding a phone number is personal data whatever column it came from.
None of this argues for abandoning hosted models. Moore's framing is the right one: where your prompts go is a decision, and if you do not make it, the lowest price will make it for you. Self-hosting is the only option where nobody else reads the prompt, and the small end is cheaper than it used to be — a DGX Spark lists at $6,950 — but for most teams the honest answer is a short list of named providers, terms that match the marketing, and a canary that tells you when the contract is fiction.
If you are wiring model calls into a product and want a second pair of eyes on the routing, our engineering team does this work. When the answer for a workload is "keep it in-house," that is a private cloud problem more than a prompt problem.
Source: Dragos, "Three breaches, endless leaks: your AI supply chain needs canaries," 19 November 2026, and Ars Technica's coverage of the same research.
Header photo: "Network switches" by ShakataGaNai, CC BY-SA 3.0, via Wikimedia Commons.