[…]
Streaming models send responses to users incrementally, in small chunks or tokens, as opposed to sending the complete responses all at once. This makes them susceptible to an attacker-in-the-middle scenario, where someone with the ability to intercept network traffic could sniff those LLM tokens.
“Cyberattackers in a position to observe the encrypted traffic (for example, a nation-state actor at the internet service provider layer, someone on the local network, or someone connected to the same Wi-Fi router) could use this cyberattack to infer if the user’s prompt is on a specific topic,” researchers Jonathan Bar Or and Geoff McDonald wrote.
“This especially poses real-world risks to users by oppressive governments where they may be targeting topics such as protesting, banned material, election process, or journalism,” the duo added.
Redmond disclosed the flaw to affected vendors and says some of them – specifically, Mistral, Microsoft, OpenAI, and xAI – have all implemented mitigations to protect their models from the type of side-channel attack.
[…]
Proof-of-concept shows how the attack would work
Redmond’s team produced a Whisper Leak attack demo and proof-of-concept code that uses the models to conclude a probability (between 0.0 and 1.0) of a topic being “sensitive” – in this case, money laundering.
For this proof-of-concept, the researchers used a language model to generate 100 variants of a question about the legality of money laundering, mixed them with general traffic, and then trained a binary classifier to distinguish the target topic from background queries.
Then they collected data from each language model service individually, recording response times and packet sizes via network sniffing (via tcpdump). Additionally, they shuffled the order of positive and negative samples for collection, and introduced variants by inserting extra spaces between words – this helps avoid caching interference risk.
[…]
The duo then measured the models’ performance using Area Under the Precision-Recall Curve (AUPRC).
In several of the models, including ones hosted by providers Alibaba, DeepSeek, Mistral, Microsoft, xAI, and OpenAI, classifiers achieved over 98 percent AUPRC, indicating near-perfect separation between sensitive and normal traffic.
They then simulated a “more realistic surveillance scenario” in which an attacker monitored 10,000 conversations, with only one about the target topic in the mix. They performed this test several times, and in many cases had zero false positives, while catching the money-laundering messages between 5 percent and 50 percent of the time. They wrote:
For many of the tested models, a cyberattacker could achieve 100% precision (all conversations it flags as related to the target topic are correct) while still catching 5-50% of target conversations … To put this in perspective: if a government agency or internet service provider were monitoring traffic to a popular AI chatbot, they could reliably identify users asking questions about specific sensitive topics – whether that’s money laundering, political dissent, or other monitored subjects – even though all the traffic is encrypted.
There are a few different ways to protect against size and timing information leakage. Microsoft and OpenAI adopted a method introduced by Cloudflare to protect against a similar side-channel attack: adding a random text sequence to response fields to vary token sizes, making them unpredictable, and thus mostly defending against size-based attacks.
[…]
Source: LLM side-channel attack could allow snoops to guess topic • The Register
Robin Edgar
Organisational Structures | Technology and Science | Military, IT and Lifestyle consultancy | Social, Broadcast & Cross Media | Flying aircraft