AI model distillation moves from terms of service to a joint advisory

NSA, CISA and the FBI released joint advisory AA26-251A on 8 September, naming industrial-scale AI model distillation by China-based developers as a threat to U.S. AI companies. Per the advisory, the campaigns have run since at least late 2024, and six companies (DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI) "extracted billions of tokens across millions of exchanges/requests from U.S. frontier AI models," likely with Chinese government awareness. When three agencies co-sign a warning about copying a model through its own answers, the problem has moved from the terms-of-service desk to counterintelligence.

The advisory's framing is blunt: "Distillation is not a supplement to these companies' AI model development, but the critical core of it." Part of the tradecraft is commercial, not technical. The companies "create user accounts obfuscating their country of origin" and buy premium subscriptions in bulk, in campaigns spanning "days to months with query volumes in the thousands to millions per domain," per the agencies. Named targets include multiple Claude, GPT, Gemini and Grok models.

The seam is identity, not malware. A distillation campaign presents as paying customers: accounts that obscure their country of origin, bulk premium subscriptions, and query volumes sustained for days to months. Nothing breaks while it runs. The advisory lists per-account tells, such as new subscriptions that run at maximum usage at once. The campaign itself appears when registration, subscription and query signals are correlated across accounts and, as the advisory urges, across providers.

• Vet high-volume API and subscription customers with the rigor of a foreign-ownership review.
• Correlate registration, payment and usage signals across accounts.
• Write a playbook for suspected distillers, including the advisory's option to subtly alter responses.
• Open an indicator-sharing channel with peer providers before the next campaign.

OBSIDIAN — Counterintelligence is designed for the program-protection side of this problem: mapping what an organization protects, here its models and research, to the threat actors and collection priorities aimed at it, and supporting beneficial-ownership and foreign-ownership screening of the resellers, aggregators and partners it trusts with access. Detecting abuse inside a provider's own API traffic stays with its fraud and trust-and-safety teams. CI officers validate each alert and choose countermeasures; OBSIDIAN is designed to give them early warning.

Frontier models now sit beside the chip trade secrets and restricted AI hardware at the center of the summer's semiconductor espionage cases. The practical shift is ownership. The controls that counter distillation live with fraud, trust-and-safety and counterintelligence teams, and they work best when those teams share what they see.