New safeguards sharply reduce AI distillation attacks: Anthropic
Sign up now: Get ST's newsletters delivered to your inbox
Anthropic said safeguards restricting access to Claude’s reasoning have sharply reduced extraction attempts.
PHOTO: LIANHE ZAOBAO
- Anthropic said new safeguards cut Claude distillation attacks by 90% to 99%, or “10 to 100 times lower” reasoning tokens extracted.
- The company said it blocked proxy networks and repeated probing, making large-scale copying of Claude’s protected reasoning much harder.
- Anthropic banned tens of thousands of accounts and warned that distillation, including by Z.ai, poses safety and national security risks.
AI generated
SAN FRANCISCO – Anthropic said on Oct 12 that new measures designed to prevent rivals from copying the capabilities of its Claude AI models have reduced data harvested in so-called distillation attacks by about 90% to 99%.
Model distillation has emerged as a key concern for leading artificial intelligence companies due to intensifying US-China competition, with firms including Anthropic, OpenAI and Google arguing that the practice enables rivals to reproduce advanced capabilities more quickly and at a lower cost.
Anthropic said safeguards restricting access to Claude’s reasoning have sharply reduced extraction attempts, with “the number of reasoning tokens extracted by distillers” now “10 to 100 times lower than it was before”.
The California-headquartered company said it had made it harder to use proxy networks to repeatedly probe Claude’s protected reasoning, blocking extraction attempts and hindering large-scale distillation efforts.
Anthropic said its safety system is “roughly as robust as those that prevent Claude from helping to build bioweapons” and has “a false positive rate of less than 0.1%”.
Anthropic banned tens of thousands of accounts linked to distillation campaigns and cited Chinese AI start-up Z.ai, also known as Zhipu, as an example of a company that attempted to distil capabilities from Claude while developing its GLM-5.3 model, warning that such practices pose safety and national security risks.
Z.ai did not immediately respond to a request for comment. REUTERS