US Agencies Warn of AI Model Distillation: How Companies Can Protect Proprietary AI Systems

US national security agencies have issued a joint warning about large-scale efforts to extract capabilities from American frontier artificial intelligence models.

The National Security Agency, Federal Bureau of Investigation and Cybersecurity and Infrastructure Security Agency said China-based artificial intelligence companies have engaged in high-volume model distillation activity designed to replicate or approximate the capabilities of advanced US AI systems.

The agencies stressed that model distillation itself is a legitimate and widely used machine-learning technique.

The concern arises when large volumes of queries are used to systematically extract proprietary capabilities from restricted models in ways that may violate platform controls, contractual restrictions or security safeguards.

For US companies developing or deploying advanced AI systems, the advisory highlights an emerging cyber security risk: valuable intellectual property may increasingly exist not only in source code and datasets, but also in the behavior and capabilities of the models themselves.

What Is AI Model Distillation?

Model distillation is a machine-learning technique in which a smaller or different model is trained to reproduce some of the outputs or behavior of a more capable model.

Used legitimately, it can help organizations create more efficient systems.

However, the same concept can also be abused.

A threat actor may submit large numbers of carefully designed prompts to a target model and use the responses to train another system.

Over time, this can allow the second model to reproduce parts of the target model’s functionality without having direct access to its underlying weights, training data or source code.

This creates a different type of intellectual property and cyber security challenge.

The information being extracted may not exist as a single downloadable file.

Instead, the value is derived from repeated interaction with the system.

Why US Agencies Are Concerned

According to the joint advisory, the activity identified by US agencies involved large-scale and coordinated attempts to obtain restricted capabilities from frontier AI models.

The agencies said these efforts can involve:

• High-volume automated querying.
• Multiple accounts or identities.
• Distributed infrastructure.
• Attempts to bypass usage restrictions.
• Prompt strategies designed to reproduce model behavior.
• Collection of outputs for use in training another model.

This type of activity can be difficult to distinguish from legitimate heavy usage unless organizations are actively monitoring account behavior and technical patterns.

The concern is therefore not simply whether unauthorized access has occurred in the traditional sense.

A system may remain online and operational while its capabilities are being systematically harvested through apparently valid interactions.

AI Systems Create a New Form of Proprietary Exposure

Traditional cyber security has often focused on protecting files, credentials, networks and databases.

Advanced AI systems introduce another category of valuable corporate asset.

That can include:

• Model behavior.
• Training methodologies.
• Proprietary prompts.
• Fine-tuning data.
• Safety systems.
• Evaluation frameworks.
• Model weights.
• Internal APIs.
• Specialized capabilities developed through significant investment.

An organization may protect its source code and internal infrastructure effectively while still exposing valuable model capabilities through public or partner-facing interfaces.

This means cyber security controls need to consider not only who can access a system, but also how that access is being used.

What Does Suspicious Model Extraction Look Like?

No single pattern proves that model extraction is taking place.

However, organizations operating advanced AI systems may need to assess combinations of indicators such as:

• Extremely high query volumes.
• Repetitive or systematically varied prompts.
• Multiple accounts displaying similar behavior.
• Sudden changes in usage patterns.
• Automated interaction at unusual scale.
• Requests designed to probe model boundaries.
• Activity distributed across multiple IP addresses.
• Repeated attempts to access restricted functionality.

The objective is not to treat every high-volume user as malicious.

It is to identify activity that is inconsistent with normal commercial use and warrants closer review.

Why Access Controls Alone May Not Be Enough

A company may require users to create accounts, accept terms of service and authenticate before accessing a model.

Those controls are important, but they may not be sufficient if an adversary can create multiple accounts or distribute activity across different locations.

This is where layered controls become important.

Organizations may need to combine:

• Account verification.
• Rate limiting.
• Behavioral monitoring.
• API restrictions.
• Anomaly detection.
• Device and network intelligence.
• Usage thresholds.
• Contractual controls.
• Investigation of suspicious activity.

The appropriate controls will depend on the nature of the system and the sensitivity of the capabilities being exposed.

Conflict International USA’s Cyber Security services can support organizations where suspicious access, data exposure or potential misuse of digital systems requires investigation.

Protecting Proprietary AI Requires More Than Traditional Network Security

A company can have strong perimeter security and still face model-extraction risk.

That is because the threat may occur through authorized access to a public-facing or partner-facing system.

This changes the security question.

Instead of asking only whether someone broke into the network, organizations may also need to ask:

• Is the user behaving in a way that suggests systematic extraction?
• Are account controls being bypassed?
• Are multiple accounts linked to the same activity?
• Are model outputs being collected at unusual scale?
• Are restrictions being deliberately tested or evaded?

This requires close coordination between cyber security, engineering, legal and product teams.

Commercial Relationships Can Also Create Exposure

Not all AI access is public.

Some organizations provide model access to customers, suppliers, research partners or commercial collaborators.

These relationships can create additional risks if access levels are not clearly defined or monitored.

Organizations should consider:

• What capabilities each third party needs.
• Whether access is limited appropriately.
• Whether usage can be monitored.
• Whether technical restrictions match contractual restrictions.
• Whether suspicious behavior can be investigated quickly.
• Whether access can be revoked when necessary.

This is particularly important where a partner receives broader access than a normal customer.

Incident Response for Suspected AI Model Extraction

If an organization suspects that proprietary model capabilities are being extracted, the response should be evidence-led.

Potential steps may include:

• Preserving logs.
• Identifying affected accounts.
• Reviewing query patterns.
• Mapping linked infrastructure.
• Assessing whether multiple accounts are connected.
• Determining what capabilities may have been exposed.
• Reviewing relevant contractual and access controls.
• Restricting or suspending suspicious access where appropriate.

The goal is to establish what happened, how long the activity continued and what information or capabilities may have been obtained.

That analysis can then support technical remediation and any legal response.

AI Security Is Becoming an Intellectual Property Issue

The latest US government warning also shows how closely cyber security and intellectual property protection are beginning to overlap.

For companies investing heavily in AI development, proprietary value may increasingly exist in system behavior and model capability rather than only in conventional trade-secret documents.

That means organizations may need to think differently about what constitutes sensitive information.

A model can reveal valuable knowledge through interaction even where its internal architecture remains inaccessible.

Protecting advanced AI therefore requires both technical controls and a clear understanding of how those systems can be probed, replicated or abused.

What US Companies Should Consider Now

Organizations developing or deploying proprietary AI systems should review whether their current controls address model-extraction risk.

Key questions include:

• Are unusual usage patterns monitored?
• Are rate limits appropriate?
• Can linked accounts be identified?
• Are APIs segmented by risk?
• Are sensitive capabilities exposed unnecessarily?
• Can suspicious activity be investigated quickly?
• Are customer and partner access rights clearly defined?
• Are logs retained in a way that supports investigation?

These controls should be proportionate to the value and sensitivity of the system.

A New Cyber Security Challenge for AI Companies

The joint NSA, FBI and CISA advisory reflects a broader shift in the cyber threat landscape.

As AI systems become more commercially valuable, attackers may not need to steal source code directly to obtain useful capabilities.

Instead, they may attempt to reproduce those capabilities through systematic interaction.

That means protecting proprietary AI requires organizations to monitor not only unauthorized access, but also potentially abusive use of authorized interfaces.

For US companies operating advanced AI systems, model security is therefore becoming an important part of wider cyber security and intellectual property protection.