Vaikora › Blog › Developer Guides

AI Supply Chain Security: Models, Weights, and Dependencies

Developer Guides · June 30, 2026 · 10 min read

AI supply chain security is the practice of assessing and controlling risk introduced by third-party models, weights, datasets, and open-source dependencies before deployment into production systems. It encompasses model provenance verification, unsafe serialization detection, dependency auditing, and runtime policy enforcement to ensure that untrusted or compromised models cannot cause unauthorized actions, data exfiltration, or system compromise. As organizations increasingly rely on pre-trained models and open-source components, supply chain threats have become a critical vector for AI-powered attacks.

The Scope of AI Supply Chain Risk

The software supply chain has matured over the past decade. We now have Software Bill of Materials (SBOM) standards, dependency scanning tools, and signed releases as baseline hygiene. The AI supply chain is years behind.

Most teams downloading models from Hugging Face, PyTorch Hub, or ModelZoo have no way to verify what code runs inside those weights. A model checkpoint is typically serialized Python object data (pickle format), which deserializes into arbitrary code execution during loading. A compromised weight file, a man-in-the-middle attack on the download, or a typosquatted model name can inject backdoors, data exfiltration hooks, or prompt injection payloads directly into production inference pipelines.

Supply chain risk in AI takes three main forms: model provenance (where did this model come from, and can we trust its authors), unsafe deserialization (does loading the model execute untrusted code), and dependency risk (what libraries does the model pull in at inference time, and are they vulnerable).

Model Serialization: The Pickle Problem

The dominant model serialization format in the Python ML ecosystem is pickle. It is fast, widely supported, and inherently unsafe. Pickle can encode arbitrary Python function calls inside a model checkpoint. When you load a pickled model with torch.load() or similar, the unpickling process reconstructs those objects by calling those functions, executing code embedded by whoever created the model file.

This risk is not theoretical. Pickle deserialization can be exploited to execute arbitrary code when a model is loaded, a capability that researchers have documented in multiple publications and proofs-of-concept. Attacks can extract training data, exfiltrate secrets from environment variables, and establish reverse shells on the host machine simply by loading a compromised weight file, requiring no code changes in the downstream application. The attack occurs during deserialization itself.

The standard mitigations are:

1. Use safer serialization formats. PyTorch and TensorFlow both support non-pickle alternatives like SafeTensors (for weights) and ONNX (for inference graphs). SafeTensors is explicitly designed to avoid deserialization vulnerabilities.
2. Verify model signatures. Some hosting platforms (Hugging Face, via the Hub's signed models initiative) provide cryptographic signatures verifying that a model checkpoint has not been modified since upload.
3. Scan model files before loading. Use tools like scan-safetensors or dedicated model security scanners to detect pickle bytecode and known malicious patterns inside weight files.
4. Pin and audit dependencies. Ensure that models declare their dependencies explicitly and that the downstream application audits those dependencies for known vulnerabilities.

Supply Chain Threats at Each Layer

Model Acquisition

When downloading a pre-trained model, you are trusting several parties: the model author, the hosting platform, and the network between you and the server. Mitigations include:

• Verify the source. Use official model repositories (Hugging Face's transformers library directly, not via third-party forks) and check the model card for author information and usage examples.
• Check for model signing. Hugging Face supports optional GPG signing of model repositories. Verify signatures before deployment.
• Use supply chain verification tools. Tools like huggingface-hub with integrity checking can verify checksums against the remote model card.

Dependency Resolution

A model's inference code depends on libraries: transformers, PyTorch, ONNX Runtime, CUDA kernels, tokenizers. Each dependency is a potential supply chain weak point.

Create an AI SBOM (Software Bill of Materials) for every model you deploy. This should list not only the model name and version, but also its direct and transitive dependencies. Tools like pip-audit and safety can scan for known vulnerabilities in the dependency tree.

Runtime Execution

Even if a model is verified as safe during acquisition and deployment, the runtime environment presents risk. A model that calls external tools (APIs, databases, file systems) can exfiltrate data or perform unauthorized actions if its outputs are not validated. This is where runtime policy enforcement becomes essential.

Building an AI Supply Chain Security Process

Step 1: Establish Model Provenance

Before downloading a model, verify:

• Who uploaded it and to which platform.
• Whether the model card includes a clear statement of training data sources and license.
• Whether the upstream repository is still actively maintained.
• Whether the model has been scanned for vulnerabilities by the hosting platform or third parties.

Hugging Face has introduced a model safety card feature that flags potentially harmful models, but it is not foolproof. Read the model card yourself.

Step 2: Scan for Unsafe Serialization

Create a CI pipeline stage that runs before any model is added to your model registry. Example using Python:

import torch
import zipfile
import pickle
import sys

def scan_model_for_pickle(model_path: str) -> bool:
    """
    Scan a PyTorch model file for embedded pickle code.
    Returns True if unsafe patterns are detected.
    """
    try:
        # Check if the file is a ZIP (PyTorch .pt files are ZIPs)
        with zipfile.ZipFile(model_path, 'r') as z:
            # Look for pickle files inside
            for name in z.namelist():
                if name.endswith('.pkl') or 'pickle' in name:
                    print(f"WARNING: Found potential pickle file: {name}")
                    return True

        # Attempt to load with restricted unpickler to catch code execution
        with open(model_path, 'rb') as f:
            try:
                # This will raise an exception if unpickle tries to call a function
                data = pickle.load(f)
            except Exception as e:
                if 'global' in str(e) or 'reduce' in str(e):
                    print(f"ALERT: Potential code execution detected: {e}")
                    return True

        return False
    except Exception as e:
        print(f"Error scanning model: {e}")
        return False

if __name__ == "__main__":
    model_file = sys.argv[1]
    if scan_model_for_pickle(model_file):
        print(f"FAIL: Model {model_file} contains unsafe serialization patterns.")
        sys.exit(1)
    else:
        print(f"PASS: Model {model_file} passed basic serialization checks.")
        sys.exit(0)

Add this to your CI pipeline before model registration:

- name: Scan model for serialization vulnerabilities
  run: python scripts/scan_model.py models/${{ env.MODEL_NAME }}

Step 3: Audit Dependencies

Create an AI SBOM for each model. Extract dependencies from the model card and the requirements.txt or environment.yml that accompanies it:

# Install pip-audit
pip install pip-audit

# Scan a model's dependency requirements
pip-audit --file model_requirements.txt --desc

Document any known vulnerabilities, the versions you are pinning, and the business justification if you cannot upgrade immediately.

Step 4: Implement Runtime Policy Enforcement

A compromised or malicious model can still cause harm during inference. If a model is designed to call external APIs, write to databases, or access file systems, those actions should be gated by policy.

Define policies that specify which actions the model is permitted to perform:

{
  "policy": "model-inference-sandbox",
  "version": "1.0",
  "rules": [
    {
      "action": "execute_tool",
      "target": "database_query",
      "condition": "tool_name == 'search_products'",
      "effect": "ALLOW",
      "metadata": {
        "max_rows": 100,
        "allowed_fields": ["id", "name", "price"]
      }
    },
    {
      "action": "execute_tool",
      "target": "file_access",
      "effect": "BLOCK",
      "metadata": {
        "reason": "Models cannot access filesystem during inference"
      }
    },
    {
      "action": "http_request",
      "target": "external_api",
      "condition": "url.startswith('https://internal-api')",
      "effect": "ALLOW",
      "metadata": {
        "max_requests_per_inference": 5,
        "timeout_seconds": 10
      }
    }
  ]
}

At inference time, every model output that attempts an action is evaluated against this policy. The policy layer can ALLOW, LOG, CONSTRAIN (apply guardrails), or BLOCK before the action executes.

Step 5: Continuous Monitoring and Incident Response

Monitor model inference logs for anomalous patterns: unusual API calls, attempts to access restricted resources, or outputs that exceed expected bounds. Set up alerts if a model's behavior deviates from its training profile.

Maintain an incident playbook for supply chain events:

• If a model dependency has a critical CVE, have a process to rebuild and redeploy within hours.
• If a hosting platform is compromised (e.g., a malicious model uploaded to Hugging Face), have a rollback procedure.
• If a model's behavior becomes adversarial or unsafe (e.g., it starts generating harmful outputs), have a decision tree for quarantine and removal.

NIST and Industry Guidance

The NIST AI Risk Management Framework (AI RMF) emphasizes supply chain risk as a core governance pillar. NIST recommends:

• Mapping all AI components and their sources.
• Evaluating vendor security posture and SLAs.
• Implementing change control and integrity verification for models and datasets.
• Auditing third-party models for bias, performance, and security properties before production use.

CISA's secure AI development and deployment guidance similarly advises:

• Use verified, reproducible model sources.
• Document model lineage and maintain audit logs.
• Implement cryptographic verification of model artifacts.
• Apply principle of least privilege to inference environments.

The EU AI Act requires that high-risk AI systems (systems that could affect fundamental rights or safety) maintain documentation of the training data, validation methods, and security measures. This naturally extends to supply chain verification for any high-risk model you incorporate.

Runtime Controls: The Missing Link

Supply chain scanning and dependency auditing catch many threats, but they cannot prevent a legitimate model from being misused or a model trained with malicious intent from causing harm during inference. A model might pass all static security checks and still be designed to maximize false negatives in fraud detection, or to subtly shift outputs toward sensitive PII extraction.

Runtime policy enforcement adds a deterministic layer between model inference and action. Every output from the model is evaluated against a security policy that you control. If a model attempts to perform an unauthorized action (calling a tool it should not have access to, returning data from restricted fields, making external API calls beyond its scope), the policy layer intercepts the action before it causes harm.

This approach treats the model as a potential adversary and enforces security from the bottom up, rather than assuming the model is trustworthy once it is acquired.

How Vaikora Helps

Vaikora's runtime control platform integrates model policy enforcement into your inference pipeline. The Vaikora gateway sits between your application and the LLM, intercepting every model output and evaluating it against deterministic policies you define. If a model (whether compromised during supply chain transit or trained with backdoor intent) attempts to exfiltrate data or perform unauthorized actions, Vaikora blocks it and logs the attempt into a tamper-proof audit chain. For organizations deploying open-source models or third-party APIs, this adds the security layer that scanning and auditing alone cannot provide. The open-core Vaikora gateway and MCP server are available to self-host, enabling enforcement without vendor lock-in.

Frequently Asked Questions

What is AI supply chain security?

AI supply chain security is the practice of assessing and controlling risk from third-party models, weights, datasets, and dependencies before they run in production. It involves verifying model provenance, scanning for unsafe serialization, auditing dependencies, and enforcing runtime policies to ensure untrusted or compromised models cannot cause unauthorized actions or data loss.

Are models on Hugging Face safe to use?

Hugging Face is the most widely used model repository, but it is not risk-free. Models are community-contributed and not vetted by default. Use the official Hugging Face transformers library directly, check model cards and author reputation, verify signatures where available, and scan models locally before deployment. Hugging Face's safety features (model safety cards, community flags) help, but they require your own due diligence.

How do you scan AI models for vulnerabilities?

Scan models for unsafe serialization (pickle bytecode), known malicious patterns, and vulnerable dependencies. Use tools like pip-audit for dependency checking and custom scanners that inspect weight files for embedded code. Add model scanning to your CI/CD pipeline as a gate before any model is registered or deployed. Additionally, maintain an AI SBOM documenting every dependency and its known vulnerabilities.

What risks exist in third-party AI model dependencies?

Third-party models depend on libraries like PyTorch, transformers, ONNX Runtime, and tokenizers. Each dependency can have unpatched vulnerabilities that your model inherits. A compromised dependency can inject code into inference pipelines. Mitigate by pinning versions, regularly scanning dependencies for CVEs, updating on a defined schedule, and using tools like pip-audit and safety to detect known vulnerabilities early.

What is an AI SBOM and why is it important?

An AI SBOM (Software Bill of Materials) is a comprehensive inventory of all components in an AI system, including the model, weights, datasets, training code, and direct and transitive dependencies. It is important because it enables you to track which systems are affected by a vulnerability, to audit supply chain provenance, and to comply with regulatory requirements (EU AI Act, HIPAA, PCI DSS) that increasingly mandate transparency in AI system composition.

Can runtime policy enforce security on a backdoored model?

Yes. Runtime policy enforcement treats the model as a potential adversary. Even if a model has been compromised or trained with malicious intent, a policy layer between the model and the rest of your system can intercept its outputs and block unauthorized actions. This does not eliminate the risk, but it constrains what damage a compromised model can cause, adding defense in depth.

More from the Vaikora blog