AI Security Threat Modeling for Engineers

AI Threat Models Need AI-Specific Failure Modes

AI systems introduce attack surfaces that normal web application threat models may miss. Engineers must consider model behavior, prompt injection, retrieval poisoning, tool misuse, data leakage, and unsafe automation paths.

Threat Model Areas

• User input and prompt injection attempts.
• Retrieved documents, untrusted content, and poisoned knowledge sources.
• Tool access, permission boundaries, and action side effects.
• Cross-tenant data exposure through retrieval or memory.
• Secrets, credentials, logs, transcripts, and stored prompts.
• Model outputs used by downstream systems without validation.

Model the Data Flow

Follow sensitive data from input to context assembly to model call to tool execution to logs and storage. Any place data moves is a place controls may be needed.