Skip to main content

Prompt Injection

Prompt injection attempts to make an AI system ignore its intended instructions, reveal protected information, or take an unauthorized action. It is an AI-native risk and must be handled separately from ordinary input validation.

Direct and Indirect Attacks​

Direct prompt injection is supplied by the user. Examples include requests to reveal system instructions, disregard policies, or use a tool outside the task.

Indirect prompt injection is embedded in content the agent reads, such as an email, webpage, document, ticket, tool response, or retrieved knowledge chunk. This is especially important for agents because the malicious content may try to trigger a real external action.

Defense in Depth​

No single filter reliably removes the risk. Combine controls:

  • Treat retrieved content and tool output as untrusted data, not as instructions.
  • Give each agent only the datasets and functions needed for its purpose.
  • Separate read, draft, send, update, delete, publish, and spend operations.
  • Require confirmation for actions with external, financial, legal, permission, or destructive impact.
  • Keep credentials and secrets outside prompts, memory, uploaded files, and tool results.
  • Validate high-impact arguments such as recipients, amounts, record IDs, and target environments.
  • Log the prompt context, selected function, sanitized arguments, approval, and result.
  • Test with malicious instructions in user prompts, documents, emails, webpages, and tool responses.

Siesta AI Configuration​

Enable Prompt Shield from Organization Security, but do not treat it as an authorization system. Authorization still comes from roles, sharing, data access, connection ownership, function policy, and approval requirements.

For a public agent, also remove private sources and unnecessary tools, disable file upload when it is not required, and verify that page-context injection cannot expose authenticated page data.

Test Cases​

Before launch, verify that the agent refuses or safely contains attempts to:

  1. reveal its system prompt or hidden configuration,
  2. follow instructions embedded in a retrieved file,
  3. send data to a new recipient supplied by untrusted content,
  4. bypass an approval step,
  5. access a dataset outside the user's permissions,
  6. use a disabled tool function.

External reference: OWASP LLM01: Prompt Injection.

Last updated on