NLP & LLMs Medium
What is prompt injection, and what is the difference between direct and indirect injection?
The short answer
Prompt injection is an attack where adversarial instructions override the system's intended behavior. Direct injection comes from the user input itself, such as ignore previous instructions, while indirect injection hides malicious instructions in external content the model ingests, such as a web page, document, or tool output, that the model then follows.
How to think about it
Prompt injection is an attack where adversarial instructions override the system’s intended behavior. Direct injection comes from the user input itself, such as ignore previous instructions, while indirect injection hides malicious instructions in external content the model ingests, such as a web page, document, or tool output, that the model then follows.