Lenny Rachitsky: Final bit of context. Tell us about essentially the problem that you found. Sander Schulhoff: For the past couple years, I've been continuing to run AI red teaming competitions and we've been studying all of the defenses that come out. And AI guardrails are one of the more common defenses. And it's basically, for the most part, it's a large language model that is trained or prompted to look at inputs and outputs to an AI system and determine whether they are valid or malicious or whatever they are. And so they are kind of proposed as a defense measure against prompt injection and jailbreaking. And what I have found through running these events is that they are terribly, terribly insecure and frankly, they don't work. They just don't work.
And it's basically, for the most part, it's a large language model that is trained or prompted to look at inputs and outputs to an AI system and determine whether they are valid or malicious or whatever they are.