One of the most challenging and resource-intensive tasks in large language model (LLM) red teaming is identifying jailbreaks — prompts that bypass a model’s safety mechanisms to produce harmful or prohibited responses.
Finding these prompts is essential for testing the resilience of LLM-based systems, yet it often requires deep domain expertise and extensive manual effort.
Given these challenges, it’s natural to ask: can we use AI to help test AI? Specifically, can we automate the search for jailbreaks by having one AI system attack another in a controlled, repeatable way?
In this whitepaper, we evaluate two existing frameworks that approach this problem from different angles. The first, tree-of-attack with pruning (TAP), was designed specifically to automate prompt injection and jailbreak discovery.
The second, Declarative Self-improving Python (DSPy), is a general-purpose prompt optimization framework that can be repurposed to craft adversarial inputs. We tested both frameworks in practical experiments to measure their effectiveness and identify their limitations. Following this experimentation, we assessed their value in real-world cybersecurity strategies.