Red-Teaming the Agent Before the Attacker Does: Pre-Production Findings Across Models, RAG, and Tool Calls
Red-teaming an LLM product is not a checklist audit - it's an adversarial simulation across prompts, retrieval, and tool calls run before the system is trusted with real customers.
What we were solving
Enterprise SaaS company two weeks from launching a customer-facing RAG assistant with tool-calling access to account data.
- The assistant could call internal tools on a user's behalf, but nobody had tested what happens when the user's input is adversarial rather than well-intentioned.
- Standard application security testing doesn't cover prompt injection, retrieval poisoning, or tool-call abuse - it's a different attack surface entirely.
- Launch was two weeks out; the team needed real findings, not a theoretical framework.
What we built
- Ran structured adversarial testing across three layers: direct prompt injection, indirect injection via retrieved documents, and tool-call abuse attempting to escalate scope beyond the current user.
- Prioritised findings by exploitability and blast radius rather than raw finding count, so the team fixed the two issues that mattered before launch instead of twenty low-risk ones.
- Left the client with a repeatable adversarial test harness so the next model or prompt change could be re-tested without re-engaging a red team from scratch.
Key results
- A critical retrieval-poisoning path was found and fixed before any customer saw it
- Findings ranked by exploitability, not just volume - two fixes mattered, not twenty
- Client's own team can now re-run the harness on every model or prompt change
What it was built on
Representative tools and patterns — exact vendors vary per client environment.
Adversarial testing
Coverage
Handover
What we'd tell the next team
- LLM red-teaming is a different discipline from application pentesting - injection and retrieval attacks don't show up in an OWASP checklist.
- Fix the two exploitable findings before launch; a long low-severity list creates false confidence that everything's covered.
- Hand over the test harness, not just the report - the system will change again before the next scheduled engagement.
Questions this engagement anticipated
How is AI red teaming different from a standard penetration test?
It targets prompts, retrieval pipelines, and tool-calling behavior specifically - attack patterns like indirect prompt injection via retrieved documents don't exist in traditional web/API testing and require a different methodology entirely.
Should this happen once before launch or on an ongoing basis?
Both - a pre-launch pass catches what would otherwise reach real users first, but every model swap, prompt change, or new tool integration reopens the attack surface, which is why the test harness was handed over rather than kept as a one-off report.
This is one of several case studies on ai agent security & autonomous operations.
See the rest of the cluster →Compare your situation to this case.
Bring your constraints - environment, timeline, and budget. We scope before we quote.