How AI Is Changing Offensive Security (and What Defenders Must Do)
AI is speeding up every stage of offensive security, from reading code to writing exploits. Here is what AI-assisted testing really does well, where it still fails, and what defenders need to change.

Web security work is mostly reading: HTTP traffic, JavaScript bundles, source code, documentation, scanner output, and reports. Large language models are very good at reading. So it is no surprise that AI has quickly become one of the most important tools in offensive security, for professional testers and for criminals alike.
Here is a clear-eyed look at what is changing.
Where AI genuinely helps testers
Understanding code and applications. Paste in a minified JavaScript bundle and a model can list the API endpoints, explain the authentication flow, and point out suspicious logic in seconds. For authorized code reviews, it can trace data from an input to a dangerous sink and explain why that matters.
A useful trick: tell the model to say “not visible” instead of guessing about code it has not been shown. Asking a model to admit uncertainty noticeably reduces confident nonsense.
Generating test cases. Models are good at brainstorming what could go wrong in a feature, and at generating encoding variations and context-specific payloads for a parameter you are testing.
Writing glue code. So much security work is small scripts: parse this tool’s output, replay these requests with a different session, diff these two responses. AI turns a 30-minute chore into a two-minute one.
Triage and summarization. Recon and scanning produce more data than any person can read. Models can group similar findings, flag the interesting hosts, and filter likely false positives from scanner output (with a human checking the result).
Reporting. Turning rough notes into a clear report that explains impact to both executives and engineers is a strength of language models.
Autonomous testing agents
Beyond assistance, AI agents can now plan and run portions of a penetration test themselves: running recon tools, reading results, choosing next steps, and attempting exploitation. Published research and industry experience show agents that can exploit known vulnerabilities, and agentic systems have found real, previously unknown vulnerabilities in mature software.
They still have clear weaknesses. Agents are much weaker at business logic, which requires understanding what an application is for. They struggle with long, complex engagements, lose track of context, and can report convincing findings that turn out to be false.
The risks of letting an agent loose
Autonomous testing creates risks human testers manage instinctively:
- Scope violations. An agent that follows a link off-scope may attack a system you have no permission to touch.
- Destructive actions. An agent might delete data, lock accounts, or flood a service.
- Prompt injection from the target. This is the unusual one. The agent reads the target’s pages and responses, and those can contain instructions. A malicious or simply weird site can steer your testing agent.
The lesson: scope and safety limits must be enforced in deterministic code, not in the agent’s prompt. A hardcoded allowlist of in-scope hosts, checked by the tool before any request is sent, cannot be talked out of its job. A system prompt saying “only test staging.example.com” can.

How attackers use AI
AI providers have publicly reported disrupting accounts linked to criminal and state-backed groups using their models. The main uses are not science fiction. They are scale and speed:
- Phishing that is fluent, personalized, and produced in bulk, in any language.
- Research on targets, technologies, and published vulnerabilities.
- Tooling and malware development, lowering the skill needed to build working tools.
- Exploit development, especially turning freshly published patches into working exploits faster.
What defenders must change
If attackers move faster, defenders have to as well:
- Patch faster. The window between disclosure and exploitation is shrinking. Know your assets and dependencies so you can act within days, not months.
- Adopt phishing-resistant authentication. Better phishing emails matter much less when passkeys and WebAuthn cannot be phished in the first place.
- Use AI defensively. The same strengths that help attackers, such as reading code, triaging alerts, and writing fixes, help defenders.
- Invest in fundamentals. AI makes it cheaper to find the bugs you already have. Fewer bugs is still the best defense.
Responsible use
The ethics of testing do not change because a model is involved:
- AI-assisted testing must stay within authorized scope.
- Protect client data. Do not paste confidential code, credentials, or customer data into tools that are not approved for it.
- Verify before reporting. Every finding must be confirmed by a human. “The AI said so” is not evidence.
For learners: use AI as a tutor
The best way to build skills now is to use AI to explain, not just to do. Ask it why a payload works, what the fix is, and what you should try next, then do the work yourself. The fundamentals are what let you notice when the model is wrong.
Key takeaways
- AI speeds up code comprehension, test generation, scripting, triage, and reporting.
- Autonomous agents can exploit known bugs and have found real zero-days, but still struggle with business logic.
- Enforce agent scope and safety in code, because targets can prompt-inject your testing agent.
- Attackers mainly use AI to scale phishing, research, tooling, and exploit development.
- Defenders should patch faster, adopt passkeys, use AI defensively, and keep humans verifying results.
Go deeper with the book
This article is adapted from Web Security in the Age of AI: Attacks, Defenses, Tools, and Bug Hunting for Modern Web Applications and LLM Systems by Rupak (see Chapter 22, “AI-Assisted Offensive Security”).
The book pairs every attack with its fix across 27 chapters, from core vulnerabilities and professional tools to bug bounty hunting, attacking and defending LLM applications, and lessons from real breaches. It includes hands-on labs, review questions, a full testing checklist, a security headers reference, and an LLM red teaming playbook.
Related reading
- Prompt Injection Explained: How Attackers Hijack LLM Apps
- AI Code Review and Autofix: Can AI Fix Your Security Bugs?







