There is a great deal of noise about AI in offensive security, and a fairly small amount of substance underneath it. The substance is worth understanding, because it has genuinely changed how engagements are run, and because the parts it has not changed are the parts that determine whether a report is worth reading.
What Actually Changed
The honest summary is that AI improved the phases of an engagement where volume matters and left the phases where judgment matters largely alone.
Attack surface discovery improved most. Mapping an organization's internet-facing footprint means correlating domains, certificates, cloud resources, code repositories, breach corpora and public records, and the limiting factor was always analyst time. Automating that correlation means engagements now routinely begin with a more complete picture of the target than the target has of itself, which is a real change in what a fixed budget buys.
Correlation and triage improved substantially. The traditional problem with automated tooling was volume: thousands of findings, a large proportion of them false, arriving without context. Automated triage that cross-references findings against the specific environment, clusters duplicates and ranks by reachability turns that into something an operator can work through in hours rather than days.
Reporting improved in the mechanical sense. Drafting the descriptive portions of a finding is repetitive work, and automating the first draft frees the operator to spend their time on the impact assessment and the attack narrative, which are the parts clients actually use.
What Did Not Change
The determination of whether a finding matters remains human, and this is not a temporary limitation waiting on a better model.
Judging severity requires knowing what a system is for. That a medium-severity issue sits on a domain controller and a critical sits on an isolated test box is not information present in the finding. That a particular record should be invisible to a particular user is a business rule, and knowing it requires understanding the business.
Chaining is similar. The most serious findings in most engagements are combinations: a low-severity information disclosure that reveals an internal hostname, plus a medium-severity misconfiguration on that host, plus a credential found in a share. Recognizing that these three combine into a critical outcome is the work, and it depends on holding the whole environment in mind at once.
And the decision to report something is a judgment about consequence. An operator who reports a hundred findings when eight matter has not served the client, whatever the tooling produced.
Where the Line Sits in Practice
The structure that works, and the one we use, is that automation performs reconnaissance, correlation and repetitive validation, and a qualified operator reviews and confirms every finding before it appears in a report.
That division is worth stating explicitly when evaluating a provider, because the phrase "AI-powered" currently covers everything from a genuinely restructured methodology to a scanner with a language model writing the summaries. The distinguishing question is simple: what does a human do, and what do they sign off on?
A related question is what happens to your data. Vulnerability detail for a live environment is among the most sensitive material an organization holds, and it should only enter a third-party AI service under enterprise terms that prohibit training on inputs and provide zero data retention. Providers who have thought about this will answer precisely. Those who have not will change the subject.
The Attacker Side
Attackers adopted the same capabilities, in the same phases, for the same reasons.
The clearest effect is on social engineering. Generated phishing messages no longer carry the language errors that were, for two decades, the most reliable signal available to an ordinary user. Advice built around spotting bad grammar is now obsolete, which matters for how awareness training is written and for why phishing-resistant authentication has become more important than teaching people to spot fakes.
Reconnaissance became cheaper for attackers exactly as it did for testers, which means the effort required to research a target properly no longer restricts targeting to large organizations.
Claims of fully autonomous AI attacks remain, for now, mostly marketing. The realistic assessment is that AI made competent attacks cheaper to produce at volume rather than producing a new category of attack.
What This Means for Buyers
Two practical implications.
First, expect more coverage for the same money. If a provider's methodology has not changed in three years, the discovery phase of your engagement is doing less than it should be.
Second, ask harder questions about validation. The risk in an AI-augmented market is not that the tooling is bad but that automation makes it cheap to produce a large report full of unvalidated findings, which looks thorough and is worse than useless, because it trains your team to ignore findings.
The report you want is short, validated, chained and specific about impact. That has always been true. What changed is how much ground the operators can cover before they get there.
To discuss how we run AI-augmented engagements, get in touch.
Frequently asked questions
Can AI replace human penetration testers?
No, and the reason is specific rather than sentimental. AI is strong at breadth, pattern recognition and repetitive validation, and weak at understanding what a system is for. Judging whether a finding matters requires knowing that this record should be invisible to that user, or that this medium-severity issue sits on a system the business cannot lose. That is context, and models do not have it.
What does AI genuinely improve in an engagement?
Coverage and speed in the phases where volume matters: attack surface discovery, correlating findings across sources, triaging scanner output, and drafting the repetitive portions of a report. These are the parts of an engagement that consume the most operator time for the least judgment, so automating them redirects human effort toward the work that needs it.
Are attackers using AI too?
Yes, mostly for the same reasons and in the same phases. The clearest effect is on phishing quality, where generated messages no longer carry the language errors that used to be a reliable signal, and on reconnaissance speed. Claims of fully autonomous AI attacks remain largely marketing; the practical shift is that competent attacks became cheaper to produce at scale.
Is it safe to put our findings into an AI service?
Only under the right contractual terms. Vulnerability detail for a live environment is among the most sensitive material an organization holds. Ask any provider whether they use enterprise agreements with zero data retention and no training on inputs, and whether client data is isolated. If they cannot answer precisely, that is your answer.
How do I tell AI-assisted testing from a scanner with marketing?
Ask what a human did. A credible provider will describe which phases are automated, what the operator validates, and who signs off on each finding before it reaches you. If the answer is vague, or if the sample report contains hundreds of unvalidated findings with no chained attack path, you are buying scanner output regardless of what the tooling is called.