AI in cybersecurity: how defenders use it and where it falls short

What AI does in cybersecurity

"AI" in a security product covers several techniques. Machine learning models that score files and emails have been standard for years; the newer layer is large language models (LLMs) that read an alert, query logs and write up what they found. Each answers a different problem:

The common thread is a division of labour, and it is how AI and security teams actually work together: models read and score at machine speed, and people decide what the output means.

How machine learning detects malware on endpoints

Signatures, the hash or byte pattern of a known sample, miss anything new or repacked. Machine learning models learn the features malicious files and behaviours share, so they can judge a file nobody has seen before. Microsoft's technical overview of Defender Antivirus shows how the work is split:

• On the device, lightweight models reach a verdict within milliseconds, with specialized models for the file types attackers abuse, such as Windows executables, PowerShell, Office macros, JavaScript and PDF.
• In the cloud, when the client meets a file it cannot judge, it sends metadata or the file itself. File-type and feature-specific models, including what Microsoft calls adversary-hardened monotonic models, analyze a featurized description, and stacked ensemble classifiers combine their results into a verdict before the file runs.
• For harder cases, the file is held from running while deep neural networks classify its full contents and reply within seconds, or it is detonated in a sandbox where deep learning classifiers judge what it does.

Cloud protection is on by default, and Microsoft recommends keeping it on: it identifies new threats quickly, sometimes before any device is infected.

Once a program runs, behaviour-based models watch what it does. In Defender for Endpoint, behavioural blocking and containment detects threats on the device through machine learning, blocks and remediates them automatically, and uses a feedback loop to stop the same threat on other devices; the on-device blocking and the feedback loop are both on by default. Microsoft lists techniques these capabilities have blocked, among them credential dumping from LSASS, process hollowing, tampering with antivirus and pass-the-hash attacks. Endpoint detection and response (EDR) collects the signals behind these detections and groups related alerts into incidents, so an analyst can see how an attack unfolded. For a single PC rather than a fleet, see how to choose antivirus software.

How AI filters phishing and spam email

In a 2022 overview, Google said Gmail's filters block more than 99.9 percent of spam, phishing and malware from reaching inboxes. They use machine learning powered by user feedback: the models weigh the sending IP address, domains and subdomains, whether bulk senders authenticate, and what users do, so marking a message as spam is a training signal.

Attackers write to get past classifiers, with homoglyphs (lookalike characters), invisible characters and keyword stuffing. In November 2023 Google introduced RETVec, a text vectorizer built to resist those tricks; swapping it into the Gmail spam classifier improved the spam detection rate over the baseline by 38% and reduced the false positive rate by 19.4%.

Microsoft's anti-phishing policies expose the same ideas as settings:

• Every Microsoft 365 mailbox gets spoof intelligence, a first contact safety tip and unauthenticated sender indicators.
• Mailbox intelligence in Defender for Office 365 uses AI to learn each user's patterns with frequent contacts. A vendor you write to often who shares your CEO's name is not flagged; a first-time sender using that name can be. It is on by default, but acting on its detections needs a second setting, "Enable intelligence for impersonation protection", which is off by default.
• Phishing email thresholds, from 1 (Standard) to 4 (Most aggressive), set how strictly the machine learning verdicts are applied. Each step treats lower-confidence verdicts as very high confidence, and Microsoft notes that false positives become more likely.

How anomaly detection and UEBA find compromised accounts

Classifiers judge an object: this file, this email. Anomaly detection judges behaviour against a baseline: this account, on this device, at this hour. User and entity behaviour analytics (UEBA) applies that to identities, devices and applications, which is how it catches a stolen password that no signature would flag.

In Microsoft Sentinel, UEBA uses machine learning to build behavioural profiles for users, hosts, IP addresses and applications, and compares current activity with them to surface compromised accounts, insider attacks and lateral movement. It weighs deviations across locations and devices, from the entity's own history, from its peer group (the top 20 peers, ranked from security group membership, mailing lists and other associations) and from organization-wide patterns.

Sentinel produces two scores, and Microsoft's own example shows why unusual is not the same as malicious:

UEBA is included with Sentinel at no extra cost; its data follows standard Sentinel pricing. It is only as good as the sources you connect, starting with Microsoft Entra ID, Defender for Identity and Office 365.

How SOAR and automated response contain attacks

Security orchestration, automation and response (SOAR) is the automation layer of a security operations center (SOC). Microsoft's Sentinel documentation starts from the problem: alert volumes so large that many alerts are ignored and many incidents never investigated. Sentinel, a SIEM, answers with automation rules that tag, assign or close incidents and create task lists for analysts, and playbooks built on Azure Logic Apps that handle enrichment, response and remediation across other systems, automatically or on demand.

A playbook is only as good as the rule that starts it. The newer step is a model deciding when to act. Microsoft Defender's automatic attack disruption correlates signals from endpoints, identities, email and SaaS apps into one high-confidence incident, identifies the assets the attacker controls, and contains them by isolating devices, disabling or suspending users and revoking sessions. Microsoft says it maintains a confidence level of 99% or higher for containment actions, measured on real production data, validates new detectors in audit mode first, and uses graph models, boosted decision trees, neural networks and small language models. The SOC stays in control of investigation and recovery: every automatic action can be undone, and critical assets can be excluded.

How LLM assistants help triage and investigate alerts

An LLM can read an alert, pull context from logs and threat intelligence, and explain its conclusion in plain language. Microsoft and Google both now offer agents that do this first pass of triage. How these models produce their answers, and why they can be fluent and wrong, is covered in what generative AI is.

Microsoft Security Copilot runs in a standalone portal and inside Defender XDR, Sentinel, Intune and Entra, with plugins for third-party services such as ServiceNow. Documented uses include turning complex alerts into actionable summaries with step-by-step response guidance, writing KQL queries from natural language, analyzing suspicious scripts and drafting reports. Security Copilot grounds each prompt with data from its plugins before the language model sees it, and the analyst reviews the response. Microsoft began including it in Microsoft 365 E5 and E7 in a rollout that started on November 18, 2025: 400 Security Compute Units (SCUs) a month per 1,000 paid user licences, up to 10,000, at no additional cost. Microsoft says usage beyond that will later be throttled, with extra SCUs at USD 6 each, pay as you go.

Security Copilot's Phishing Triage Agent in Microsoft Defender uses LLM-based analysis, file and URL detonation, screenshot analysis and threat intelligence to classify each user-reported email as a real threat or a false positive, and explains its reasoning. It resolves the false alarms itself; confirmed threats stay open for an analyst. Analysts correct it in plain language, and the corrections go into the agent's memory for similar alerts. It needs Defender for Office 365 Plan 2 and Security Copilot capacity, and Microsoft recommends a dedicated agent identity with only the permissions it requires.

Gemini in Google Security Operations builds searches from natural language, generates YARA-L detection rules and playbooks from a prompt, answers threat intelligence questions and summarizes cases with suggested resolutions. Google's Triage and Investigation Agent runs an investigation plan on an alert (searches, Google Threat Intelligence enrichment, command-line analysis, process tree reconstruction) and returns a disposition, true or false positive, with a confidence level, an explanation and suggested next steps. Google says most investigations finish in about 60 seconds, with a 20-minute maximum.

Does it help? The best measurement so far is observational: across more than 150 organizations, Microsoft researchers found that adopting Security Copilot was associated with a 30.13% reduction in mean time to resolve incidents three months later, against a control group. Because organizations chose whether to adopt it, the authors note, the study cannot prove cause.

How AI detects payment fraud and scam pages

Stripe Radar scores each payment in real time with adaptive AI models that use hundreds of risk factors and data from across Stripe's network of businesses. Where the plan shows it, the risk score runs from 0 to 99: by default, 65 or above is elevated risk and 75 or above is high risk. High-risk payments are blocked by default; elevated-risk payments go through, or into a manual review queue on plans that support one. Refunding and reporting fraudulent payments feeds back into the models.

Browsers apply the same idea to scam pages. Since Chrome 137 in May 2025, Chrome runs the on-device Gemini Nano model on pages that show signs of a tech support scam, such as use of the keyboard lock API, to extract signals like the page's intent; Safe Browsing makes the final call with other intelligence. Google's reason for working on the device: the average malicious site exists for less than 10 minutes. The signals are sent only for users who opted into Enhanced Protection.

What the evidence says about AI in security operations

The most quoted figures come from IBM's Cost of a Data Breach Report 2026. Ponemon Institute conducted it, and IBM sponsored and analyzed it, from breaches at 602 organizations between March 2025 and February 2026, according to IBM's July 29, 2026 announcement.

Read these as correlations, not experiments. Organizations that use security AI extensively can differ in other ways, such as staffing and budget, so not all of the USD 1.93 million gap is the AI's doing. Still, IBM's figures and the Security Copilot study point the same way: automated detection and triage go with faster resolution and cheaper breaches. IBM also warns that the low use of agents in vulnerability management leaves known exposures open while AI shortens the time attackers need to exploit them.

Where AI in cybersecurity falls short

AI changes how defenders spend their time. It does not remove the first two problems below, and it creates the third.

False positives and alert fatigue

Every detector trades missed attacks against false alarms. In a 2022 USENIX Security study, SOC practitioners confirmed high false positive rates from their tools, and most of those turned out to be benign triggers: real alarms explained by legitimate activity. Validating them is tedious, and the authors link it to alarm burnout and eventually desensitization. They ask for alerts that are reliable, explainable, analytical, contextual and transferable.

AI moves this trade-off; it does not escape it. A stricter phishing threshold catches more phishing and flags more good mail. UEBA ranks a first-time action high for investigation and low as an anomaly. An agent that closes false positives saves time only while its mistakes are rare, which you learn by sampling what it closed, not from its confidence score.

Attackers study and fool the models

A classifier is software an attacker can study. NIST's taxonomy of adversarial machine learning (AI 100-2 E2025, March 2025) sorts attacks on predictive models into evasion, poisoning and privacy attacks. It notes such attacks have been shown against malware classifiers, PDF malware detection, Android app classification and network intrusion detection, and that needing very low false negative and low false positive rates at once makes these models susceptible.

MITRE ATLAS catalogues these attacks in practice; as of September 2026 it lists 16 tactics, 208 techniques, 40 mitigations and 73 case studies. Two involve security products:

• In a 2019 exercise, researchers studied a vendor's public talks and patents, used verbose logging to see its reputation scoring, found a secondary model that could override the main one, and fused attributes of known good files into malware to bypass CylancePROTECT.
• In a December 2022 incident reported by Norton Research Group, phishing sites blurred, cropped and stretched brand logos to evade the visual-similarity model of a commercial phishing detector. That model was fooled; other components of the system still flagged the sites.

The practical lessons: prefer products that layer independent detectors, keep model scores away from anyone who does not need them, and ask vendors how they harden their models.

Prompt injection and data leakage when AI agents act

The OWASP Top 10 for LLM Applications 2026, published in August 2026, ranks prompt injection first (LLM01), sensitive information disclosure second (LLM02) and excessive agency third (LLM03). It states the root cause plainly: LLMs make no architectural distinction between instructions and data, so there is no clean equivalent to the parameterized queries that stopped SQL injection.

Security is a hard case, because much of what a security agent reads was written by the attacker: the phishing email being triaged, the command line in an alert, the page behind a reported link. A hidden instruction can try to steer the agent toward a benign verdict, and for an agent that resolves false positives itself, a benign verdict closes the alert. Leakage is the other risk: an assistant grounded in your logs and mailboxes can repeat what it read, and OWASP counts tool-call arguments, retrieved passages and logs as disclosure channels, not only the final answer. AI systems are targets in their own right; in IBM's 2026 study, more than 20% of organizations reported a breach targeting AI models or applications, most often through compromised APIs, applications or plug-ins (27%) or cloud misconfigurations affecting AI workloads (27%).

OWASP's mitigations become rules for security agents:

1. Give each agent its own identity with the least privilege it needs, as Microsoft recommends for its phishing agent.
2. Require explicit human confirmation before any privileged, irreversible or externally visible action, and show the reviewer the exact action, not a summary.
3. Apply the Rule of Two, which OWASP adopts from Meta: an agent with untrusted input, access to sensitive data and the ability to change state or communicate externally needs per-action human approval. Many security agents have all three.
4. Treat writes to an agent's memory, analyst feedback included, as privileged changes, and log them.

Why people stay in the loop

AI and security teams work well together because each covers the other's weakness. Models read everything, quickly and consistently; people know the business, notice what does not fit and carry the accountability. The NIST AI Risk Management Framework (AI RMF 1.0, January 2023; NIST says a revision is in progress) describes human-AI configurations that range from fully autonomous to fully manual, and suggests collecting data on how often, and why, people overrule AI output. Its core asks organizations to define, assess and document processes for human oversight (MAP 3.5), and to have mechanisms and assigned responsibility to supersede, disengage or deactivate an AI system whose performance or outcomes are inconsistent with its intended use (MANAGE 2.4).

The vendors' designs follow this. Attack disruption leaves investigation and recovery to the SOC and makes every action reversible. Google's triage agent suggests next steps to the analyst. Microsoft's phishing agent leaves confirmed threats to a person. Keeping a human in the loop has its own failure mode, which OWASP names: approval fatigue degrades reviewer judgment at volume. Ask people to approve few, consequential actions, and let rules handle the routine ones.

How a small or mid-sized organization can start

You do not need a SOC or a new platform. Most organizations already pay for more AI-based protection than they use.

1. Start with the AI in the licences you have. Microsoft 365 Business Premium includes Defender for Business, built for up to 300 users, with next-generation protection, EDR, automated investigation and remediation, and automatic attack disruption. In Google Workspace, administrators can let Gmail automatically apply future recommended settings for phishing and malware protection. If you take card payments through Stripe, Radar is already scoring them.
2. Check it is switched on everywhere. Confirm cloud protection and behavioural blocking are on, that impersonation protection can act and not only observe, and that every laptop and phone is enrolled; our guide to managing security on your team's devices covers enrolment.
3. Name who reads the alerts. Automation contains an attack; a person still investigates it and restores what was contained. An alert nobody reads protects nobody.
4. Add an LLM assistant where you have volume. With E5 or E7 licences, Security Copilot is included, or on its way in Microsoft's rollout: start with incident summaries and the phishing triage agent, and review a sample of the alerts it closes each week before relying on it.
5. Set rules before any agent acts: its own least-privilege identity, approval before it changes data or accounts, and a log of what it saw and did.
6. Measure before and after: false positives, time to triage and time to resolve, so you know whether a tool earns its cost.

None of this replaces patching, phishing-resistant sign-in (our man-in-the-middle guide shows why codes can be relayed), backups and least privilege for people; our cybersecurity checklist for small business puts those in order.

For an outside view of what you expose, our security testing service scans your sites, servers and cloud accounts on a schedule and ranks findings by real risk, with every result reviewed by an engineer and false positives removed. When a workflow needs AI you do not have yet, our AI and automation service builds it so that nothing changes in your systems until a rule or a person approves it; and when the answer is a feature you already pay for, we say so.