HOW CRIMINALS ARE USING AI TO MAKE PHISHING EMAILS INDISTINGUISHABLE FROM REAL ONES

There was a time when phishing emails were relatively easy to spot. Poor spelling, generic greetings, implausible scenarios, and suspicious sender addresses were the telltale signs that something was off. Staff training focused on teaching people to look for exactly these red flags.

That time is over.

Artificial intelligence has fundamentally changed what phishing looks like, and the vast majority of businesses have not updated their defences to reflect this. The result is that employees who would confidently have spotted a phishing email two years ago are now being deceived by messages that are, to all intents and purposes, indistinguishable from legitimate correspondence.

‍

What Has Changed

Until recently, crafting a convincing phishing email required skill and effort. Criminals had to write plausible content, avoid obvious errors, and hope their generic approach would land with someone. The success rate was relatively low, which is why attacks were sent in such high volumes.

AI tools — including large language models similar to those powering consumer chatbots — have removed these barriers entirely. Criminals can now generate highly personalised, grammatically perfect, contextually accurate phishing emails at scale, for almost no cost.

What does this look like in practice? An attacker can feed an AI tool publicly available information about your business — your website, LinkedIn profiles, recent news mentions, Companies House filings — and generate an email that references your actual suppliers, your real leadership team, current projects your business is known to be working on, and the correct tone and terminology for your industry. The email will be perfectly written, appropriately formatted, and entirely plausible.

‍

Spear Phishing at Scale

Traditionally, there was a distinction between broad phishing attacks — mass emails sent to thousands of recipients — and spear phishing, which involved highly targeted, personalised attacks that required significant research and were therefore relatively rare.

AI has collapsed that distinction. Attackers can now run spear phishing campaigns at the same scale as mass phishing, generating individually tailored emails for hundreds or thousands of targets simultaneously. Every recipient receives something that feels personal and specific to them.

‍

Voice and Video Are Next

Text is only part of the picture. AI is now being used to clone voices with a small amount of audio sample — enough for a criminal to impersonate your CEO, your bank manager, or one of your suppliers in a phone call. Several UK businesses have already lost significant sums to so-called vishing attacks where staff were convinced by a cloned voice to authorise urgent payments.

Deepfake video technology is advancing rapidly too. While not yet widely used in business fraud, security experts expect this to become more common within the next 12 to 18 months.

Why Traditional Training Is No Longer Enough

Most staff security awareness training still focuses on spotting the old warning signs — spelling mistakes, suspicious links, unexpected attachments, generic greetings. These remain worth knowing, but they are increasingly insufficient.

If your training programme has not been updated in the past 12 months, it is already out of date. Employees need to understand that a well-written, personally addressed email from what appears to be a known contact can still be an attack.

‍

What Actually Works Now

The answer is not to abandon staff training — it remains essential — but to combine it with the right technical controls and updated processes.

On the technical side, ensure your email security platform includes AI-powered threat detection that can analyse email behaviour and context, not just scan for known malicious links or attachments. Tools like Microsoft Defender for Office 365 or specialist email security gateways can catch a significant proportion of AI-generated phishing attempts before they reach inboxes.

Implement DMARC, DKIM, and SPF email authentication records for your domain. These make it significantly harder for attackers to send emails that appear to come from your own domain, protecting both your staff and your clients.

On the process side, introduce verification protocols for any request involving money movement or sensitive data, regardless of how legitimate the request appears. A simple callback policy — where any request to change payment details or transfer funds is verified by phone to a known number before acting — can prevent the majority of business email compromise attacks.

Update your phishing simulations to reflect the new reality. Generic simulations that send obviously suspicious emails are no longer adequate. Your team needs to be tested against realistic, well-crafted scenarios that reflect what actual attacks look like today.

Finally, create a culture where staff feel comfortable questioning and verifying unusual requests without fear of embarrassment. The most dangerous environment is one where employees are too worried about appearing unhelpful or inefficient to double-check something that feels slightly off.

‍

The Bigger Picture

AI-powered phishing is not a future threat — it is happening now to businesses like yours. The criminals using these tools are organised, well-resourced, and continually refining their methods. The businesses that will weather this shift are those that treat security as an ongoing discipline rather than a box-ticking exercise.

If you would like us to assess your current email security configuration or update your staff training programme to reflect today's threat landscape, contact our team for a consultation.