AI and Cybersecurity – understanding the potential security risks

What is GenAI?

Generative AI, or GenAI, is a type of artificial intelligence that can create new content in response to a user prompt. This content may include text, images, audio, video, code or summaries of existing information.

Examples of GenAI tools include ChatGPT, DALL-E, Google Gemini and Microsoft Copilot. These tools can be useful, but they should be used with care, especially when handling business, employee or pension scheme information.

Employers should make sure employees understand what AI tools can and cannot be used for, what information can be entered into them, and what approvals are needed before using AI-generated content in business processes.

Risks and threats of using AI

AI can bring clear benefits. It can help automate routine tasks, improve employee experience, support cyber threat detection and increase productivity. But like any technology, it also carries risks if used without appropriate controls.

Criminals can use AI to make scams more realistic. A phishing email may be better written, more personalised and harder to spot. A fake voice message may sound like a senior colleague. A fraudulent request may use publicly available information to appear credible.

Employers should be particularly alert to AI-enabled threats involving workforce data, payroll instructions, pension communications, supplier requests, employee records and provider relationships.

The main risks to consider are identity theft, data leakage and misinformation.

Identity theft

AI can be used to create realistic fake content, including deepfake audio, images and videos. Criminals may use this to impersonate senior leaders, employees, suppliers, providers or trusted contacts.

For example, a fraudster could use AI-generated voice or video to support a request to change payment details, release confidential information, approve an urgent transfer or bypass normal controls. They may also use AI-generated emails or messages to make impersonation attempts more convincing.

Employers should encourage employees to verify unusual requests through trusted channels, especially where the request involves sensitive data, payment details, payroll changes, pension information or urgent action.

A message should not be trusted just because it appears to come from a familiar name, email address, voice or image.

Data leaks

AI tools often rely on information entered by users. If sensitive information is entered into a public or unapproved AI tool, that information may be stored, used to improve the tool or exposed in ways the employer did not intend.

Employers should be clear that confidential business information, employee data, payroll information, pension scheme data and member records must not be entered into public AI tools unless this has been specifically approved.

This includes:

• employee personal details
• salary or bonus information
• bank details
• National Insurance numbers
• pension contribution data
• scheme member records
• internal reports or governance papers
• provider or supplier information
• commercially sensitive documents

AI can be helpful, but it must be used in line with data protection, information security and internal policy requirements.

Misinformation

AI-generated content can sound confident and professional even when it is inaccurate, incomplete or misleading. This is sometimes described as an AI “hallucination”.

For employers, the risk is that inaccurate AI-generated information could be used in employee communications, pension materials, internal guidance, training documents or decision-making processes without proper review.

Misinformation can cause confusion, create operational risk and damage trust. It is especially important to check content that relates to pensions, benefits, payroll, tax, legal obligations, financial wellbeing or employee rights.

AI-generated outputs should always be reviewed by a person with the right knowledge before they are used or shared. Human oversight remains essential.

Key takeaways

Here are some key tips to help employers use AI securely and responsibly:

1. Verify unusual requests. Criminals can use AI to create convincing impersonation scams, including fake emails, voice messages and videos. Review, verify and authenticate unusual or sensitive requests before acting.
2. Protect sensitive data. Do not enter employee data, payroll information, pension scheme data, member records or confidential business information into public AI tools unless this has been approved.
3. Keep a human in the loop. AI can produce plausible but inaccurate information. Check outputs carefully before using them in employee communications, governance papers, pension materials or business decisions.
4. Set clear internal rules. Employers should make sure employees understand which AI tools are approved, what they can be used for and what information must not be entered into them.
5. Train employees on AI-enabled scams. Awareness training should include phishing, deepfakes, impersonation, fake payment requests, suspicious links, malicious attachments and social engineering.
6. Pause before acting. AI can make scams look more convincing, but the warning signs often remain the same: urgency, secrecy, pressure, unusual requests or attempts to bypass normal process.

Final thought

AI can help employers work more efficiently, but it also gives criminals new ways to target people, data and systems.

For employers, the priority is to use AI safely: protect sensitive information, verify unusual requests, challenge suspicious content, review AI-generated outputs and make sure employees know how to escalate concerns.

Used responsibly, AI can be a valuable tool. Used carelessly, it can create new routes for fraud, data loss and misinformation.