Generative AI and Cybercrime: Threat Scenarios

Generative AI and cybercrime are connected risk areas. This article was first written in March 2023, when public generative-AI tools were becoming widely available. It records threat scenarios and defensive questions from that period; current capability claims should be checked against the newer reports listed in the sources.

Cybercrime was already being sold as a service before generative AI. Criminal markets offered stolen credentials, malware, access to infected devices and ransomware, often through brokers and closed channels. Generative AI lowers the effort needed for some language, translation, coding and research tasks, so it can make existing campaigns faster or more convincing. It does not remove the need for access, infrastructure, operational security or human decisions.

How can generative AI change cybercrime?

Threat actors can use a model to draft and translate phishing messages, adapt wording for a particular audience, summarise stolen material or help a novice understand unfamiliar code. The same functions are available to defenders for awareness training, triage and secure development. The security outcome depends on access controls, monitoring and how the system is integrated, not on the label “AI” alone.

The combination with crime-as-a-service matters. A criminal group can buy access or data from one supplier, use a language model to prepare a campaign and outsource delivery or cash-out to another. That division of labour makes attribution and prevention harder, while each individual step may look ordinary in isolation.

Generative AI can also produce convincing images, audio or video. Deepfakes may support impersonation, extortion or disinformation, but a convincing file is not proof that an incident occurred. Investigators should preserve the original evidence, verify identity and provenance through independent channels and avoid amplifying unverified material.

Threat scenarios worth preparing for

• Social engineering at scale: messages can be localised and varied so that repeated campaigns are harder to recognise by wording alone.
• Reconnaissance and analysis: a model can organise public information or help an attacker search through material they already obtained. Access to that material remains the decisive precondition.
• Malware assistance: models may explain or transform code. They can also produce incorrect or detectable output, so this is an accelerator rather than an autonomous capability.
• Impersonation media: generated voice, images or video can add credibility to a payment diversion or account-takeover attempt. A second communication channel and approval workflow remain effective countermeasures.
• Attacks on AI systems: prompt injection, data poisoning and leakage can turn an AI feature into a new route to sensitive information. Treat model inputs and outputs as untrusted data.

AGI is a scenario, not a current incident

Artificial general intelligence (AGI) describes a hypothetical system with broad, human-like problem-solving ability. In 2023 it was reasonable to discuss how a much more capable system could change cyber risk, but that discussion was a scenario analysis, not evidence that criminals controlled such a system. Avoid presenting predictions about an intelligence explosion or the defeat of modern encryption as established facts.

A proportionate security programme addresses capabilities that can be observed today: credential theft, phishing, vulnerable services, malicious insiders, fraud and abuse of exposed AI interfaces. It also plans for change by keeping asset inventories, access reviews, backups, logging and incident exercises current.

Defensive priorities for organisations

1. Harden identity. Require MFA for administrators and remote access, use least privilege and remove stale accounts and tokens.
2. Protect the data used by AI. Classify prompts and uploads, restrict retention, test for prompt injection and prevent secrets from entering untrusted services.
3. Verify unusual requests. Use out-of-band confirmation for payment changes, password resets and urgent executive requests. Train staff that polished language is no longer a reliable authenticity signal.
4. Monitor and rehearse. Log model access, authentication and high-risk transactions; alert on abnormal patterns; keep offline or immutable backups and rehearse containment.
5. Share evidence responsibly. Coordinate with providers, national CSIRTs and law enforcement when appropriate. Preserve timestamps, headers, hashes and relevant model or system logs.

Europol’s 2023 assessment of large language models and the NCSC’s later assessments describe both misuse and defensive uses. They support a balanced conclusion: generative AI increases the speed and scale of some activities, while conventional security controls still address many of the underlying attack paths.

What is generative AI?

Generative AI describes models that produce text, images, audio, video or code from an input. Large language models are one category. Their output can be useful, wrong or unsafe, so it needs the same review and access controls as other software output.

How can threat actors use generative AI?

Threat actors may use public or customised models for translation, social-engineering drafts, reconnaissance summaries or coding assistance. Europol documented these risks in its 2023 report. The model does not replace stolen credentials, vulnerable services or a delivery channel, and defenders can apply the same technology to detection and response.

What can organisations do to reduce the risk?

Start with MFA, least privilege, patching, secure backups, phishing-resistant approval workflows and monitoring. Treat prompts, uploaded files and model output as untrusted; prevent sensitive data from reaching services that are not authorised for it. Keep an incident plan and update it as your AI tools and suppliers change.