AP® Cybersecurity review sheet from Aim for Five (aimforfive.com/cybersecurity/units/1/1-4)
Unit 1 · Topic 1.4
1.4 AI-Based Cybersecurity Attacks
AI tools make old attacks more convincing and new attacks possible, from cloned voices on phone calls to flawless phishing emails. This topic covers how adversaries use AI and the habits that protect you, like family code words, MFA and keeping personal data out of chatbots.
Key terms
- deepfake
- voice cloning
- large language model (LLM)
- AI-written phishing
- poisoned training data
- shared secret
Fake voices and faces
AI tools can study existing recordings and photos of a person and build a digital avatar of them: a cloned voice, or a fake face that moves and talks on video. These fakes are often called deepfakes. With them, an adversary can call your family pretending to be you, or join a video meeting pretending to be a boss who approves a payment.
The harm is usually money or secrets: a victim wires funds or shares private information because the request seems to come from someone they trust. The risk grows as more banks and companies use voice-based authentication, where your voice itself is used to confirm who you are. A good voice clone could pass that check.
Smarter phishing and data leaks
- Polished phishing in any language. Large language models (LLMs), the AI systems behind chatbots, can write fluent messages in any language. Phishing used to be easier to spot because it was often written by non-native speakers and had odd grammar. That clue is disappearing, so a well-written message is not proof it's real.
- Pulling secrets out of chatbots. Adversaries write prompts designed to make an LLM reveal sensitive information. That information can come from what users typed into the tool or from the huge data sets used to train it.
- Poisoning what AI learns. Adversaries can publish websites, or change existing ones, to contain false information so that it ends up in an LLM's training data. The model may then repeat the falsehood as if it were fact.
- Faster reconnaissance. AI tools can scan social media and public websites to gather facts about a target far faster than a person could.
- Help writing malware. AI coding tools can help adversaries write new malware, change ordinary programs to do harmful things, or find vulnerabilities in large amounts of code.
How to protect yourself
- Set up a shared secret: a word or phrase known only to you and a close friend or relative. In a high-stakes moment, like a panicked call asking for money, ask for it. A voice clone won't know it.
- Turn on MFA. If an adversary clones your voice to get past voice authentication, a second factor, such as a code on your phone, can still stop them.
- Keep personal or sensitive data out of AI tools like chatbots and virtual assistants. Some tools use what you type to keep training the model, and adversaries may be able to extract it later.
- Check AI output before you trust it. Confirm important facts with reputable, stable sources that aren't AI-generated, like an official website, a textbook or a known news outlet.
A note on dates
AI tools change fast, and so do the tricks built on them. The attacks and defenses in this topic are the ones the course framework names as of fall 2026. The habits (verify through a second channel, use MFA, don't overshare) stay useful even as the tools change.
Worked examples
Try each one yourself first, then open the solution.
- Example 1
The panicked phone call
Ms. Ortiz gets a call that sounds exactly like her son. He says he's been in a car accident, his phone is broken, and he needs $900 sent to a friend's payment app right away. Explain how AI could make this attack possible and give two actions that would protect her.
Show the solutionHide the solution
- Step 1: Name the AI technique: an adversary can clone a voice from existing recordings, such as videos the son posted online.
- Step 2: Note the social engineering: the urgency ("right away") and fear are meant to stop her from thinking it through.
- Step 3: First protection: ask for the family's shared secret word. A cloned voice won't know it.
- Step 4: Second protection: verify through a separate channel, such as calling her son's own number or another family member, before sending any money.
Answer: An adversary could use AI to clone the son's voice from online recordings. Ms. Ortiz should ask for a pre-agreed shared secret word and verify by calling her son or family directly before sending money.
- Example 2
What not to paste into a chatbot
A student asks an AI chatbot to fix the formatting of a spreadsheet that lists classmates' names, home addresses and phone numbers. Explain the risk.
Show the solutionHide the solution
- Step 1: Identify the data: names, addresses and phone numbers are personal information that could be used to impersonate or contact people.
- Step 2: Recall how some AI tools work: they may feed user input back into the model for continued training.
- Step 3: Connect to the attack: adversaries can craft prompts to extract information that users entered, so the data could leak to strangers.
- Step 4: Give the fix: remove or replace the personal data with made-up placeholders before using the tool, or don't use it for this task.
Answer: Some AI tools use what users type to keep training, and adversaries can write prompts to extract that data, so classmates' personal information could leak. Remove the personal data before using the tool.
Common mistakes
- Assuming a message with perfect grammar is legitimate. AI lets attackers write fluent phishing in any language.
- Trusting a familiar voice or face on a call as proof of identity. Voices and faces can be cloned; a shared secret or a call back on a known number is stronger proof.
- Thinking only what you type is at risk in an LLM. Sensitive information can also come from the model's training data.
- Treating AI answers as automatically correct. Training data can be poisoned with false information, so verify with non-AI sources.
On the exam
- Questions usually describe an AI-assisted attack and ask either how AI helped the adversary or which defense fits best. Match the defense to the attack: shared secret for voice or video impersonation, MFA for voice authentication, no personal data for chatbot leaks, and outside verification for false AI output.
Connected topics
Videos
Check yourself: 1.4 AI-Based Cybersecurity Attacks
5 questions on 1.4 AI-Based Cybersecurity Attacks. Pick an answer to see if you got it, and why.
A payroll clerk at Lakeshore Logistics joins a video call that appears to include the company's chief financial officer (CFO). The CFO's face and voice look and sound exactly right.
The "CFO" says a secret deal must close within the hour and asks the clerk to send $48,000 to a new vendor account. The clerk sends the money. Later the company learns the CFO was never on the call.
The real CFO often posts video interviews and conference talks on social media.
Invented scenario about Lakeshore Logistics, an invented company
Which technique did the adversary most likely use?
Why did the CFO's habit of posting videos online increase the risk of this attack?
Which control would best help employees confirm that a caller is really the CFO before sending money?
An adversary publishes dozens of web pages claiming that a popular free password manager has shut down and that users must download a "replacement" from a site the adversary controls.
A few months later, students notice that several AI chatbots repeat this claim when asked which password manager to use.
Invented scenario
Which AI-based attack does this describe?
What should a student do before acting on the chatbot's advice?
0 of 5 answered