Unit 3 · Topic 3.5

3.5 Detecting Network Attacks

Detection tools watch network traffic and logs for signs of attack and raise alerts when they find them. This topic covers IDS, IPS and SIEM tools, how AI scores threats, how to choose between signature-based, anomaly-based and hybrid detection, and the specific clues each network attack leaves behind.

Key terms

NIDS vs. NIPS
SIEM
signature-based detection
anomaly-based detection
false positive and alert fatigue
indicator of compromise (IoC)

Detection tools

Automated tools analyze data from switches, routers, servers, firewalls and user computers, usually stored in log files.

A network intrusion detection system (NIDS) analyzes network data and raises an alert when it finds malicious activity.
A network intrusion prevention system (NIPS) does the same analysis and can also act: closing ports, blocking specific IP or MAC addresses, or rejecting certain protocols.
A security information and event management (SIEM) system pulls in data from many sources (firewalls, NIDS or NIPS, device logs, application logs) and looks for patterns that suggest an attack. Analysts investigate each alert, decide whether it's real, and follow standard procedures to resolve or escalate it.

AI and alert thresholds

A medium-sized organization can log millions, even tens of millions, of events a day, far more than any team can read. Threat detection teams build AI models that classify activity as malicious or normal. These models are probabilistic: they report a percentage, like "83% likely malicious."

Each organization sets a threshold, the percentage that triggers an alert. Too high a threshold lets real attacks pass without an alert. Too low a threshold buries the team in alarms about harmless activity.

Choosing a detection method

Signature-based detection checks activity against a library of patterns from attacks seen before. Each pattern is a signature, a known indicator of compromise (IoC), and the library must be updated as new attacks appear. Anomaly-based detection compares activity to a baseline of normal traffic, recorded while the network was known to be clean, and alerts on anything outside a set tolerance. Hybrid detection combines both.

Factor	Signature-based	Anomaly-based	Hybrid
Best for	High-traffic networks	Networks with consistent traffic patterns	Very sensitive or critical networks
Speed	Fastest	Slower	Slower
Cost	Lowest	Higher (needs more powerful hardware)	Highest
False positives	Almost none	Higher	Higher, and the most alerts overall
New (novel) attacks	Can't detect them	Can detect them	Can detect them
Ease of bypassing	Easiest	Harder	Harder

False positives and false negatives

A false positive is an alert for harmless activity. Too many waste investigators' time and cause alert fatigue: responders get so used to false alarms that they assume the next alert is false too. A false negative is a real attack that gets past detection without an alert, which can lead to loss, harm or destruction of data and systems.

Clues each attack leaves

Evil twin: scan regularly for SSIDs that copy or resemble yours; use signal triangulation to locate and shut down the rogue access point.
Jamming: no wireless devices in one area can connect, and a scan shows electromagnetic noise in the Wi-Fi range.
ARP poisoning: unusual ARP messages, especially one MAC address claiming more than one IP address, and wrong entries in the default gateway's ARP table.
MAC flooding: a sudden surge of Ethernet frames from many different MAC addresses, and an overflowing MAC address table on a switch.
DNS poisoning: hard to detect; an abrupt, unexplained drop in visits to your website is a reason to check your DNS records.
Smurf attack: a sudden jump in ICMP requests sent to the network's broadcast address.
General network IoCs, often found in packet captures: traffic to or from addresses already known to be malicious, someone scanning the network without permission, traffic that suddenly surges or crawls, and an application using a port that doesn't match it (like web traffic on the email port).

Worked examples

Try each one yourself first, then open the solution.

Example 1
Spotting ARP poisoning in a log
The default gateway at Summit Dental is 198.51.100.1, and its real MAC address is 00:00:5e:00:53:01. A monitoring log shows: 09:14:02 ARP reply: 198.51.100.1 is-at 00:00:5e:00:53:01. 09:20:47 ARP reply: 198.51.100.37 is-at 00:00:5e:00:53:9c. 09:20:51 ARP reply: 198.51.100.1 is-at 00:00:5e:00:53:9c. Identify the attack and the evidence.
Show the solutionHide the solution
1. Step 1: Check each IP-to-MAC pairing. At 09:14:02 the gateway's IP maps to its real MAC, which is normal.
2. Step 2: At 09:20:47, the device at 198.51.100.37 announces MAC 00:00:5e:00:53:9c, also normal on its own.
3. Step 3: At 09:20:51, the gateway's IP is suddenly claimed by 00:00:5e:00:53:9c, the same MAC as the device at .37. One MAC address now claims two IP addresses, including the gateway's.
4. Step 4: Conclude: the device at .37 is sending fake ARP replies so traffic meant for the gateway flows to it. That's ARP poisoning, an on-path attack. Next, check the gateway's ARP table and isolate the device.
Answer: ARP poisoning. At 09:20:51 the MAC 00:00:5e:00:53:9c (which belongs to 198.51.100.37) claims the gateway's IP 198.51.100.1, so one MAC is claiming two IPs.
Example 2
Choosing a method
Recommend signature-based, anomaly-based or hybrid detection for each: (a) a busy online store with heavy, rapidly changing traffic and a tight budget; (b) a water utility's control network that carries the same small set of messages all day and expects attackers to try new techniques; (c) a hospital that can afford the most protection for its patient systems.
Show the solutionHide the solution
1. Step 1: (a) High volume and low budget favor signature-based detection: it's fast, cheap and has almost no false positives. Changing traffic patterns would also make an anomaly baseline unreliable.
2. Step 2: (b) Very consistent traffic makes anomaly-based detection effective, and only anomaly-based (or hybrid) can catch novel attacks. If hybrid costs too much, anomaly-based is the choice.
3. Step 3: (c) Highly sensitive and critical systems with budget available suit hybrid detection, accepting the higher cost and extra alerts.
Answer: (a) Signature-based. (b) Anomaly-based. (c) Hybrid.
Example 3
Setting the alert threshold
An AI model scored yesterday's 10,000 events. Two were real attacks, scored 96% and 71%. At a 90% threshold the team gets 6 alerts; at a 60% threshold it gets 140. Explain the trade-off.
Show the solutionHide the solution
1. Step 1: At 90%: only events scoring 90% or more raise alerts. The 96% attack is caught, but the 71% attack is missed, a false negative. The other 5 alerts are false positives.
2. Step 2: At 60%: both attacks (96% and 71%) are caught, but 138 of the 140 alerts are false positives.
3. Step 3: Weigh it: the lower threshold catches more attacks but risks alert fatigue; the higher one is manageable but lets an attack through.
Answer: A 90% threshold gives 6 alerts but misses the 71% attack (a false negative). A 60% threshold catches both attacks but produces 138 false positives, which risks alert fatigue.

Common mistakes

Saying a NIDS blocks attacks. A NIDS only alerts; a NIPS can also block.
Claiming signature-based detection catches new attacks. It only knows signatures already in its database.
Thinking anomaly-based detection works well on any network. It needs consistent traffic and a clean baseline.
Mixing up false positives and false negatives. A false positive is an alarm with no attack; a false negative is an attack with no alarm.

On the exam

Expect log excerpts that ask which attack is happening. Point to the exact lines and values (the duplicate MAC, the surge of ICMP to the broadcast address, the look-alike SSID) as evidence.
For method choices, use the course's criteria by name: traffic volume, consistency of traffic, sensitivity of the network, likelihood of novel attacks, plus speed, cost and false-positive and false-negative rates.

Connected topics

Videos

Intrusion Prevention - CompTIA Security+ SY0-701 - 3.2
Professor MesserWatch on YouTube (opens in a new tab)
IDS and IPS
Sunny ClassroomWatch on YouTube (opens in a new tab)
What Is SIEM?
IBM TechnologyWatch on YouTube (opens in a new tab)
IDS vs IPS: Which to Use and When
CBT NuggetsWatch on YouTube (opens in a new tab)
Security Monitoring - CompTIA Security+ SY0-701 - 4.4
Professor MesserWatch on YouTube (opens in a new tab)

Check yourself: 3.5 Detecting Network Attacks

5 questions on 3.5 Detecting Network Attacks. Pick an answer to see if you got it, and why.

Question 1 of 5

A college installs a network tool that analyzes traffic for malicious activity. When it detects an attack, it can block the attacking IP address and close the targeted port on its own. What kind of tool is this?

Question 2 of 5

A company wants one system that gathers data from its firewalls, intrusion detection systems, device logs and application logs, looks for patterns across all of them, and alerts analysts to possible attacks. Which system fits best?

Network 1: A video-streaming company's network carries an extremely high volume of traffic. The company mainly wants to catch well-known attacks quickly.

Network 2: A small factory's control network carries the same machines sending the same kinds of data, in about the same amounts, every day. The company worries about brand-new attacks but can't afford the most expensive option.

Network 3: A hospital's network carries patient records and controls critical medical systems. The hospital has a large security budget.

Invented networks

Question 3 of 5

Which detection method best fits Network 1?

Question 4 of 5

Which detection method best fits Network 2?

Question 5 of 5

Why would hybrid detection likely suit Network 3?

0 of 5 answered