Overview
Flow data summarizes network connections at scale, enabling broad traffic analysis. This lesson covers flow data in network detection.
Learning objectives
- Define network flow data.
- Explain flow data vs full packet capture.
- Connect flow data to detection and investigation.
Core concept
Flow data is a summary of network connections — recording, per connection, metadata like source/destination IPs and ports, protocol, byte/packet counts, and timing — without capturing the full packet contents. It contrasts with full packet capture (PCAP), which records everything (including payloads) but is far more storage-intensive. Flow data's advantage is scale and efficiency: it can be retained broadly across the whole network for long periods, giving a comprehensive who-talked-to-whom picture ideal for spotting patterns — volume anomalies, beaconing, scanning, unexpected connections — and for investigating an incident's network scope. Flow data is a workhorse of network monitoring: lightweight enough for broad coverage, detailed enough to reveal connection patterns, complementing full-packet capture used selectively.
How it works
Network devices export flow records summarizing each connection: the 5-tuple (source/destination IP, source/destination port, protocol) plus byte and packet counts, duration, and timing. Because a flow record is small (metadata, not payload), flow data can be collected and retained at scale across the entire network — providing a broad, long-lived record of all connections. Analysts use it to detect and investigate patterns: identifying volume anomalies (large transfers → exfiltration), beaconing (periodic connections → C2), scanning (one source to many destinations → recon/lateral movement), and unexpected communications (connections that shouldn't happen). For an incident, flow data reveals the network scope — what an involved host connected to, when, and how much data moved — even long after the event (given retention). Its limitation is the lack of payload: flow shows that a connection happened and its size/timing, but not its contents — for which selective full packet capture (PCAP) is used (e.g. to examine a specific suspicious connection's payload). The two complement each other: flow for broad, scalable pattern detection and scoping; PCAP for deep, targeted content analysis. Flow data pairs naturally with IDS alerts and SIEM correlation.
Practical example
Investigating a suspected compromise, the analyst turns to flow data: they query all connections for the involved host over the past weeks (feasible because flow is retained at scale). The flow records reveal beaconing to an external address (periodic, consistent-size connections — likely C2) and, later, a large outbound transfer to an unfamiliar destination (possible exfiltration) — with exact byte counts and timing. This mapped the incident's network scope without needing full packets. To examine what was in the suspicious transfer, they pull selective PCAP for that specific connection. Flow data gave the broad picture and scope; PCAP gave targeted content — the two complementing each other.
Cybersecurity relevance
Flow data is a workhorse of network detection and investigation: by summarizing every connection's metadata (5-tuple, byte/packet counts, timing) without payloads, it's lightweight enough to retain at scale across the whole network for long periods — ideal for spotting patterns (volume anomalies/exfiltration, beaconing/C2, scanning/lateral movement) and mapping an incident's network scope even long after the fact. It complements full packet capture (PCAP), which is storage-heavy but provides payload contents for targeted deep analysis: flow for broad scalable detection/scoping, PCAP for selective content. Flow data pairs with IDS alerts and SIEM correlation, building on the networking path's traffic concepts — extending the SOC's network visibility efficiently.
Common mistakes
- Expecting payload contents from flow data (flow is metadata only — use PCAP for contents).
- Not retaining flow data long enough to investigate late-discovered incidents' network scope.
- Overlooking flow patterns (beaconing, volume anomalies, scanning) that reveal threats.
Key takeaways
- Flow data summarizes each connection's metadata (5-tuple, byte/packet counts, timing) without payloads — lightweight enough to retain at scale across the whole network.
- It reveals connection patterns (volume anomalies/exfiltration, beaconing/C2, scanning/lateral movement) and maps an incident's network scope, even long after the event.
- It complements full packet capture (PCAP, payload-rich but storage-heavy): flow for broad scalable detection/scoping, PCAP for selective deep content analysis; both pair with IDS and SIEM.
Check your understanding
- How does flow data differ from full packet capture, and what advantage does that give it?
- What threat patterns can flow data reveal, and when would you need PCAP instead?