correlation

Overview

A single log file tells part of a story; an intrusion spans many. The attacker's web request lands in the access log, the login they achieved lands in the auth log, and what they did afterward lands in application or system logs. Correlation is the skill of stitching these separate sources into one coherent timeline — following a single actor across files to reconstruct exactly what happened. This is the capstone of log analysis and, in a CTF forensics challenge, usually the final step that produces the answer.

You practice correlation on provided or self-generated logs in an authorized, isolated lab. It is a defensive, investigative discipline: turning scattered evidence into a narrative.

Learning objectives

• Explain why correlating multiple log sources beats analyzing any one alone.
• Identify the pivot fields (IP address, timestamp, username, session ID) that link records across sources.
• Build a normalized, chronological timeline from heterogeneous logs.
• Reconstruct an attack sequence: recon, access, and post-access activity.

Core concept

Correlation means linking records from different logs that refer to the same actor or event, so you can see a sequence no single log reveals. The links are pivot fields — values that appear in more than one source and tie their records together:

• Source IP address — the same IP in the web log and the auth log likely means the same actor.
• Timestamp — events close in time (once normalized to UTC) are probably related.
• Username / account — connects an authentication success to subsequent actions by that user.
• Session or request ID — when present, the strongest link, following one session across services.

Correlation depends entirely on the earlier lessons: you must read each format to extract the pivot fields, and you must normalize timestamps to UTC so events from different sources can be ordered against each other.

How it works

1. Normalize. Convert every source's timestamps to UTC and parse each format to pull out the pivot fields (IP, user, path, outcome, session). This is where log-format fluency and timestamp discipline pay off.
2. Pick a pivot. Start from a known suspicious value — often an IP flagged in the web log or an account with a failure burst in the auth log.
3. Gather. Collect every record matching that pivot across all sources.
4. Order. Sort the gathered records by UTC time into a single sequence, watching for clock skew between hosts (treat very close cross-host events as uncertain).
5. Read the story. The ordered sequence typically shows a recognizable arc: reconnaissance (scanning in the web log) → access (a login success in the auth log) → post-access activity (actions in application or system logs).

Where a shared IP or session ID exists, you can follow the actor even as they move from the web tier to the login system to backend actions.

Practical example

Three files from a lab exercise, all normalized to UTC:

Web access log:

203.0.113.9 - - [25/Sep/2025:14:03:00 +0000] "GET /admin HTTP/1.1" 404 209 "-" "curl/8.4.0"
203.0.113.9 - - [25/Sep/2025:14:03:02 +0000] "GET /.git/config HTTP/1.1" 200 1180 "-" "curl/8.4.0"

Auth log (syslog, host in UTC):

Sep 25 14:05:11 web01 sshd[3001]: Failed password for admin from 203.0.113.9 port 40122 ssh2
Sep 25 14:05:41 web01 sshd[3050]: Accepted password for admin from 203.0.113.9 port 40460 ssh2

Application log (JSON):

{"time":"2025-09-25T14:06:03Z","event":"file_download","user":"admin","path":"/exports/customers.csv","src_ip":"203.0.113.9"}

The pivot is the IP 203.0.113.9, confirmed by the admin username linking the auth success to the app action. Ordered by UTC, the timeline reads: 14:03 scanned for admin paths and found an exposed .git/config (recon) → 14:05 brute-forced and gained SSH access as admin (access) → 14:06 downloaded a sensitive export (post-access impact). No single log showed all three; correlation on IP and username, ordered by normalized time, revealed the full intrusion. The challenge's answer — the exfiltrated file, the compromise time, the account used — lives in this assembled sequence.

Cybersecurity relevance

Correlation is the essence of incident response and the core function of a SIEM, which ingests logs from many sources, normalizes them, and links them on shared fields so analysts can query across an entire environment at once. Real investigations always span sources — network, host, authentication, application — and the analyst's job is to reconstruct the attacker's path through all of them. The reconstructed timeline answers the questions that matter: how the attacker got in, what they accessed, and what the impact was. It also drives the response: knowing the compromised account and source IP tells defenders what to disable, block, and reset. Every skill in this module — reading formats, handling timestamps, spotting auth and web attack patterns — exists to feed this final act of correlation.

Common mistakes

• Analyzing sources in isolation. The story emerges only when logs are linked; a single file rarely shows the whole attack.
• Skipping normalization. Correlating without converting to UTC produces a wrong order and a wrong conclusion.
• Ignoring clock skew. Cross-host events seconds apart may be misordered; corroborate with pivot fields, not timestamps alone.
• Fixating on one pivot. An attacker may change IPs; follow username or session ID too, not just the address.
• Building the timeline but not reading it. The ordered sequence is a means to an end — interpret the recon → access → impact arc it reveals.

Key takeaways

• Correlation links records across logs via pivot fields — IP, timestamp, username, session/request ID — to build one timeline.
• It depends on the whole module: parse each format for pivots and normalize all timestamps to UTC before ordering.
• The method: normalize, pick a pivot, gather matching records, order by UTC (mind clock skew), then read the sequence.
• A correlated timeline typically reveals the attack arc: recon → access → post-access activity, which no single log shows alone.
• Correlation is the heart of incident response and what a SIEM automates; the assembled timeline answers how the attacker got in, what they took, and what to remediate.

Check your understanding

• Which fields commonly serve as pivots to link records across different logs, and why is a session ID especially strong?
• Walk through the steps to turn three unrelated log files into a single chronological attack timeline. Where do earlier lessons (formats, UTC normalization) come in?
• Why can correlating multiple sources reveal an intrusion that no single log makes visible?