web logs

Overview

Every web server keeps an access log: one line per HTTP request, recording who asked for what and how the server answered. For an analyst, this is a rich narrative of everything that touched an application. In a CTF forensics challenge, web logs reveal reconnaissance scans, exploitation attempts, and the exact request that succeeded. This lesson teaches you to read web logs analytically — interpreting status codes, user agents, and requested paths to separate normal traffic from suspicious activity.

You work on provided or self-generated logs in an authorized, isolated lab. The goal is defensive: recognizing an attack from the trail it leaves in the access log.

Learning objectives

• Interpret HTTP status codes as signals of what happened to a request.
• Read the User-Agent string and understand its value and its limits.
• Spot suspicious requested paths: sensitive files, admin panels, and injection-like payloads.
• Combine status, path, and user agent to distinguish scanning from a successful attack.

Core concept

A web access log line (recall the combined format) records, for each request: the client IP, the timestamp, the request line (method, path, protocol), the status code, the response size, the Referer, and the User-Agent. Analysis means reading these together. The path tells you what the attacker wanted; the status code tells you whether they got it; the User-Agent hints at the tool making the request; and the size can reveal whether a response contained real data. No single field convicts — the combination tells the story.

How it works

Status codes group by first digit:

• 2xx (success) — the request worked. 200 OK on a sensitive path is a red flag: the attacker got what they asked for.
• 3xx (redirect) — the client was sent elsewhere (e.g. 301, 302).
• 4xx (client error) — the request failed at the client's fault. 404 Not Found in bulk is the signature of scanning (probing for files that mostly don't exist); 401/403 mean unauthorized/forbidden; 400 is malformed.
• 5xx (server error) — the server failed. A 500 right after an odd payload can indicate an exploit attempt that broke the application.

User agents. The User-Agent string identifies the client software. Real browsers send long, specific strings (Mozilla/5.0 (...) ...). Automated tools sometimes send default agents naming a scanner or a scripting library, and empty or bizarre agents are suspicious. But the field is attacker-controlled and trivially forged, so treat it as a hint, never proof. A default scanner user agent is a gift; its absence proves nothing.

Suspicious paths. The requested path often reveals intent:

• Probes for sensitive files: /.env, /.git/config, /wp-admin, /admin, /phpmyadmin, backup files like /backup.zip.
• Path traversal attempts: ../ sequences trying to escape the web root, e.g. /download?file=../../etc/passwd.
• Injection-like payloads in query strings: SQL-injection fragments (' OR '1'='1), or cross-site-scripting fragments (<script>). Note: these are illustrative patterns to recognize in logs, not instructions to attack anything.
• A burst of many different paths from one IP in seconds — classic automated directory scanning.

Reading them together. A hundred 404s to random admin paths from one IP is scanning — noisy but unsuccessful. The line that matters is when one of those probes returns 200 (or a 500 on an injection payload): that is where scanning turned into a hit.

Practical example

A challenge gives you access.log. Grouping by client IP, one address stands out with 200+ requests in under a minute — nearly all 404, hitting paths like /admin, /.git/config, /backup.zip, /phpmyadmin. That is a directory scan, and its curl-like User-Agent reinforces the guess. You keep reading and find:

203.0.113.9 - - [25/Sep/2025:14:07:40 +0000] "GET /.git/config HTTP/1.1" 200 1180 "-" "curl/8.4.0"

Among the wall of 404s, this one is 200 with a non-trivial response size — the scanner found an exposed .git config. That single line is the pivot: it tells you which probe succeeded, at what time, and from which IP. In a correlation challenge, this timestamp and IP link straight to whatever the attacker did next. The flag usually hides in that successful request.

Cybersecurity relevance

Web access logs are frontline evidence for detecting attacks against applications. Security teams and WAFs (Web Application Firewalls) watch for exactly these signals: floods of 404s (scanning), requests for sensitive paths, traversal sequences, and injection-like payloads in query strings. Distinguishing attempted from successful attacks — a 403 denial versus a 200 hit — is what turns raw logs into an incident assessment. Because the User-Agent is forgeable, mature analysis leans on the harder-to-fake combination of path, status, timing, and volume. Reading web logs well lets you answer whether an application was merely probed or actually breached, and pinpoint the request that did it.

Common mistakes

• Trusting the User-Agent as identity. It is attacker-controlled; use it as a hint, corroborated by behavior.
• Reading status codes in isolation. A 200 is only alarming in context — on a sensitive path or after a payload. Pair status with path.
• Ignoring response size. A successful probe often returns a larger body than an error; size can distinguish a real hit from an empty one.
• Missing the one success in a sea of failures. Bulk 404s are just scanning; the 200/500 among them is the event that matters.
• Overlooking encoded payloads. Traversal and injection fragments may be URL-encoded in the path; decode before judging.

Key takeaways

• Web access logs record IP, time, request line, status, size, Referer, and User-Agent — read them together.
• Status codes: 2xx success, 3xx redirect, 4xx client error (bulk 404 = scanning), 5xx server error (possible broken exploit).
• The User-Agent hints at the client tool but is forgeable — never treat it as proof.
• Suspicious paths (sensitive files, ../ traversal, injection-like query strings, path-scanning bursts) reveal intent; recognizing them is a defensive skill.
• The decisive event is scanning that turns into a hit: a 200 or 500 on a sensitive path or payload, which becomes your timeline pivot.

Check your understanding

• You see hundreds of 404 responses to varied admin and config paths from one IP, then a single 200 to /.git/config. What does the sequence tell you, and which line matters most?
• Why is the User-Agent field a weak basis for identifying an attacker, and what should corroborate it?
• Which combination of fields best distinguishes an attempted attack from a successful one in a web access log?