RedTeam Nexus Labs
Module 21 · Lesson 01 // Reverse Engineering Basics

Executables

Reverse Engineering Basics · CTF

Freereading · 10 min
On this page
LESSON

Overview

Reverse engineering (RE) challenges hand you a compiled program and ask you to figure out what it does — often to find the condition that reveals a flag. Before you can analyze a binary, you need to understand what an executable actually is: how human-readable source code becomes a file the computer runs, and what that file contains. This lesson builds that foundation in a safe, authorized CTF setting, where every binary you examine was provided specifically for you to study.

Learning objectives

  • Explain how source code becomes an executable (compilation).
  • Describe what machine code is and why it is hard for humans to read directly.
  • Identify the two most common executable formats: ELF and PE.
  • Recognize what a binary still contains that an analyst can use.

Core concept

An executable is a file the operating system can load into memory and run directly on the CPU. When a developer writes a program in a language like C, that human-readable source code is not what runs. A tool called a compiler translates the source into machine code — raw numeric instructions the processor understands — and packages it, together with data and bookkeeping information, into an executable file in a specific format.

The key insight for reverse engineering is that this translation is largely one-directional and lossy. The compiler discards things that mattered to the programmer but not to the CPU: variable names, comments, and much of the original structure. Recovering meaning from a compiled binary is exactly the challenge RE poses.

How it works

The typical journey from source to running program has a few stages:

  1. Source code — human-readable text (for example, C or C++).
  2. Compilation — the compiler translates source into assembly, a low-level, still-somewhat-readable representation of CPU instructions.
  3. Assembly to machine code — an assembler converts assembly into binary machine code (opcodes and operands as bytes).
  4. Linking — the linker combines your code with libraries and produces the final executable file.

The result is stored in a standard container format so the operating system knows how to load it:

  • ELF (Executable and Linkable Format) — used on Linux and most Unix-like systems.
  • PE (Portable Executable) — used on Windows (.exe, .dll).

These formats define sections inside the file, such as a section holding the executable instructions (commonly called .text), a section for initialized data, and headers describing how everything is laid out in memory. An analyst reads these headers and sections to orient themselves before diving into the code.

Practical example

Consider a tiny C program that prints a message:

#include <stdio.h>

int main(void) {
    printf("Access granted\n");
    return 0;
}

After compiling it into an ELF executable, the file no longer contains the words main, printf, or your source layout as readable prose. Instead it contains machine-code bytes that, when executed, call into the C library to print the string. However, the literal text Access granted is typically stored in a data section of the file, so it survives compilation as readable bytes. That surviving string is the kind of clue an analyst looks for first — the topic of the next lesson.

If you inspect the file type of a compiled Linux program, a tool reports something like ELF 64-bit executable, immediately telling you the format and therefore which tools and expectations apply.

Cybersecurity relevance

Understanding executables underlies both offensive analysis and defense. Malware analysts, incident responders, and CTF players all start from the same reality: they usually have only the compiled artifact, not the source. Knowing that a binary is ELF or PE tells you which loader, calling conventions, and tooling to use. Recognizing which information survives compilation (strings, imported library names, overall structure) versus what is lost (variable names, comments) lets you set realistic expectations and choose an efficient approach. This is the groundwork for malware triage, vulnerability research, and software forensics.

Common mistakes

  • Expecting to see the original source. Compilation discards names and comments; you work with machine code and whatever artifacts survived.
  • Ignoring the file format. Treating a PE file like an ELF (or vice versa) wastes time; always identify the format first.
  • Assuming the binary is unreadable. Even without source, headers, sections, strings, and imported function names carry a lot of meaning.

Key takeaways

  • An executable is compiled machine code packaged in a container format the OS can load and run.
  • Compilation is lossy: source names and comments are gone, but instructions, data, and some artifacts remain.
  • ELF (Linux/Unix) and PE (Windows) are the two dominant executable formats; identifying the format is the first step.
  • Reverse engineering is the discipline of recovering meaning from that compiled artifact — always in an authorized, isolated setting.

Check your understanding

  • What does a compiler discard when turning source code into an executable, and why does that matter for reverse engineering?
  • What is the difference between ELF and PE, and why should you identify the format before analyzing a binary?
FINISH THIS LESSON

Sign in to save your progress.