Overview
Cryptographic algorithms like AES are extremely strong when used correctly — and easy to undermine when used incorrectly. Most real-world crypto failures are not broken math; they are misuse: the wrong mode, a reused value that should be unique, a weak or outdated primitive, a hard-coded secret, or home-grown "encryption." This lesson catalogs the classic mistakes so you can recognize them in CTF challenges and avoid them in real systems. The theme throughout: use standard, correct constructions; do not roll your own crypto.
Learning objectives
- Explain why ECB mode leaks patterns and should be avoided.
- Describe the danger of IV/nonce reuse.
- Recognize weak or broken primitives (MD5, SHA-1) and where they must not be used.
- Understand why hard-coding keys and inventing your own crypto are dangerous.
Core concept
Cryptography fails safely only when it is used as designed. The recurring mistakes are:
- ECB mode — In Electronic Codebook mode, each block is encrypted independently, so identical plaintext blocks produce identical ciphertext blocks. This leaks structure and repetition in the data. (The well-known illustration is encrypting a simple bitmap image with ECB: the outline of the picture is still visible in the ciphertext.) Use an authenticated mode instead.
- IV / nonce reuse — Many modes need a unique initialization vector (IV) or nonce per encryption. Reusing one with the same key is dangerous: in stream/counter modes (CTR, and GCM) it causes keystream reuse, so XORing two ciphertexts reveals the XOR of the plaintexts, and for GCM it also breaks the authentication guarantee. In CBC, a predictable or reused IV leaks information about message relationships. IVs must be unique (and for CBC, unpredictable).
- Weak/broken primitives — MD5 and SHA-1 have practical collision attacks and must not be used where collision resistance matters; likewise, fast general-purpose hashes are the wrong tool for passwords.
- Hard-coded keys — a key embedded in source or a binary is extractable, so the encryption protects nothing.
- Rolling your own crypto — custom or obfuscation-based schemes almost always contain fatal flaws that expert review would catch.
How it works
These weaknesses turn a "strong" cipher into a solvable puzzle. With ECB, an analyst sees repeated ciphertext blocks and infers repeated plaintext — sometimes enough to decode structured data without any key. With nonce reuse in a stream cipher, two messages C1 = P1 XOR KS and C2 = P2 XOR KS share the same keystream KS, so C1 XOR C2 = P1 XOR P2, and known or guessable plaintext peels the rest apart. With MD5/SHA-1, an attacker can craft two different inputs sharing a digest, undermining signatures and integrity checks that assumed uniqueness. A hard-coded key is simply read out of the code or binary. Home-grown crypto typically leaks patterns (like ECB) or uses reversible encoding dressed up to look secure.
Practical example
An isolated CTF challenge provides a login token that is AES-ECB-encrypted, one block per field. Because ECB encrypts identical blocks identically, a solver can duplicate and rearrange ciphertext blocks — a cut-and-paste manipulation — to forge a valid-looking token without ever knowing the key. The flaw is not AES; it is the mode. Switching to an authenticated mode with a unique IV per message removes the weakness entirely.
A second common challenge gives you two messages encrypted with a stream cipher under the same key and same nonce. XORing the two ciphertexts cancels the shared keystream and exposes the XOR of the plaintexts; with a guess at one message's content, the other falls out. The takeaway for real systems: never reuse an IV/nonce under the same key.
Cybersecurity relevance
These misuse patterns are exactly what security reviews and CTF crypto challenges look for, because they appear constantly in real code: databases encrypted with ECB, tokens generated with reused nonces, integrity checks still trusting MD5, and secrets hard-coded into shipped applications. Recognizing the anti-pattern is the whole game — the underlying primitive is often fine, and the fix is to use it correctly: an authenticated mode (which provides confidentiality and integrity), unique IVs/nonces, modern hashes (SHA-256 family for integrity; bcrypt/scrypt/Argon2 for passwords), externalized keys, and vetted libraries rather than custom schemes.
Common mistakes
- Using ECB mode, which leaks plaintext patterns and enables block cut-and-paste attacks.
- Reusing an IV or nonce with the same key, breaking confidentiality (and, for GCM, authentication).
- Relying on MD5 or SHA-1 for signatures or integrity where collision resistance is required.
- Hard-coding keys in source or binaries, making the encryption pointless.
- Rolling your own crypto or treating reversible encoding as encryption, instead of using standard, reviewed constructions.
Key takeaways
- Most crypto failures are misuse, not broken algorithms — the details (mode, IV, primitive, key handling) decide security.
- Avoid ECB; use an authenticated mode and a unique IV/nonce per encryption under a given key.
- Do not use MD5/SHA-1 where collisions matter; use modern hashes, and slow salted hashes for passwords.
- Never hard-code keys and never roll your own crypto — rely on standard, well-reviewed libraries.
Check your understanding
- Why does ECB mode allow an attacker to detect (and sometimes manipulate) repeated data without knowing the key?
- If two messages are encrypted with a stream cipher using the same key and the same nonce, what can an attacker recover, and why?