How Does Return-Oriented Programming Bypass DEP?

TL;DR
Return-oriented programming bypasses data execution prevention by chaining existing instruction sequences that end in returns, so an exploit can perform malicious computation without injecting new executable code. A practical exploit first redirects the stack pointer into attacker-controlled data, then uses a return-oriented payload either to perform the computation directly or to prepare a conventional machine-code payload.
Transcript
Hello, my name is Dino Dai Zovi, and I'm here to talk about practical applications of return-oriented programming. So what I wanna talk about is how to use this technique that is ref-- called return-oriented programming to exploit memory corruption vulnerabilities. These are the larger class of vulnerabilities that most people colloquially refer to... Read More
Key Insights
- Return-oriented programming is an exploitation technique that combines existing instruction sequences followed by return instructions, allowing an attacker to construct malicious computations without adding new executable code to the target process.
- Return-to-libc simulates calls to functions in loaded libraries, while return-oriented programming specializes the idea by returning into smaller instruction sequences that may begin in the middle of instructions and need not have been intentionally written by the application.
- Return-oriented gadgets are combinations of reusable instruction sequences that perform higher-level operations, including placing chosen values into registers, writing a selected value to a selected memory location, performing arithmetic, and calling shared-library functions.
- A return-oriented program is assembled by scanning executable regions and shared libraries for useful sequences ending in returns, cataloging those sequences, and chaining them into gadgets that can support a Turing-complete language or compiler-generated payload.
- DEP is an exploit mitigation that uses processor features to stop memory pages containing data from being executed directly, but it does not prevent attackers from reusing code that already resides in executable memory.
- A stack pivot is the step that redirects the stack pointer into attacker-controlled data, allowing that data to supply the addresses and values used by a return-oriented payload. Stack overflows may provide this control directly, while other vulnerabilities require a pivot sequence.
- ASLR materially strengthens DEP because it makes the locations of reusable instruction sequences uncertain. If modules do not opt into ASLR, exploitation remains possible, while full ASLR can force an attacker to obtain a memory address disclosure before building a payload.
- The Aurora Internet Explorer exploit path uses a reference-counting error, a stale pointer, controlled heap allocations, crafted objects, and a forged vtable to reach an exchange EAX and ESP sequence that pivots the stack into attacker-controlled data.
Install to Summarize YouTube Videos and Get Transcripts
Explore YouTube Video Summarizer or Get YouTube Transcript Extractor
Questions & Answers
Q: What is return-oriented programming?
Return-oriented programming is a technique for exploiting memory corruption vulnerabilities by reusing instruction sequences that already exist in executable memory. The attacker finds sequences followed by return instructions and chains them together into gadgets. Those gadgets can perform operations such as loading registers, writing values, doing arithmetic, and calling library functions without introducing new executable code.
Q: How does return-oriented programming differ from return-to-libc?
Return-to-libc redirects execution into loaded library functions and simulates a function call, such as calling a function that executes shell commands. Return-oriented programming goes further by redirecting execution into short instruction sequences followed by returns. These sequences can even start inside other encoded instructions, and multiple sequences can be combined into gadgets that perform broader computations.
Q: How does return-oriented programming bypass DEP?
Return-oriented programming bypasses DEP by using instructions that are already located in executable memory instead of executing newly injected code from a data page. The attacker redirects the stack pointer into controlled data and chains addresses of useful instruction sequences. The resulting return-oriented stage can perform the computation itself or set up execution of a traditional machine-code payload.
Q: What is a gadget in return-oriented programming?
A gadget is a higher-level operation built by combining short instruction sequences that end in return instructions. For example, sequences that pop chosen values into EAX and ECX can be combined with a sequence that stores EAX through ECX. Together, those sequences let an attacker write a chosen value to a chosen memory location.
Q: What is a stack pivot in a return-oriented exploit?
A stack pivot redirects the stack pointer into data controlled by the attacker, allowing that data to act as a return-oriented payload. A basic stack buffer overflow may already provide control of the stack. With a heap overflow or another complex vulnerability, the attacker must redirect execution through a short sequence that explicitly moves the stack pointer into controlled memory.
Q: Why is ASLR important when DEP is enabled?
ASLR makes exploitation harder by changing the loaded locations of executable modules and the instruction sequences needed for return-oriented programming. Without ASLR, or when one or more modules do not opt into it, attackers can use preassembled components at known addresses. If every executable module uses ASLR, exploitation may require a separate memory address disclosure to reveal a DLL location.
Q: How was the Aurora Internet Explorer flaw exploitable?
The Aurora flaw involved an EventParam reference-counting error. When copied through createEventObject, it did not increment the reference count of a C Tree Node object. Removing the related HTML element could free that object while EventParam retained a stale pointer. Controlled heap allocations could then replace the freed block with crafted attacker-controlled object data.
Q: How did the Aurora exploit gain control of the stack pointer?
The exploit used controlled heap allocations to create a crafted C Tree Node that pointed to sprayed data representing a controlled C Element object. That object pointed to a forged vtable. A virtual function call through the forged vtable reached an exchange EAX and ESP instruction sequence, which redirected the stack pointer into attacker-controlled data containing the return-oriented payload.
Summary & Key Takeaways
-
Return-oriented programming reuses instruction sequences already present in executable memory, including shared libraries. Each sequence ends with a return and can be chained with others into gadgets. These gadgets perform operations such as loading registers, writing chosen values to chosen locations, doing arithmetic, or calling functions without introducing new executable code.
-
The technique challenges DEP, which prevents memory pages containing data from being executed directly. An attacker first performs a stack pivot, redirecting the stack pointer into controlled data. A return-oriented stage can then execute entirely through reused instructions or prepare the conditions needed to launch a traditional machine-code payload.
-
The Aurora Internet Explorer example begins with a reference-counting flaw that leaves an EventParam object holding a stale pointer to freed memory. Controlled heap allocations replace that memory with crafted objects, leading a virtual function call through a forged vtable to a stack-pivot sequence and attacker-controlled return-oriented payload data.
Read in Other Languages (beta)
Share This Summary π
Summarize YouTube Videos and Get Video Transcripts with 1-Click
Try YouTube Summary with ChatGPT & Claude or YouTube Transcript Generator
Explore More Summaries from RSAC Cybersecurity π






Summarize YouTube Videos and Get Video Transcripts with 1-Click
Try YouTube Summary with ChatGPT & Claude or YouTube Transcript Generator