Malicious Input: How Hackers Use Shellcode

It’s Black Hat 2019 this week, with Def Con following hard on its heels, and even if you’re not going to either of these events, if you have a stake in the world of cybersecurity your social media feeds are going to be filled with plenty of “hacker talk” over the next 7 days or so. Of course, you know all about hashes in cybersecurity and how to decode Base64; you’re likely also familiar with steganography, and maybe you can even recite the history of cybersecurity and the development of EDR. But how about explaining the malicious use of shellcode? You know it has nothing to do with shell scripts or shell scripting languages like Bash, but can you hold your own talking about what shellcode really is, and why it’s such a great tool for attackers?

Not sure? No problem. We’ve got just the post for you. In the next ten minutes, we’ll take you through the basics of shellcode, what it is, how it works and how hackers use it as malicious input.

What is Shellcode?

We know shellcode has nothing to do with shell scripting, so why the name? The connection with the shell is that shellcode was originally mainly used to open or ‘pop’ a shell – that is, an instance of a command line interpreter – so that an attacker could use the shell as a means to compromise the system. Imagine if you could get a user to input some seemingly innocent string to a legitimate program on their system that would magically open a reverse shell to your machine? That’s the ultimate pwning prize. It also takes very little code to spawn a new process that will give a shell, so popping shells is a very lightweight, efficient means of attack.

In order to achieve it, you’d need to find an exploitable program and fashion some malicious input string – the shellcode – containing small chunks of executable code to force the program into popping a shell. This is possible because for most programs, in order to be useful, they need the ability to receive input: to read strings and other data supplied by the user or piped in from another program.

Shellcode exploits this requirement by containing instructions telling the program to do something it otherwise wouldn’t or shouldn’t. Of course, almost no program is going to easily misinterpret data as code without a bit of persuasion. Programs are designed to take in data only of a certain type – numbers, strings, dates and such like – and anything else of a different type will just be rejected. However, we can trick programs into treating specially-formatted strings, aka “shellcode”, as program instructions by means of another hacking conversation favorite: the buffer overflow.

What is a Buffer Overflow?

A buffer overflow occurs when a program writes data into memory that is larger than the area of memory, the buffer, the program has reserved for it. This is a programming error, as code should always check first that the length of any input data will not exceed the size of the buffer that’s been allocated. When this happens the program may crash, but maliciously-crafted input may instead allow an attacker to execute their own code when it overflows into an area of executable memory. Here’s a simple example of a buffer overflow waiting to happen.

The program reserves 16 bytes of memory for the input, but the size of the input is never checked. If the user enters a string longer than 16 bytes, the data will overwrite adjacent memory – a buffer overflow. The image below shows what happens if you try to execute the above program and supply it with input greater than 16 bytes:

However, just causing a buffer overflow in a program isn’t on its own much use to attackers, unless all they want to do is bring the application to a crashing halt. While that would represent a win of sorts for attackers whose objective is some kind of denial of service attack, the greater prize in most cases is not just causing the overflow but using it as a means to take control of execution. Once this is achieved, the target device can fall under the hacker’s complete control.

Controlling Code Execution

When we create a buffer overflow, the aim is to write a sufficiently large amount of data into the program’s memory so that two things happen. First, we fill up the allocated buffer, and second we supply enough extra data so that we overwrite the address that will be executed next with our own code.

This isn’t simple, but it might sound harder to do than it actually is. Because of the nature of how program memory is mapped out, when any function is called, there’s always a pointer held in memory to the address of the next function that should be executed after the currently executing one; this pointer is known as the Instruction Pointer, sometimes referred to as EIP (32 bit) or RIP (64 bit). I’ll use RIP throughout the rest of this post, but the same observations apply to EIP.

By reverse engineering a particular program and with a lot of fuzzing and experimenting, we can determine both whether a given program contains any functions that are vulnerable to a buffer overflow and, if so, the address of the Instruction Pointer when that vulnerable function has finished calling.

Knowing the offset – the memory address – of the Instruction Pointer at that point in code means we can determine precisely how much extra data we need to overflow the buffer and insert our own code at the address of the Instruction Pointer. When we do that, the program will try to execute the code at the address we’ve written to the RIP register. If that code is junk, like in the example above, the program will crash, but if it isn’t – if it’s a valid address, things start to get more interesting.

From Buffer Overflow to Shellcode

Having achieved a buffer overflow and mapped out the memory addresses and offsets, the attacker has two options. Either the malicious input could write the address of another function of the program to RIP or the attacker could try to jump to an address in which attack code has already been inserted.

The first case may often times be all that is needed. Suppose the program contains a function that is normally only reached if the user is authorized to take that action. In such a case, the attacker can use the buffer overflow to write the address of that function directly into the Instruction Pointer, bypassing any earlier function that would have checked for authorization.

The second scenario is far more tricky. Suppose I want the program to do something it’s not programmed to do, like open up a shell? In that case, I need to overflow the buffer not just with junk or an address to jump to, but with executable code – shellcode – as well as the address of where that code begins.

How To Create Shellcode

Now that we understand the mechanism that shellcode exploits, how do we go about creating it? As we saw already, we need to send executable code in the input data, but we can’t just write a bunch of C or C++ instructions into the input.

If we want the program to write executable instructions into memory, then we need to send it raw assembly code in our string. However, there’s a difficulty.

When C-like programs read strings – an array of chars – from user input, they need to know when the input has come to an end. Character strings in C are terminated by appending a special null-byte character to the end of the character array. This null-byte character has the value of zero. It is represented by x0 in shellcode and 00 in hex. When the program encounters the null-byte character, signalling the end of the string, it will cease reading further input and move on to the next instruction in its code.

And herein lies the difficulty: the string-terminating null-byte character has the value of zero, but it is not the same thing as the integer 0. However, they both share the same hex value, 00, and are thus represented in shellcode the same way, as x0.

That means we cannot pass the integer 0 directly in our shellcode, as any x0 will be interpreted by the program as a null-byte character and signal the end of input. As a result, the rest of our code after x0 will be discarded.

In order to solve this problem and create well-formed shellcode, we need to go through several steps.

1. Create the program we want to execute in a high-level language like C. As it has to fit in a small amount of memory – the size of the buffer plus the offset to RIP – it should be as concise as possible. The shellcode used in this example, which spawns a shell via execve, is a mere 23 bytes.

2. After compiling the code and checking that it does what we expect, use a disassembler to view the raw assembly.

3. Optimize the assembly, such as replacing any 00 hex with other instructions. In the image above, the first column (left of the colon) shows us the memory address, the second column shows us the opcodes and operands (program instructions) in hex, and the remaining columns on the right show us the same in a human-readable language.

This code has already been optimized. Notice at line 8 the hexadecimal 48 31 f6, which represents the instructions for the following assembly:

xor %rsi, %rsi

The use of XOR here is an example of sidestepping the restriction on not being able to use the hex 00 we mentioned above. This particular program needs to push the integer 0 onto the stack. To do so, it first loads 0 into the %rsi register. The natural way to do that would be:

mov $0x0, %rsi

But that would produce the hex 00 to represent $0x0. We can get around that by using XOR on the same value, %rsi. When both input values to XOR are the same the result will be 0, so this instruction returns 0 as a result but doesn’t require a 00 in the assembly (the opcode for XOR is 0x31 in Intel 32-bit and 64-bit architectures).

4. Next, we need to extract the hexedecimal program instructions and create a shellcode string. To do that we prefix each hex byte with x and concatenate them all as a single string.

x48x31xf6x56x48xbfx2fx62x69x6ex2fx2fx73x68x57x54x5fx6ax3bx58x99x0fx05

5. Finally, we can now feed our shellcode to a vulnerable program, or create our own and convince a user to execute it, like this one.

Some other great examples of this process can be seen here.

Protecting Against Shellcode

You would think that buffer overflows, which have been known about for decades, should be becoming rarer, but in fact the opposite is true. Statistics from the CVE database at NIST show that vulnerabilities caused by buffer overflows increased dramatically during 2017 and 2018. The number known for this year is already higher than every year from 2010 to 2016, and we still have almost 5 months of the year left to go.

Clearly, there’s a lot of unsafe code out there, and the only real way you can protect yourself from exploits that inject shellcode into vulnerable programs is with a multi-layered security solution that can not only use Firewall or Device controls to protect your software stack from unwanted connections, but also that uses static and behavioral AI to catch malicious activity both before and on execution. With a comprehensive security solution that uses machine learning to identify malicious behavior, attacks by shellcode are seen just like any other attack and stopped before they can do any damage.

Conclusion

In this post, we’ve taken a look at what shellcode is and how hackers can use it as malicious input to exploit vulnerabilities in legitimate programs. Despite the long history of the dangers of buffer overflows, even today we see an increasing number of CVEs being attributed to this vector. Looking on the bright side, attacks that utilize shellcode can be stopped with a good security solution. On top of that, if you find yourself in the midst of a thread or a chat concerning shellcode and malicious input, you should now be able to participate and see what more you can learn from, or share with, others!

Like this article? Follow us on LinkedIn, Twitter, YouTube or Facebook to see the content we post.

Read more about Cyber Security

• EternalBlue & The Lemon_Duck Cryptominer Attack
• Firewall Vulnerabilities | Is Your Data Leaking Like Capital One?
• Can Tricky TxHollower Malware Evade Your AV?
• MegaCortex | Malware Authors Serve Up Bad Tasting Ransomware
• 7 Ways Hackers Steal Your Passwords
• The Good, the Bad and the Ugly in Cybersecurity – Week 31