Protecting Linux Against Overflow Exploits

Thursday, September 23, 2010

Jamie Adams


An overflow is an anomaly where a program, while writing data to a memory buffer, overruns the buffer's boundary and overwrites adjacent memory—which could be maliciously exploited.

While the onus is on software developers to perform proper bounds checking there are some things you can do on an operational system to help protect against code which has been overlooked during development.

Most overflows are addressed during the development process because testing will often uncover erratic program behavior, memory access errors, incorrect results, and unexpected program terminations (crashes). Those which aren't identified and make it into production are often abused by attackers who manage to inject hostile code into this memory.

Program Memory Basics

Kernels are complicated and as such can be difficult to understand. For the purposes of this post, I will try to keep it simple but if any of my facts are incorrect please, add a comment so we all might learn. With that said, running programs establish two key areas in random access memory (RAM): stack and data segment.

The stack is a last in, first out (LIFO) structured linear list. In this structure, elements can be added or taken off from only one end, called the “top”.

The data segment comprises of “data”, BSS, and heap.

  • The Data area contains global and static variables used by the program that are initialized. This segment can be further classified into initialized read-only area and initialized read-write area.
  • The BSS segment also known as uninitialized data starts at the end of the data segment and contains all uninitialized global variables and static variables that are initialized to zero by default. (Geek Trivia: Historically, BSS (from Block Started by Symbol) was a pseudo-operation in UA-SAP (United Aircraft Symbolic Assembly Program), the assembler developed in the mid-1950s for the IBM 704.)
  • The Heap space is dynamic memory. The heap grows due to the dynamic allocation of memory storage for use by a program during the runtime of that program and shrinks when it is released. Dynamically allocated memory exists until it is released either explicitly by the programmer, or by a garbage collector (e.g., Java and Python).

One approach to buffer overflow protection is called executable space protection. This approach prevents the execution of code on the stack or the heap. An attacker may use buffer overflows to insert arbitrary code into the memory of a program, but with executable space protection, any attempt to execute that code will cause an exception.

Hardware Support

Some CPUs support a feature called NX ("No eXecute") or XD ("eXecute Disabled") bit, which in conjunction with software, can be used to mark memory pages of data (such as those containing the stack and the heap) as readable and writeable but not executable.

Generically and on AMD processors, this ability is called NX, while on Intel processors it is called XD. AMD has also marketed this technology as “Enhanced Virus Protection”. In the context of this post, when I refer to the “NX feature” I am including XD.

To determine if your CPU has such support, check the value of flags in /proc/cpuinfo to see if it includes pae (physical address extensions) and nx:

$ egrep '^flags' /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 constant_tsc up pni monitor nx

You will get a flags line for each CPU available to your operating system. If your operating system resides on a virutalized guest and you know your hardware supports it but you don't see the appropriate flags, check your virtual machine settings. You will typically see a setting like “Enable PAE/NX”.

It should also be noted that other processors which do not support PAE are AMD K6 and earlier, Transmeta Crusoe, VIA C3 and earlier, and Geode GX and LX. VMware Workstation versions older than 4.0, Parallels Workstation versions older than 4.0, and Microsoft Virtual PC and Virtual Server do not support PAE on the guest.

Linux Kernel Support

If your hardware supports this feature, you should ensure that this protection is enabled in kernels running on 32-bit x86 systems. Other processors, such as Itanium, POWER, and 64-bit x86 (both AMD64 or Intel 64), have included such support since inception and the standard kernel for those platforms already supports the feature.

Most Linux distributions bundle NX support with a PAE-enabled kernel (kernel-PAE). However, some people don't install the Physical Address Extension (PAE) kernel because they think it is just to provide support for physical memory above 4GB. To install the package use either yum(8) or Zypper:

# yum install kernel-PAE

Reboot using the new kernel and then use the -r option to the uname(1) utility to ensure the running kernel is PAE-enabled (you will see the letters ‘PAE’ in the kernel name).

These are the recommended procedures detailed in section of NSA's Guide to the Secure Configuration of Red Hat Enterprise Linux 5.

I have received phone calls from Security Blanket customers attempting to address DISA UNIX STIG PDI “GEN003540 – Disable Executable Stack”. The guideline states:

“Linux kernels must support the NX feature. Red Hat Enterprise 4 and SuSE 9.1 and later do support this feature. This will be a finding on systems prior to the above releases. This is a manual review.

However, the guideline provides no procedures on how to perform the “manual review”. I hope this blog post helps those of you who must perform this manual check. The upcoming release of Security Blanket scheduled for the Fall of 2010 will include some modules to perform these checks for you.

NX memory protection has always been available in Ubuntu for any systems that had the hardware to support it and ran the 64-bit or 32-bit server kernel. The 32-bit PAE desktop kernel (linux-image-generic-pae) in Ubuntu 9.10 and later, also provides the PAE mode needed for hardware with the NX CPU feature. For systems that lack NX hardware, the 32-bit kernels now provide an approximation of the NX CPU feature via software emulation that can help block many exploits an attacker might run from stack or heap memory.

By default, Red Hat-based and Fedora systems have enabled “Exec Shield”. To read the announcement made by the developer of this kernel patch, Ingo Molnar, click here.

Exec Shield's legacy CPU support approximates (Ingo Molnar's word for it) NX emulation by tracking the upper code segment limit. This imposes only a few cycles of overhead during context switches, which is for all intents and purposes immeasurable.

Add the following lines to /etc/sysctl.conf:

kernel.exec-shield = 1 kernel.randomize_va_space = 1

If you don't want to reboot, use the -w option to the sysctl(8) utility to set these parameters in the current running kernel.

Enabling Exec Shield is a recommended procedure detailed in section of NSA's Guide to the Secure Configuration of Red Hat Enterprise Linux 5.

Prevent Core Dumps

In the event a program experiences memory access errors or unexpectedly terminates (crashes), the kernel can create a core dump file. The most notorious cause of core dumps (and dreaded by developers) is the segmentation violation (SIGSEGV).

The core dump consists of the recorded state of the working memory, including processor registers, which may include the program counter and stack pointer, memory management information, and other processor and operating system flags and information.

Because the core dump may contain sensitive information, many security guidelines recommend preventing them from occurring on production systems. Here are some guidelines which recommend disabling core dumps:

  • NSA's Guide to the Secure Configuration of Red Hat Enterprise Linux 5, Sections and
  • CIS RHEL4 Benchmark (1.0.5), Section 8.10 - Disable Core Dumps
  • CIS RHEL5 Benchmark (1.1.2), Section 9.10 - Disable Core Dumps
  • CIS SUSE Benchmark (2.0), Section 8.11 - Disable Core Dumps
  • DISA UNIX STIG, PDI GEN003500 - Disable Core Dumps

To disable core dumps for all users, add or correct the following line in /etc/security/limits.conf:

* hard core 0

In addition, to ensure that core dumps can never be made by setuid programs, edit /etc/sysctl.conf and add or correct the line:

fs.suid_dumpable = 0 Summary

So if your hardware or virtualization framework supports it, enable it and ensure your kernel is taking advantage of it. Security Blanket has many modules to address the above security guidelines. If you are interested in an easier way to implement NSA's Guide to the Secure Configuration of Red Hat Enterprise Linux 5, keep an eye out for our 2010 Fall release.

Cross-posted from Security Blanket Technical Blog

Thank you Wikipedia for the hardware specifics!

Possibly Related Articles:
Operating Systems
Information Security
Linux operating system
Post Rating I Like this!
Jamie Adams @Mike.. thanks for the information. I will most certainly give it a read. As I said before, I don't have much experience with Gentoo. However, I noticed in the link it mentioned SELinux -- which I am familiar with... my company actually authored many components of it for NSA. I am curious if they are delivering custom policies for it. Thanks again.
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.