Hardware, Software Advances Help Protect Operating Systems From Attack

The operating system (OS) is the backbone of your computer. If the OS is compromised, attackers can take over your computer – or crash it. Now researchers at North Carolina State University have developed an efficient system that utilizes hardware and software to restore an OS if it is attacked.

At issue are security attacks in which an outside party successfully compromises one computer application (such as a Web browser) and then uses that application to gain access to the OS. For example, the compromised application could submit a “system call” to the OS, effectively asking the OS to perform a specific function. However, instead of a routine function, the attacker would use the system call to attempt to gain control of the OS.

“Our goal is to give the OS the ability to survive such attacks,” says Dr. Yan Solihin, an associate professor of electrical and computer engineering at NC State and co-author of a paper describing the new system. “Our approach has three components: attack detection; security fault isolation; and recovery.”

The concept is to take a snapshot of the OS at strategic points in time (such as system calls or interrupts), when it is functioning normally and then, if the OS is attacked, to erase everything that was done since the last “good” snapshot was taken – effectively going back in time to before the OS attack. The mechanism also allows the OS to identify the source of the attack and isolate it, so that the OS will no longer be vulnerable to attacks from that application.

The idea of detecting attacks and re-setting a system to a safe state is a well-known technique for restoring a system’s normal functions after a failure, but this is the first time researchers have developed a system that also incorporates the security fault isolation component. This critical component prevents the OS from succumbing to the same attack repeatedly.

The concept of taking snapshots of the OS and using it to replace the OS if it is compromised was previously viewed as impractical, since taking these snapshots and running such a system significantly slowed computer operating speeds. “But we’ve developed hardware support that allows the OS to incorporate these survivability components more efficiently, so that they take up less time and energy,” Solihin says. The researchers say the survival system takes up less than 5 percent of the OS’s operating overhead.

The paper, “Architectural Framework for Supporting Operating System Survivability,” was co-authored by Solihin and former NC State Ph.D. student Xiaowei Jiang. The paper will be presented Feb. 16 at the IEEE International Symposium on High-Performance Computer Architecture in San Antonio, Texas. The research was supported, in part, by the National Science Foundation.

NC State’s Department of Electrical and Computer Engineering is part of the university’s College of Engineering.

-shipman-

Note to Editors: The study abstract follows.

“Architectural Framework for Supporting Operating System Survivability”

Authors: Xiaowei Jiang and Yan Solihin, North Carolina State University

Presented: Feb. 16, 2011, at the IEEE International Symposium on High-Performance Computer Architecture in San Antonio, Texas

Abstract: The ever increasing size and complexity of Operating System (OS) kernel code bring an inevitable increase in the number of security vulnerabilities that can be exploited by attackers. A successful security attack on the kernel has a profound impact that may affect all processes running on it. In this paper we propose an architectural framework that provides survivability to the OS kernel, i.e. able to keep normal system operation despite security faults. It consists of three components that work together: (1) security attack detection, (2) security fault isolation, and (3) a recovery mechanism that resumes normal system operation. Through simple but carefully-designed architecture support, we provide OS kernel survivability with low performance overheads (<5% for kernel intensive benchmarks). When tested with real world security attacks, our survivability mechanism automatically prevents the security faults from corrupting the kernel state or affecting other processes, recovers the kernel state and resumes execution.