\svnInfo $Id$

Commodity operating systems are tasked with storing and processing
our most sensitive information, from managing financial
and medical records to carrying out online purchases. Unfortunately, the
assurance of these systems often falls short. This poor assurance
appears to be the result of the complexity inherent in these systems
--- while the generality of a commodity operating system makes it
useful for many tasks, it also makes it difficult to secure.

Many solutions have been proposed for enhancing the assurance of these
systems. Some use microkernels to enhance assurance by running parts
of the OS, such as the file system and network stack, in separate
processes~\cite{appcore-smaller-tcb}. Others, such as NGSCB (formerly
Palladium)~\cite{england03:_trust_open_platf},
Proxos~\cite{ta-min06:_split_inter},
XOM~\cite{lie00:_archit_suppor_for_copy_and}, and
Overshadow~\cite{chen08:_overs}, attempt to retrofit an orthogonal,
higher assurance execution environment alongside a commodity OS,
allowing applications to run in a protected environment, but still use
the services of the OS.  The implications of allowing protected
applications to continue to rely on existing, untrusted OS services are
poorly understood. Relatively little is known about the problems that
arise for application security when part of the OS behave maliciously.

We explore this problem in the context of Overshadow, a
virtualization-based system we have developed that protects
applications inside a VM from the guest operating system running in
that VM, attempting to maintain the secrecy and integrity of an
applications data in the face of OS compromise. Our discussion
considers the implications of a malicious Linux OS. For each OS
component, we examine how malicious behavior could undermine
application secrecy and integrity, and suggest potential mitigations.
While we present our analysis and solutions in the context of Linux
and Overshadow, they are more generally applicable to any system
attempting to secure execution applications in the face of a
compromised OS.

We begin with a review of systems that enforce isolation between
protected applications and untrusted OSes in
Section~\ref{sec:background}. Next, we explain how a malicious OS can
subvert them, using false system call return values to trick an
application into revealing its secrets, and argue for a solution based
on a verifiable system call interface
(Section~\ref{sec:security}). Finally, we examine the implications of
making specific OS components untrusted, and propose defenses against
possible attacks, in Section~\ref{sec:components}.

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "paper"
%%% TeX-command-default: "Make"
%%% End: