\chapter{Architecture and Model}
\label{cha:arch}

\section{System Architecture}
\label{sec:arch:arch}

Our system is designed to be used in an environment like the one
depicted in \autoref{fig:arch:arch:arch}. We assume an application
that consists of one or more application servers that interact with a
storage system, typically a database server. Clients (\ie users)
interact with the system by sending requests to application servers;
they do not interact directly with the cache or storage.  The
application servers could be web servers running embedded scripts
(\eg Apache with \texttt{mod\_php} or \texttt{mod\_perl}), or dedicated
application servers that receive requests from front-end web servers
and implement the business logic (as with
with Sun's Enterprise Java Beans); both are common deployment options.

\begin{figure}[tbp]
  \centering
  \includegraphics[scale=0.75]{figures/txcache-arch}
  \caption{System architecture}
  \label{fig:arch:arch:arch}
\end{figure}

The storage layer is typically a relational database, though other
types of storage like key-value stores are possible. We assume that
the application uses this for storing all of its persistent state. In
order for our system to provide transactional guarantees, the storage
system itself must of course provide transaction support. We require
that it provides either serializable isolation of transactions, or
snapshot
isolation~\cite{berenson95:_critiq_of_ansi_sql_isolat_level}. Databases
that provide serializable isolation must do so using a concurrency
control mechanism that ensures that the commit order of transactions
matches a serial order; most common concurrency control
strategies, including strict two-phase
locking~\cite{eswaran76:_notion_consis_predic_locks_datab_system} and
optimistic concurrency
control~\cite{kung81:_optim_method_for_concur_contr}, have this
property. \autoref{cha:serializability-appendix} explains the
rationale for this requirement, and describes how to extend the system
to support databases where it does not hold.

\txcache's consistency protocol is based on versioning, and requires
some additional support from the storage layer, imposing some
additional requirements.  It must expose the ability to run read-only
transactions on a specific recent snapshot, and must report certain
validity information. These requirements are unique to our system; we
specify them precisely in
\autoref{sec:consistency-protocol:storage-requirements} and show how
to modify an existing multiversion concurrency control
system\footnote{Though not a strict requirement, these modifications
  are only likely to be practical for databases that use multiversion
  concurrency control. This requirement is not particularly
  restrictive: multiversion concurrency control is a common isolation
  mechanism implemented by most of today's major database management
  systems, and is popular for web applications because it offers
  higher performance for read-only operations.} to
support them in \autoref{cha:storage-layer}. The requirements are not
onerous; we added this support to the \postgres DBMS with less than 2000
lines of modifications.


Like the typical application-level cache architecture shown in
\autoref{fig:intro:memcache-arch}, \autoref{fig:arch:arch:arch} shows
that our cache is not a part of the database, nor is it an
intermediary between the cache and database. However, unlike in
previous systems, applications do not interact with the cache
explicitly. \txcache introduces a new component: a cache library that
runs on each application server and mediates all accesses to the
cache. The library implements \txcache's cacheable-function
programming model eliminating the need for the application to access
the cache directly and allowing application code to remain largely
oblivious to the cache. It is also responsible for part of the
consistency protocol, ensuring that the application always sees a
transactionally-consistent view of the storage.

The cache is partitioned across a set of cache nodes, each running an
instance of our cache server. These nodes may be run on dedicated
hardware, or they may share resources with other servers. For example,
the cache servers may run on the same machines as the application
servers, in order to take advantage of memory on these servers that
would otherwise go unused.

\section{Assumptions}
\label{sec:arch:assumptions}

We assume that all application servers and all cache nodes know the
membership of the system, \ie the IP addresses of every cache
server. In particular, we assume that they agree on the \emph{same}
view of the system membership. We developed a membership
service~\cite{cowling09:_census} that provides consistent membership
views; other group communication
services~\cite{rodrigues12:_autom_recon_large_scale_reliab_storag_system,
  birman85:_reliab_commun_in_presen_of_failur,
  amir98:_spread_wide_area_group_commun_system} are also options.  We
expect the membership to change infrequently, so the cost of
maintaining these membership views is negligible.  (In smaller
deployments, it may even be practical for an administrator to maintain
the system membership configuration by hand.)

Cache nodes may fail; we assume that they fail by crashing and lose
their state. We discuss how the system responds to failures of cache
nodes in \autoref{sec:programming-model:implementation:cache}. Fault
tolerance of the storage layer is beyond the scope of this work; we do
not discuss how to recover from failures of the storage system beyond
simply restarting the cache after a database failure.  Application
servers may also fail by crashing, but as this has no real impact on
the system design we do not discuss it.

All application servers, cache nodes, and the storage layer are fully
trusted. We also trust the network that connects them to not modify or
corrupt requests. We assume that only
trusted application servers are able to send requests to the cache
nodes and storage system; this could be enforced using firewall rules
or access control policy.

One of our other assumptions is that the application servers and cache
nodes within a deployment are on the same local network, \eg within
the same datacenter. This is not a requirement for correctness of the
protocol, but a low latency interconnect is important for performance.

%\drkp{Are there other assumptions worth mentioning? If not, should we
%  eliminate this section/combine it with the previous one?}

% \section{Programming Model}
% \label{sec:arch:programming-model}

% One of our goals is to provide application developers with an
% interface that is easier to use. Towards that end, our cache uses a
% different programming model than traditional application-level caches
% where the application interacts with the cache directly. We provide a
% brief overview of the programming model here. We give a more detailed
% description and rationale in \autoref{cha:programming-model}.

% Applications group their operations into transactions using an
% interface provided by the \txcache library. Within a transaction,
% \txcache guarantees transactional consistency (isolation) as discussed
% in \autoref{cha:consistency-model}.

% Applications do not need to start and end transactions on the storage
% layer; thee \txcache library does so automatically on their
% behalf. All storage layer operations (\ie database queries) also pass
% through the \txcache library. The library monitors operations in order
% to gather metadata (validity intervals) needed for the consistency
% protocol. However, the \txcache library does not interpret
% the semantics of the operations themselves; for example, it does not
% need to parse queries.

% Our programming model is based on function
% memoization~\cite{michie68:_memo_funct_machin_learn}. Programmers
% designate certain functions in the application code as \emph{cacheable
%   functions}. These functions must depend only on their arguments and
% the database state, and must have no side-effects. When these
% functions are invoked, the \txcache library checks whether the cache
% contains the result of a previous call to the same function with the
% same arguments, and whether that result is still valid. If so, it
% returns the result to the application. Otherwise, it executes the
% function -- which may involve making requests from the storage
% layer. Once complete, the result is added to the cache.

%%% Local Variables: 
%%% mode: latex
%%% TeX-command-default: "Make"
%%% TeX-PDF-mode: t
%%% TeX-master: "main.tex"
%%% End: 

%  LocalWords:  transactional versioning multiversion memoization
%  LocalWords:  transactionally datacenter cacheable metadata
%  LocalWords:  serializable