\label{sec:validation}

\Gizmoball is currently and will continue to be tested rigorously,
comprehensively, and extensively. Because the implementation of the
design is proceeding in parallel, with many different components being
implemented simultaneously, a variety of different testing strategies
are appropriate. Each module is unit tested as it is implemented, and
integration tests are performed as modules are integrated. Automated
regression tests are performed regularly as changes are made. Each of
the top-down, bottom-up, and inside-out testing methodologies are
used, comprising both black-box and glass-box tests.

In particular, we are careful to test all types of inputs, including
those that can cause errors. Correct error behavior, including
exceptions raised and error messages displayed, is verified.

\subsection{Module Testing Strategies}
\label{sec:module-testing}

\paragraph{User Interface}
The user interface is essentially developed top-down, starting with
the top-level \texttt{GizmoballFrame} and working down to the
interface with the game system module and $\R^2$ renderer. Hence,
testing is performed in a top-down manner, using simple stub classes
as necessary. Very little unit testing can be performed on the user
interface, so tests are performed by hand. A test script providing a
sequence of actions to perform and the expected results (essentially a
unit test, but performed by hand) has been
written and used to verify the correct operation of the
user interface and the system as a whole.

A great deal of testing was performed through the user interface. This
verified that the user interface correctly provided access to all
desired behavior. The disadvantage was that it was more difficult to
track down the source of errors. However, extensive debug logging
using \texttt{log4j} and our comprehensive representation invariants
greatly simplified the debugging process.

\paragraph{$\R^2$ Renderer}
The $\R^2$ renderer and drawers were unit tested as well as tested
with the rest of the UI. A separate application,
\texttt{R2DrawerTestApp}, was created; it simply draws several gizmos.
This was used to verify that gizmos could be drawn before performing
integration tests with the UI.

\paragraph{Game System}
The game system was tested using stub gizmos. A simple
\texttt{StubbyGizmo} that simply counted the number of times each of
its methods was called was used to test adding and deleting gizmos and
triggers. A more complex \texttt{ScriptedGizmo} whose responses to
various queries could be scripted\footnote{As in the script to the
  play \emph{Unit Testing: A Half-Second Play In Two Acts, Entirely
    Divorced of Any Physical Meaning}} was used for testing the simulator.
This verified that simulation, including collisions, ticks, and
recalculations were performed according to the specifications. The
individual gizmos were integration tested with the game system
framework.

\paragraph{Property System}
The properties framework was unit tested using a simple
\texttt{PropertifiedClass} class that implemented properties. Its
methods and the observer system were tested thoroughly.

\paragraph{Game Serializer}
The serializer was tested using a number of sample XML files that made
use of all the entities that could be used in a \Gizmoball XML
file. Both valid and invalid XML files were used. In addition, the
supplied example XML file was used to verify that the loaded
configuration matched what was specified.

\paragraph{Usability Testing}
Since our design introduces some non-traditional user interface
elements such as frobbers, multiple user tests were performed in which
different users were prompted to simulate a game, and perform various
editing tasks. 

% Local Variables:
% TeX-master: "finaler.tex"
% End: