%======================================================================%
% Filename: Article.tex
% Project: Insight Journal Article - SharpImage
% Author: Dan Mueller - d.mueller[at]qut.edu.au, dan.muel[at]gmail.com
% Date: $Date: 2007-07-06 12:41:11 +1000 (Fri, 06 Jul 2007) $
% Revision: $Revision: 6 $
%----------------------------------------------------------------------%
% Description:
% This artcile is for the Insight Journal.
% It describes SharpImage, a volume illustration prototyping environment.
%======================================================================%
\documentclass{InsightArticle}

%\usepackage{courier}                 %Font
%\usepackage{charter}                 %Font
%\usepackage{times}                   %Font
%\usepackage{newcent}                 %Font
%\usepackage{pslatex}                 %Font
%\usepackage{lmodern}                 %Font
%\usepackage{palatino}                %Font
%\usepackage{bookman}                 %Font
\usepackage{amsfonts}                 %Maths fonts
\usepackage{amssymb}                  %Maths symbols
\usepackage{listing}                  %Used for list of listings
\usepackage{listings}                 %List program code source
\usepackage{subfigure}                %For subfigures
\usepackage{color}                    %Allow color
\usepackage{fancyhdr}                 %Use headers and footers
\usepackage{graphicx}                 %Graphics
\usepackage[dvipdfmx=true,
            bookmarks=true,
            bookmarksopen=true,
            bookmarksopenlevel=2,
            bookmarksnumbered=true,
            backref=section,
            colorlinks=true,
            linkcolor={blue},
            citecolor={blue},
            urlcolor={blue}]{hyperref}

%Set default font
%\renewcommand{\familydefault}{\rmdefault}  %Roman
\renewcommand{\familydefault}{\sfdefault}   %Sans serif
%\renewcommand{\familydefault}{\ttdefault}  %Typewriter

%Keywords
\newcommand{\keywords}[1]{\noindent\textbf{Keywords:}\it{\ #1}}

% TOC depth
\setcounter{tocdepth}{2}

%Stuff for images
\graphicspath{{images/}}

%Create indent for description
\newenvironment{indentdescription}
               {\list{}{\leftmargin \leftmargin}%
                \item\relax}
               {\endlist}
               
%Allow symbol footnotes
\long\def\symbolfootnote[#1]#2{\begingroup%
\def\thefootnote{\fnsymbol{footnote}}\footnote[#1]{#2}\endgroup}   

%Allow code listings
\input{listcode}                 

%Create conditional to include or exclude figures 
%(saves time during debug compilation).
\makeatletter
\newif\if@includefigs@
\@includefigs@true

%======================================================================%
%                   F r o n t   M a t t e r                            % 
%======================================================================%
% The title should be descriptive enough for people to be able to find
% the relevant document.
\title{SharpImage: An Image Processing\\Prototyping Environment}

% Increment the release number whenever significant changes are made.
% The author and/or editor can define 'significant' however they like.
\release{1.0}

% Name and affiliation
\author{Dan Mueller$^{\scriptscriptstyle\textsf{1}}$}
\authoraddress{$^{\textsf{1}}$Queensland University of Technology, Brisbane, Australia}

% Set custom header style
\pagestyle{fancy}                               
\fancyhead{}            % clear all header fields
\fancyhead[L]{\textsf{SharpImage: An Image Processing Prototyping Environment}}
\fancyhead[R]{\textsf{\thepage}}
\fancyfoot{}            % clear all footer fields

% Document
\begin{document}

% Show the title
\maketitle

% Make the title page header/footer empty
\thispagestyle{empty}

%======================================================================%
%                       A b s t r a c t                                % 
%======================================================================%
\begin{abstract}
\noindent
This paper describes an image processing prototyping environment called SharpImage.
SharpImage is a stand-alone Windows application geared towards the
pre-processing and creation of volume illustrations,
although other generic tasks are inherently supported.
The user interacts with the environment using a command console,
and can view images as a series slices or through volume rendering.
%SharpImage is implemented using C\# and Python (IronPython), using the ManagedITK wrappers.
We describe the design and introduce functionality using a number of examples.
We have found the environment useful for prototyping a range of image processing tasks
and provide the source-code in the hope it will be useful for others.
%
\newline
\newline
\keywords{SharpImage, ITK, prototyping, scripting}

\end{abstract}

%======================================================================%
%                     M a i n   M a t t e r                            % 
%======================================================================%
\tableofcontents

%----------------------------------------------------------------------%
% Section
%----------------------------------------------------------------------%
\section{Introduction}

The Insight Toolkit (ITK) \cite{Yoo2002a} is an open-source software system 
designed primarily for medical image segmentation and registration.
The architecture and design of ITK --- together with the complied C++ implementation ---
provides for efficiency, speed, and flexibility.

However, prototyping image processing pipelines in native ITK can be a daunting task.
As such a number of systems generate wrappers around common functionality:
\emph{WrapITK} \cite{WrapITK} uses CableSwig to automatically generate wrappers for
Python (Tcl and Java are also supported to a lesser degree by this system);
\emph{MATITK} \cite{Chu2005a} allows some ITK algorithms to be called from MATLAB;
and
\emph{ManagedITK} \cite{Mueller2007b} uses a set of templates and CMake scripts
to generate wrappers for .NET languages (such as C\# and IronPython).

A number of data-flow applications have been built using ITK, allowing users
to build pipelines by constructing block diagrams, for example
\emph{SCIRun} \cite{SciRun2007a} and \emph{MeVisLab} \cite{MeVisLab2007a}.
Various stand-alone ``turn-key'' applications also exist,
most of which support some form of extensibility,
such as \emph{VolView} \cite{VolView2007a} and \emph{Slicer} \cite{Slicer2007a}.

SharpImage can be considered a stand-alone scripting application.
It was originally designed to facilitate pre-processing and prototyping
of volume illustration techniques,
however it also supports generic image processing tasks.
The environment can be customised by editing existing scripts,
creating and invoking new scripts at run-time,
or by adding new window elements (forms).
The full source-code is provided in the hope it will be of use
to others in the medical image processing community.

%----------------------------------------------------------------------%
% Section
%----------------------------------------------------------------------%
\section{Design}

SharpImage consists of two major components:
scripts to perform processing, and renderers to preview images.
An application framework is provided to glue these components together.

%----------------------------------------------------------------------%
\subsection{Framework}
The application framework acts a skeleton on which the scripts and renderers are hung.
The framework consists of three major components:
\begin{description}
    \item[Main Window:] The main window parents the renderers, script console, and other windows
    (see \figurename~\ref{fig:Design:SharpImageApplicationFramework}).
    It also has a status label and progress bar which scripts use to report on their status.
    \item[Script Manager:] The script manager is responsible for creating, initialising, running,
    and monitoring scripts.
    It parses a command string (eg. \code{CurvatureFlow TimeStep=0.15 Iterations=8}),
    extracting the name and parameters.
    Given the script name, it searches the scripting folder for possible matches,
    sets the parameters, and starts it running.      
    \item[Dialogs:] A number of dialog windows exist for various tasks including opening and saving images.
    The most important dialog is the open image dialog which allows the user to specify an image path,
    type, and dimensionality (see \figurename~\ref{fig:Design:SharpImageApplicationFramework}).
    The dialog attempts to automatically detect the correct pixel type and dimensions, 
    but these may be overridden by the user to force the ITK IO module to cast the image
    to the desired representation.
\end{description}

\if@includefigs@
\begin{figure}[htb]
      \centering
    \subfigure[Main Window]{
    \includegraphics[height=6.5cm]
    {SharpImage-MainWindow.png}}
    \hspace{2mm}
    \subfigure[Open Image Dialog]{\includegraphics[height=6.5cm]
    {SharpImage-OpenImageDialog.png}}
    \caption{The SharpImage application framework components, including the main window and open image dialog.}
    \label{fig:Design:SharpImageApplicationFramework}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Scripts}
At the heart of SharpImage is a set of IronPython scripts,
most of which operate on images and show their results in new renderers.
Many of the existing scripts apply single ITK filters,
however more complex pipelines can also be constructed.
%
As depicted in \figurename~\ref{fig:Design:SharpImageScriptUML},
all scripts must derive from the base \code{Script} class
and consist of a number of properties and methods.
A majority of scripts inherit from \code{ImageToImageScript} and consist of the following:
\begin{description}
    \item[\code{Name}:] A string representing the name of the script.
    \item[\code{Help}:] A string describing how to use the script.
    \item[\code{Input}:] The input image.
    \item[\code{Output}:] The resultant output image.
    \item[\code{Run()}:] The entry-point used by the manager to invoke the script.
    \item[\code{Initialise()}:] Performs the script initialisation tasks.
    \item[\code{Finalise()}:] Performs the script finalisation tasks.
    \item[\code{DoWork()}:] Creates a new thread and calls \code{ThreadedDoWork()}.
    \item[\code{ThreadedDoWork()}:] Performs the main function of the script on a background thread.
\end{description}

The use of this script-based architecture is advantageous for a number of reasons:
(1) existing scripts can be edited at run-time,
(2) new scripts can be easily added and executed immediately,
(3) functionality can be extended by inheriting from existing scripts,
and
(4) scripts can encapsulate functionality allowing for reproducible experiments given only
the input image and script parameters.

\if@includefigs@
\begin{figure}[htb]
    \centering
    \includegraphics[width=0.95\textwidth]{SharpImage-Script-UML.eps}
    \caption{A UML diagram depicting the inheritance hierarchy of SharpImage scripts.
     In this figure the \code{CurvatureFlowScript} is used as an example.}
    \label{fig:Design:SharpImageScriptUML}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Renderers}
There are currently two renderers available in SharpImage:
a slice renderer for previewing image slices, 
and a volume renderer for direct volume rendering.
Other renderers (such as those exposed by VTK) could be added if desired.

The \textbf{slice renderer} (\code{siGdiSliceRenderer}) uses the Windows GDI+
library to preview a single orthogonal slice though a 3-D image
(or the only slice of a 2-D image).
This renderer supports the following scalar and vector images:
\code{unsigned char}, 
\code{signed short}, 
\code{float}, \code{double}, 
\code{RGB}, \code{RGBA}, 
\code{Vector}, \code{Covariant Vector}, and \code{Vector Image}.
It currently uses 256 greyscales (8-bit) for display 
and supports multiple transparent overlays (labels).

The \textbf{volume renderer} (\code{siVolumeRenderer}) uses OpenGL to display images using
3-D texture mapping with fragment shaders (see \cite{RezkSalama2001a, Kniss2002a}).
The user can rotate the volume using the left mouse button,
translate using the right mouse button,
and scale (zoom in/out) using the mouse wheel.
This renderer supports scaling and biasing for multiple pixel representations 
including \code{unsigned char}, \code{signed short} and \code{float} images.

The \code{siVolumeRenderer} requires a graphics processing unit (GPU) supporting
the following extensions:
\begin{itemize}
	\item\code{GL\_ARB\_fragment\_shader}
	\item\code{GL\_ARB\_shader\_objects}
	\item\code{GL\_ARB\_multitexture}
	\item\code{GL\_EXT\_texture3D}
	\item\code{GL\_EXT\_blend\_minmax}
\end{itemize}
Any good non-OEM graphics card should support these
(such as the ATI Radeon 9800 release in 2003),
but for optimal performance we suggest a more recent card
(such as the NVIDIA GeForce 8800 GTX released in 2007).

\if@includefigs@
\begin{figure}[!htb]
    \centering
    \subfigure[Slice Renderer]
    {\includegraphics[width=0.48\textwidth]{SharpImage-GdiSliceRenderer.png}}
    \hspace{2mm}
    \subfigure[Volume Renderer]
    {\includegraphics[width=0.48\textwidth]{SharpImage-VolumeRenderer-Head.png}}
    \caption{The SharpImage renderers: \code{siGdiSliceRenderer} and \code{siVolumeRenderer}.}
    \label{fig:Design:Renderers}
\end{figure}
\fi

%----------------------------------------------------------------------%
% Section
%----------------------------------------------------------------------%
\newpage
\section{Quick Start Guide}

%----------------------------------------------------------------------%
\subsection{Installation}
We hope to create an installation package at some stage,
but at the moment each component must be installed manually:
\begin{enumerate}
    \item Ensure the .NET Framework 2.0 is installed on your system.
          The .NET Framework Redistributable Package \code{dotnetfx.exe} can be obtained from 
      \href{http://www.microsoft.com/downloads/details.aspx?familyid=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5}{here}.
    \item Install the Microsoft Visual C++ 2005 Redistributable package (\code{vcredist\_x86.exe})
          available from \href{http://www.microsoft.com/downloads/details.aspx?FamilyID=200B2FD9-AE1A-4A14-984D-389C36F85647}{here}.
          The ManagedITK assemblies which are packaged with SharpImage are compiled with Visual Studio 8 SP1,
          so make sure you install the SP1 redistributable package.
    \item Download the SharpImage binary files (which include ManagedITK) from: 
            \href{http://www.insight-journal.org}{http://www.insight-journal.org}. %TODO !!!!!!!!!!!!!!!!
    \item Unzip the binary files to a suitable directory (eg. \code{C:/Utils/SharpImage}).
    \item Run \code{SharpImage.exe}.
\end{enumerate}

The source-code is available for both ManagedITK and SharpImage,
and can be compiled using the Visual Studio 8.0.50727.762 (SP1) solution files.

%----------------------------------------------------------------------%
\subsection{Usage}

This section provides the reader with the necessary knowledge to start using SharpImage.
We assume you have followed the installation process in the previous section and have
a copy of SharpImage open.
As a scripting environment, the user interacts by invoking scripts via the Script Console.
To display the console either navigate the menu to \code{Script > Console}, or press 
\code{Ctrl + Shift + C}. 
In order to effectively use SharpImage, you need only memorise two commands: \code{dir} and \code{help}.

The \code{dir [pattern]} command lists all the available scripts with their name or path
containing the given \code{pattern} string (eg. \code{dir ``Common''}).
If no \code{pattern} is provided, \emph{all} available scripts will be listed.
The following example returns all the scripts related to noise suppression:
%
\listconsolesnip
\begin{lstlisting}
> dir "Denoising"
Scripting\Filtering\Denoising\AdditiveGaussianNoise
Scripting\Filtering\Denoising\Bilateral
Scripting\Filtering\Denoising\CurvatureAnisotropicDiffusion
Scripting\Filtering\Denoising\CurvatureFlow
Scripting\Filtering\Denoising\GradientAnisotropicDiffusion
Scripting\Filtering\Denoising\ImpulseNoise
Scripting\Filtering\Denoising\Mean
Scripting\Filtering\Denoising\Median
Scripting\Filtering\Denoising\MinMaxCurvatureFlow
\end{lstlisting}

Each script has a \code{Help} string which can be displayed by calling \code{help scriptname},
where \code{scriptname} is the name of the script you want to query (eg. \code{help Mean}).
Each help string consists of three components:
the name of the script, a description of the script, and a list of parameters
(with default values indicated in brackets).
If no default parameter value is shown you are required to always provide a value.
For example:
\listconsolesnip
\begin{lstlisting}
> help CurvatureFlow
===== Help: CurvatureFlow =====
Performs denoising of the input image using curvature driven flow.
---- Parameters ----
(double) TimeStep = the finite difference time step. (0.05)
(int) Iterations = the number of iterations. (2)
\end{lstlisting}

Once you have found your desired script (with \code{dir})
and you have read the help information (with \code{help}),
you are ready to invoke the script.
To invoke a script type the name followed by the parameters you wish
to specify (unspecified parameters will assume their default value).
By default the script input is the currently selected image.
For example:
\listconsolesnip
\begin{lstlisting}
> Open "C:/Temp/BrainProtonDensitySlice.png#F2"
> CurvatureFlow TimeStep=0.15 Iterations=8
---- CurvatureFlow: Started at 8:12:04 AM ----
Input=BrainProtonDensitySlice.png
TimeStep=0.15
Iterations=8
Applying CurvatureFlowImageFilter...Done.
Output=BrainProtonDensitySlice_CurvatureFlow.png
==== CurvatureFlow: Finished at 8:12:04 AM: Completed in 00:00:109 ====
\end{lstlisting}

%The above invokes the \code{CurvatureFlow.py} script shown below:
%\begin{center}
%  \listpythonsmall
%   \lstinputlisting[linerange={1-6,16-59}]
%   {../Source/Script/Source/Filtering/Denoising/CurvatureFlow.py}
%\end{center}

%\if@includefigs@
%\begin{figure}[htb]
%     \centering
%    \subfigure[Input]{
%    \includegraphics[height=5.5cm]{SharpImage-BrainProtonDensity-Norm.png}}
%    \hspace{1mm}
%    \subfigure[Output]{
%    \includegraphics[height=5.5cm]{SharpImage-BrainProtonDensity-CVF.png}}
%    \caption{The result from applying \code{CurvatureFlow TimeStep=0.15 Iterations=8}.}
%    \label{fig:GettingStarted:Script}
%\end{figure}
%\fi


%----------------------------------------------------------------------%
% Section
%----------------------------------------------------------------------%
\newpage
\section{Examples}
In this section we will demonstrate some of SharpImage's core functionality.
It loosely follows the organisation of the ITK Software Guide.

%----------------------------------------------------------------------%
\subsection{Open}
An image can be opened by either using a dialog or the script console.
To display the dialog select \code{File \textgreater Open \textgreater Open Image...}
(see \figurename~\ref{fig:Design:SharpImageApplicationFramework}).
The dialog can also be displayed by invoking the \code{Open} script with no parameters.
To open an image using the \code{Open} script,
specify the full path and type separated by a \code{\#}:
\listconsolesnip
\begin{lstlisting}
> Open
> Open "C:/Temp/cthead1.png#UC2" 
> Open "C:/Temp/cthead1.png#SS2" 
> Open "C:/Temp/cthead1.png#F2"
> Open "C:/Temp/VisibleWomanEyeSlice.png#RGBUC2"
\end{lstlisting}

%----------------------------------------------------------------------%
\subsection{Save}
Again, an image can be saved using either a dialog or the script console.
To save the image using the dialog select \code{File \textgreater Save As...}
and specify the path, name, and and pixel type
(the dimensionality and number of components will handled automatically).
To save an image using the \code{Save} script,
specify the full path and type separated by a \code{\#}:
\listconsolesnip
\begin{lstlisting}
> Save "C:/Temp/cthead1.png#UC2" 
> Save "C:/Temp/cthead1.png#SS2" 
> Save "C:/Temp/cthead1.png#F2"
\end{lstlisting}

%----------------------------------------------------------------------%
\subsection{Change Properties}
It is often desirable to list and/or edit the properties of an image and as such
the \code{Properties} script has been provided:
\listconsolesnip
\begin{lstlisting}
> Properties
Name=C:/Temp/cthead1.png
PixelType=Float
Size=[256, 256]
Spacing=[1.00000, 1.00000]
Origin=[0.00000, 0.00000]
Direction=1 0 0 1
Buffer=248840952
MTime=812
\end{lstlisting}

The Name, Spacing, Origin, and Direction properties can also be changed using this script
(use with caution!):
\listconsolesnip
\begin{lstlisting}
> Properties Spacing=itkSpacing(0.5,0.5) Origin=itkPoint(5.0,5.0)
Name=C:/Temp/cthead1.png
PixelType=Float
Size=[256, 256]
*Spacing=[0.50000, 0.50000]
*Origin=[5.00000, 5.00000]
Direction=1 0 0 1
Buffer=248840952
MTime=1414
\end{lstlisting}

The \code{Rename} script is handy for quickly renaming an image
(the names of images are important because they are used as handles for
a number of different scripts):
\listconsolesnip
\begin{lstlisting}
> Rename "Mask1"
\end{lstlisting}

%----------------------------------------------------------------------%
\subsection{Threshold}
Global threshold operations can be very effective for simple image processing tasks.
The \code{BinaryThreshold} script sets all pixels in a given range to the given value,
both specified using the command line.
There are two user friendly specialisations which allow the range to be specified
using scroll bars:
\code{BinaryThresholdWithForm} which displays the result as a binary image,
and
\code{BinaryThresholdAsLabel} which displays the result as a label over the input image
(see \figurename~\ref{fig:Examples:BinaryThreshold}).
\listconsolesnip
\begin{lstlisting}
> BinaryThreshold Lower=0 Upper=180
> BinaryThresholdWithForm
> BinaryThresholdAsLabel
\end{lstlisting}

The \code{Threshold} script leaves unchanged all pixels with intensity values inside the
range defined by the two thresholds.
Pixels with values outside this range are assigned the \code{OutsideValue}.
Again, there is a user friendly specification allowing the range to the specified
using scroll bars:
\listconsolesnip
\begin{lstlisting}
> Threshold Lower=0 Upper=180 OutsideValue=0
> ThresholdWithForm OutsideValue=0
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \subfigure[\code{BinaryThresholdWithForm}]{
    \includegraphics[width=0.4\textwidth]{SharpImage-BinaryThresholdWithForm.png}}
    \hspace{1mm}
    \subfigure[\code{BinaryThresholdAsLabel}]{
    \includegraphics[width=0.4\textwidth]{SharpImage-BinaryThresholdAsLabel.png}}
    \caption{Binary thresholding: \code{BinaryThresholdWithForm} displays the result as a binary image,
                 whereas \code{BinaryThresholdAsLabel} displays the result as a label.}
    \label{fig:Examples:BinaryThreshold}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Cast}
As previously mentioned, SharpImage supports images of different pixel types.
By default, the \code{siGdiSliceRenderer} rescales non \code{unsigned char}
images so that the minimum and maximum values correspond to [0,255].
\figurename~\ref{fig:Examples:Intensity} depicts the same image (consisting
of two circles: left=1, right=2) displayed as \code{unsigned char}
and \code{float}.

It is common to \code{Cast} one type to another and as such a generic script 
(plus a number of short-hand aliases) have been provided to facilitate this task:
\listconsolesnip
\begin{lstlisting}
> Cast OutputPixelType=itkPixelType.F
> CastToF
> CastToSS
> CastToUC
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \includegraphics[height=6.5cm]{SharpImage-Char-vs-Float.png}
    \caption{This figure depicts the same image with \code{unsigned char} pixel type (left) and
                   \code{float} pixel type (right). The \code{float} image is automatically rescaled
                   to consume the entire [0,255] range. The \code{unsigned char} image must be explicitly
                   rescaled using the \code{RescaleIntenityToUC} command.}
    \label{fig:Examples:Intensity}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Intensity Mapping}
The \code{RescaleIntensity} script linearly scales the pixel values in such a way that
the minimum and maximum values of the input are mapped to minimum and maximum values
provided by the user:
\listconsolesnip
\begin{lstlisting}
> RescaleIntensity OutputMinimum=0 OutputMaximum=100
> RescaleIntensity OutputMaximum=100.0 OutputPixelType=itkPixelType.F
\end{lstlisting}

A number of aliases exist for quickly rescaling an image to use an appropriate range
of a given pixel type.
\code{RescaleIntensityToF} rescales the image to [0.0,1.0],
\code{RescaleIntensityToSS} rescales the image to [-32768,32767],
and \code{RescaleIntensityToUC} rescales the image to [0,255]:
\listconsolesnip
\begin{lstlisting}
> RescaleIntensityToF
> RescaleIntensityToSS
> RescaleIntensityToUC
\end{lstlisting}

The \code{ShiftScale} script applies a linear intensity mapping by firstly adding
\code{Shift} to each pixel, then multiplying each pixel by \code{Scale}:
\listconsolesnip
\begin{lstlisting}
> ShiftScale Shift=100.0 Scale=0.5
\end{lstlisting}

The \code{Sigmoid} script applies a non-linear intensity mapping useful for
focusing attention on a particular set of values and progressively attenuating the values
outside that range:
\listconsolesnip
\begin{lstlisting}
> Sigmoid Alpha=10 Beta=170 OutputMinimum=10 OutputMaximum=240
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
    \centering
    \includegraphics[width=0.6\textwidth]{SharpImage-Sigmoid.png}
    \caption{This figure depicts the result of applying the \code{Sigmoid} script.}
    \label{fig:Examples:Sigmoid}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Gradients}
Gradient vectors are useful for a variety of image processing tasks,
including simple edge detection.
SharpImage provides a number scripts for computing the gradient and gradient
magnitude.
Gradient scripts produce multi-component (vector) images, of which a
single channel is shown at one time. To change the channel,
select the renderer and type \code{0}, \code{1}, or \code{2}.
\listconsolesnip
\begin{lstlisting}
> Gradient
> GradientRecursiveGaussian Sigma=1.0
\end{lstlisting}

A number of scripts are also provided to computing the gradient magnitude,
a useful edge measure:
\listconsolesnip
\begin{lstlisting}
> GradientMagnitude
> GradientMagnitudeRecursiveGaussian Sigma=1.0
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
    \centering
    \includegraphics[width=0.60\textwidth]{SharpImage-GradientRecursiveGaussian.png}
    \caption{This figure depicts the results of applying the \code{GradientRecursiveGaussian} and
     \code{GradientMagnitudeRecursiveGaussian} scripts with \code{Sigma=1.0}.}
    \label{fig:Examples:Gradient}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Mathematical Morphology}
Morphological filtering is a powerful technique which analyses images
based on shape using a structuring element.
ITK currently provides two streams of morphological filters:
those for binary and those for greyscale images.
SharpImage brings both of these together into single scripts
using the \code{Operation} parameter:
if \code{Operation="Binary"} then the binary variant is used,
else if \code{Operation="Grayscale"} the greyscale variant is used
(the default is always greyscale).
The structuring element can be specified using the 
\code{itkFlatStructuringElement} static constructors:
\code{Ball}, \code{Box}, \code{Annulus}.
The default type is the \code{Ball} (circle in 2-D and sphere in 3-D).
\listconsolesnip
\begin{lstlisting}
> MorphologicalErode
> MorphologicalDilate
> MorphologicalOpen Operation="Binary"
> MorphologicalClose Operation="Binary"
> MorphologicalErode KernelRadius=itkSize(3,3)
> MorphologicalDilate Kernel=itkFlatStructuringElement.Ball(itkSize(2,2))
> MorphologicalOpen Kernel=itkFlatStructuringElement.Box(itkSize(2,2))
> MorphologicalClose Kernel=itkFlatStructuringElement.Annulus(itkSize(4,4),2)
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \includegraphics[width=0.80\textwidth]{SharpImage-Morphological.png}
    \caption{This figure depicts different morphological operations.}
    \label{fig:Examples:Morphology}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Noise}
SharpImage has a number of scripts for both adding and suppressing noise.
Gavin Baker has implemented additive Gaussian and impulse noise scripts
for ITK (see \href{http://www.cs.mu.oz.au/~gavinb/projects/itk.php}{his webpage})
which are available using the \code{AdditiveGaussianNoise} and \code{ImpulseNoise}
scripts:
\listconsolesnip
\begin{lstlisting}
> AdditiveGaussianNoise Mean=0.0 StdDev=20.0
> ImpulseNoise Probability=0.1
\end{lstlisting}

A number of na\"{i}ve, neighbourhood, and edge-preserving denoising scripts are also provided:
\listconsolesnip
\begin{lstlisting}
> SmoothingRecursiveGaussian Sigma=1.0
> Mean Radius=2
> Median Radius=2
> GradientAnisotropicDiffusion Conductance=1.5 Iterations=4
> CurvatureAnisotropicDiffusion Conductance=1.5 Iterations=4
> Bilateral DomainSigma=[5.0,5.0] RangeSigma=5.0
> CurvatureFlow TimeStep=0.15 Iterations=4
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \includegraphics[width=0.70\textwidth]{SharpImage-Noise.png}
    \caption{This figure depicts results from adding noise to an image.}
    \label{fig:Examples:Noise}
\end{figure}
\fi

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \includegraphics[width=0.85\textwidth]{SharpImage-Denoising.png}
    \caption{This figure depicts results from the \code{Median}, \code{GradientAnisotropicDiffusion},
    and \code{CurvatureFlow} denoising filters.}
    \label{fig:Examples:Denoising}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Pixel Math}
Per-pixel operations perform logical or mathematical operations on each pixel in an image.
ITK implements nearly twenty different pixel math operations which have been drawn together
in SharpImage using two simple scripts: \code{UnaryPixelMath} and \code{BinaryPixelMath}.
As their names suggest, \code{UnaryPixelMath} facilitates operations on single images,
while \code{BinaryPixelMath} operates on two images.
The \code{UnaryPixelMath} script supports the following operations:
\code{Abs}, \code{Exp}, \code{Log}, \code{Not}, \code{Negate}, \code{Sqrt}, or \code{Square}.
The \code{BinaryPixelMath} script supports the following operations:
\code{Add}, \code{Sub}, \code{Mult}, \code{Div}, \code{And}, \code{Or},
\code{Xor}, \code{Max}, \code{Min}, \code{Mask}, or \code{SquaredDiff}.
 
The following example masks an image using a binary image:
\listconsolesnip
\begin{lstlisting}
> Open "C:/Temp/cthead1.png#F2"
> Open "C:/Temp/mask1.png#F2"
> BinaryPixelMath Operation="Mask" Input1="cthead" Input2="mask"
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \subfigure[Input1]{\includegraphics[width=0.25\textwidth]{SharpImage-PixelMath-Input1.png}}
    \hspace{1mm}
    \subfigure[Input2]{\includegraphics[width=0.25\textwidth]{SharpImage-PixelMath-Input2.png}}
    \hspace{1mm}
    \subfigure[Result]{\includegraphics[width=0.25\textwidth]{SharpImage-PixelMath-Result.png}}
    \caption{A demonstration of the \code{BinaryPixelMath} \code{Mask} operation.}
    \label{fig:Examples:PixelMath}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Region growing}
The \code{ConnectedThreshold} script performs region growing from a seed
specified by clicking on the input image:
\listconsolesnip
\begin{lstlisting}
> ConnectedThreshold Lower=400 Upper=2000
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \includegraphics[width=0.75\textwidth]{SharpImage-ConnectedThreshold.png}
    \caption{\code{ConnectedThreshold} region growing from a user specified seed point.}
    \label{fig:Examples:ConnectedThreshold}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Active Contours}
SharpImage provides a \code{FastMarching} script
to compute the positive motion (expansion) of an active contour.
The \code{FastMarchingImageFilter} example given in the ITK Software Guide
\cite[Sec. 9.3.1]{Ibanez2007a} can be reproduced using six simple commands
(plus the seed point specified using the mouse)
(see \figurename~\ref{fig:Examples:FastMarching}):
\listconsolesnip
\begin{lstlisting}
> Open "C:/Temp/BrainProtonDensitySlice.png#F2"
> CurvatureAnisotropicDiffusion TimeStep=0.125 Iterations=5 Conductance=9.0
> GradientMagnitudeRecursiveGaussian Sigma=1.0
> Sigmoid OutputMinimum=0.0 OutputMaximum=1.0 Alpha=-0.3 Beta=2.0
> FastMarching
Added trial point=[56, 92]
> BinaryThreshold Lower=0.0 Upper=100.0
> Select "Brain"
> AddLabel "Threshold"
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \subfigure[Input]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Input.png}}
    \hspace{1mm}
    \subfigure[Smooth]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Smooth.png}}
    \hspace{1mm}
    \subfigure[Gradient]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Gradient.png}}
    \hspace{1mm}
    \subfigure[Sigmoid]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Sigmoid.png}}
    \hspace{1mm}
    \subfigure[Arrival]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Arrival.png}}
    \hspace{1mm}
    \subfigure[Threshold]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Threshold.png}}
    \hspace{1mm}
    \subfigure[Label]
    {\includegraphics[width=0.1875\textwidth]{SharpImage-FastMarching-Label.png}}
    \caption{The \code{FastMarchingImageFilter} example from the ITK Software Guide
                    \cite[Sec. 9.3.1]{Ibanez2007a} can be reproduced using six simple commands.}
    \label{fig:Examples:FastMarching}
\end{figure}
\fi

An interactive script for computing an edge-based speed image for 2-D or 3-D images
is also provided in \code{LevelSetSpeed}
(see \figurename~\ref{fig:Examples:LevelSetSpeed}):
\listconsolesnip
\begin{lstlisting}
> Open "C:/Temp/BrainProtonDensitySlice.png#F2"
> LevelSetSpeed OutputMinimum=-1.0 OutputMaximum=1.0
\end{lstlisting}

\if@includefigs@
\begin{figure}[!htb]
      \centering
    \includegraphics[width=0.50\textwidth]{SharpImage-LevelSetSpeed.png}
    \caption{The \code{LevelSetSpeed} script in action.}
    \label{fig:Examples:LevelSetSpeed}
\end{figure}
\fi

%----------------------------------------------------------------------%
\subsection{Volume Rendering}
The \code{siVolumeRenderer} implements 3-D texture mapping
using fragment shaders written in OpenGL Shading Language (GLSL).
It expects at least one input image (a `value' image),
but also supports gradient / gradient magnitude images.
%
To invoke the renderer using a given value and gradient magnitude image 
use the \code{VolumeRender} script in the following manner:
\listconsolesnip
\begin{lstlisting}
> Open "C:/Temp/engine.mhd#UC3"
> GradientMagnitude
> VolumeRender Value="engine" Gradient="Magnitude"
\end{lstlisting}
Once the script has finished, a new \code{siVolumeRenderer} is displayed
along with a number of forms for specifying the renderer properties and transfer function.
A histogram can be generated for the transfer function editor background using the 
\code{ValueEdgeHistogram} script:
\listconsolesnip
\begin{lstlisting}
> Open "C:/Temp/engine.mhd#F3"
> GradientMagnitude
> ValueEdgeHistogram Value="engine" Edge="Magnitude"
\end{lstlisting}

To change the fragment shader select the ``Renderer Editor'' tab,
select the ``FragmentProgramPath'' property, click the ``...'' button
and navigate the desired fragment shader
(a vertex shader of the same name but different extension must also exist in the same directory).
A number of existing fragment shaders are provided, but custom shaders can be easily added for
prototyping illustration techniques.

The simplest shader is \code{shader-v-copy.frag} which copies the image value to the
fragment and is useful for maximum intensity projections
(see \figurename~\ref{fig:Examples:VolumeRendering:VisibleHuman:MIP}):
\begin{center}
  \listglslsnip
    \lstinputlisting[linerange={25-27,30-32}]
    {../Source/Rendering/VolumeRendering/shader-v-copy.frag}
\end{center}

\newlength{\dvrvhdimagewidth}
\setlength{\dvrvhdimagewidth}{0.3\textwidth}

\if@includefigs@
\begin{figure}[tb]
      \centering
    \subfigure[Oblique Section]{\label{fig:Examples:VolumeRendering:VisibleHuman:Oblique}
    \includegraphics[width=\dvrvhdimagewidth]{VisibleHuman/VisibleHuman-Slice-3-0.png}}
    \hspace{1mm}
    \subfigure[Minimum Intensity Projection]{\label{fig:Examples:VolumeRendering:VisibleHuman:MIP}
    \includegraphics[width=\dvrvhdimagewidth]{VisibleHuman/VisibleHuman-MIP-3-0.png}}
    \hspace{1mm}
    \subfigure[Value]{\label{fig:Examples:VolumeRendering:VisibleHuman:Value}
    \includegraphics[width=\dvrvhdimagewidth]{VisibleHuman/VisibleHuman-V-1-0.png}}
    \hspace{1mm}
    \subfigure[Value-Magnitude]{\label{fig:Examples:VolumeRendering:VisibleHuman:ValueMagnitude}
    \includegraphics[width=\dvrvhdimagewidth]{VisibleHuman/VisibleHuman-VM-3-2.png}}
    \hspace{1mm}
    \subfigure[Phong]{\label{fig:Examples:VolumeRendering:VisibleHuman:Phong}
    \includegraphics[width=\dvrvhdimagewidth]{VisibleHuman/VisibleHuman-VM-Phong-3-0.png}}
    \hspace{1mm}
    \subfigure[Silhouette]{\label{fig:Examples:VolumeRendering:VisibleHuman:Silhouette}
    \includegraphics[width=\dvrvhdimagewidth]{VisibleHuman/VisibleHuman-Sil-Phong-3-0-9-5-0-5-0-5.png}}
    %
    \caption{Results from different fragment shaders and renderer settings.}
    \label{fig:Examples:VolumeRendering:VisibleHuman}
\end{figure}
\fi

Typically a `transfer function' is employed to transfer opacity and colour information
to each potential pixel (`fragment').
A simple transfer function is realised in \code{shader-v.frag}
by indexing a 1-D lookup table (LUT) using only the image value
(see \figurename~\ref{fig:Examples:VolumeRendering:VisibleHuman:Value}):
\begin{center}
  \listglslsnip
    \lstinputlisting[linerange={25-31,34-36}]
    {../Source/Rendering/VolumeRendering/shader-v.frag}
\end{center}
A more complex transfer function is realised in \code{shader-vm-noadjust.frag}
by indexing a 2-D LUT using image value and gradient magnitude \cite{Kniss2002a}
(see \figurename~\ref{fig:Examples:VolumeRendering:VisibleHuman:ValueMagnitude}):
\begin{center}
  \listglslsnip
    \lstinputlisting[linerange={26-33,36-38}]
    {../Source/Rendering/VolumeRendering/shader-vm-noadjust.frag}
\end{center}
Unfortunately, as shown in \figurename~\ref{fig:Examples:VolumeRendering:VisibleHuman:NoAdjust},
this approach does not cater for changes in the sampling rate
(the ratio of the distance between the interpolating proxy geometry).
Weiler et al. \cite{Weiler2000a} showed that the transparency of the fragment
($\alpha_{0}$)
must be adjusted according the sampling rate ($r_s$) as follows:
\begin{eqnarray}\label{eq:Shader:AdjustSamplingRate}
  \alpha_{k} = 1 - [ 1 - \alpha_{0}]^{\frac{1}{r_s}}
\end{eqnarray}
This adjustment is implemented in \code{shader-vm.frag}:
\begin{center}
  \listglslsnip
    \lstinputlisting[linerange={25-29}]
    {../Source/Rendering/VolumeRendering/shader-vm.frag}
\end{center}

The previous fragment shader implements what is sometimes called
`gradient magnitude opacity modulation'.
However, true direct volume rendering typically implies the use of an illumination model,
such as the Phong reflection model.
The Phong model is a simplified \emph{local} model consisting of three reflection terms
(ambient, diffuse, and specular):
\begin{eqnarray}\label{eq:Shader:PhongModel}
  I = k_a i_a + \sum_{\textrm{lights}}{ \Big[ k_d i_d (N \cdot L) + k_s i_s(R \cdot V)^{n} \Big] }
\end{eqnarray}
where $I$ is the resultant light intensity, $k_x$ are constants,
$i_x$ are the light intensities of the respective components,
$N$ is the normal vector, $L$ the vector to the light source,
$R$ the reflection vector, $V$ the viewing vector,
and $n$ the specular exponent.
For our implementation in \code{shader-vm-phong.frag}%
\footnote{This shader is written for readability rather than speed or size.
If you have an older graphics card we recommend using \code{shader-vm-phong-simple.frag}.} %
(see \figurename~\ref{fig:Examples:VolumeRendering:VisibleHuman:Phong})
we compute the normal using an online central difference method,
assume unity constants,
and use a single light positioned along the viewing vector (ie. $V = L$):
\begin{center}
  \listglslsnip
    \lstinputlisting[linerange={32-67,68-71,85-92}]
    {../Source/Rendering/VolumeRendering/shader-vm-phong.frag}
\end{center}

\newlength{\dvrvhdadjimagewidth}
\setlength{\dvrvhdadjimagewidth}{0.23\textwidth}

\if@includefigs@
\begin{figure}[tb]
      \centering
    \subfigure[$r_s=0.1$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-NoAdjust-0-1.png}}
    \hspace{1mm}
    \subfigure[$r_s=0.4$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-NoAdjust-0-4.png}}
    \hspace{1mm}
    \subfigure[$r_s=1.6$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-NoAdjust-1-6.png}}
    \hspace{1mm}
    \subfigure[$r_s=6.4$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-NoAdjust-6-4.png}}
    \\
    %
    \subfigure[$r_s=0.1$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-0-1.png}}
    \hspace{1mm}
    \subfigure[$r_s=0.4$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-0-4.png}}
    \hspace{1mm}
    \subfigure[$r_s=1.6$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-1-6.png}}
    \hspace{1mm}
    \subfigure[$r_s=6.4$]{
    \includegraphics[width=\dvrvhdadjimagewidth]{VisibleHuman/VisibleHuman-VM-6-4.png}}
    %
    \caption{A comparison of value-magnitude volume renderings generated with
                     different sampling rates.
                   First row: no alpha adjustment.
                   Second row: with alpha adjustment.}
    \label{fig:Examples:VolumeRendering:VisibleHuman:NoAdjust}
\end{figure}
\fi

Rheingans and Ebert \cite{Rheingans2001a} introduced a number of volume
illustration techniques for enhancing boundaries, silhouettes, and regions.
Some of these are realised in \code{shader-vm-enhance.frag}
(see \figurename~\ref{fig:Examples:VolumeRendering:VisibleHuman:Silhouette}):
\begin{center}
  \listglslsnip
    \lstinputlisting[linerange={50-71}]
    {../Source/Rendering/VolumeRendering/shader-vm-enhance.frag}
\end{center}

%----------------------------------------------------------------------%
\subsection{Extending SharpImage}
Although SharpImage has a decent collection of image
processing scripts (thanks to ITK),
its real strength lies in the ability to be easily extended.
%
To add a new form for use with SharpImage follow these steps:
\begin{enumerate}
	\item Open Visual Studio and select \textbf{File~$>$~New Project}.
	\item Select \textbf{Windows Application}, enter the name and location,
	and click OK.
	\item Right click on the project References, and select \textbf{Add Reference...}.
	Select the Browse tab, navigate to the SharpImage folder, select \code{SharpImage.exe},
	and click OK.
	\item In the Solution Explorer, right click	on the \textbf{project name} and select \textbf{Properties}.
	Change the Output type from Windows Application to \textbf{Class Library}.
	Save the project and close the Properties window.
	\item In the Solution Explorer, right click on \code{Program.cs} and select \textbf{Delete}.
	\item In the Solution Explorer, right click Form1.cs and select \textbf{Rename}.
	Enter the desired name, such as siFormCustom1.cs.
	\item  In the Solution Explorer, right click on the form and select \textbf{View Code}.
	Change the name of the form in the class declaration and make it extend
	\code{SharpImage.Forms.siFormTool}.
	Also change the default constructor name.
	\item In the Solution Explorer, right click on the form and select \textbf{View Designer}.
	Edit the form as desired. For this example we changed the form text in the Properties explorer
	and added a check box.
	\item In the Solution Explorer, right click on the form, select View Code,
	and \textbf{add code} as desired.
	For this example we added a property named \code{RunFilter} which returns the checkbox Checked property.
	\item Select the build type and \textbf{compile} the library.
	\item \textbf{Create a Scripting/Custom folder} in the SharpImage installation directory.
	Copy the class library to this folder.
	\item \textbf{Create a script} in this folder which uses the form (see below).
	\item \textbf{Invoke the script} from SharpImage.
\end{enumerate}

\begin{center}
  \listpythonsmall
	\lstinputlisting[linerange={1-1,4-4,17-44,46-48,54-60,74-84,94-96,101-105}]{extend/Custom1.py}
\end{center}

\if@includefigs@
\begin{figure}[!htb]
	  \centering
    \subfigure[Create a new project]{
    \includegraphics[width=0.475\textwidth]{extend/SharpImage-Extend-NewProject.png}}
    \subfigure[Add reference]{
    \includegraphics[width=0.475\textwidth]{extend/SharpImage-Extend-AddReference.png}}
    \subfigure[Edit project properties]{
    \includegraphics[width=0.475\textwidth]{extend/SharpImage-Extend-ProjectProperties.png}}
    \subfigure[Design the form]{
    \includegraphics[width=0.475\textwidth]{extend/SharpImage-Extend-Designer.png}}
    \subfigure[Invoke the script]{
    \includegraphics[width=0.60\textwidth]{extend/SharpImage-Extend-Form.png}}
    %
    \caption{Steps for extending SharpImage.}
    \label{fig:Examples:Extend:Steps}
\end{figure}
\fi

%----------------------------------------------------------------------%
% Conclusion
%----------------------------------------------------------------------%
\section{Conclusion}
This paper described an image processing prototyping environment based on ITK.
It currently supports two renderers (a slice renderer and a volume renderer)
and allows various scripts to be called from a command console.
New scripts and forms can be added to the environment at run-time very easily.
We have found the environment useful for a range of image processing
tasks including prototyping volume illustrations.
For suggestions or bugs, feel free to contact 
us\footnote{Corresponding author: Dan Mueller: %
\href{mailto:d.mueller@qut.edu.au}{d.mueller@qut.edu.au} or
\href{mailto:dan.muel@gmail.com}{dan.muel@gmail.com}.
}.

%======================================================================%
%                    B i b l i o g r a p h y                           % 
%======================================================================%
\newpage
\bibliographystyle{plain}
\bibliography{article}

\end{document}