% This file was converted to LaTeX by Writer2LaTeX ver. 0.4 % see http://www.hj-gym.dk/~hj/writer2latex for more info \documentclass[11pt,twoside]{article} \usepackage{graphicx} \usepackage[ascii]{inputenc} \usepackage[T1]{fontenc} \usepackage{amsmath,amssymb,amsfonts,textcomp} \usepackage{color} \usepackage{alltt} \usepackage{calc} \usepackage{hyperref} \hypersetup{colorlinks=true, linkcolor=blue, filecolor=blue, pagecolor=blue, urlcolor=blue} %% Define a new 'url' style for the url package that will use a smaller font. \makeatletter \def\url@urlstyle{% \@ifundefined{selectfont}{\def\UrlFont{\sf}}{\def\UrlFont{\small\ttfamily}}} \makeatother %% Now actually use the newly defined style. \urlstyle{urlstyle} % Pages styles (master pages) \setlength\parindent{0in} \setlength{\parskip}{1ex plus 0.5ex minus 0.2ex} \setlength\textwidth{6.5in} \makeatletter \newcommand\ps@Standard{% \renewcommand\@oddhead{}% \renewcommand\@evenhead{\@oddhead}% \renewcommand\@oddfoot{}% \renewcommand\@evenfoot{\@oddfoot}% \setlength\paperwidth{8.5in}\setlength\paperheight{11in}\setlength\voffset{-1in}\setlength\hoffset{-1in}\setlength\topmargin{0.5in}\setlength\headheight{12pt}\setlength\headsep{0.461in}\setlength\footskip{12pt+0.461in}\setlength\textheight{11in-0.5in-0.5in-0.461in-12pt-0.461in-12pt}\setlength\oddsidemargin{1in}\setlength\textwidth{8.5in-1in-1in} \renewcommand\thepage{\arabic{page}} \setlength{\skip\footins}{0.0398in}\renewcommand\footnoterule{\vspace*{-0.0071in}\noindent{\rule{0.25\columnwidth}{0.0071in}}\vspace*{0.0398in}} } \makeatother \pagestyle{Standard} % footnotes configuration \makeatletter \renewcommand\thefootnote{\arabic{footnote}} \makeatother \newcommand{\cmd}[1] {\mbox{\sf #1}\null} \newcommand{\code}[1] {\mbox{\bf\tt #1}\null} \newcommand{\file}[1] {\mbox{\bf\em #1}\null} \newcommand{\param}[1] {\mbox{\{\em #1\}}\null} \newenvironment{source} {\small\begin{quote}\begin{alltt}} {\end{alltt}\end{quote}\normalsize} \title{Sierra IO System} \begin{document} \clearpage\pagestyle{Standard} \title{Sierra IO System} \author{Gregory D. Sjaardema\\ Advanced Computational Mechanics Architectures\\ Sandia National Laboratories} \maketitle \setcounter{tocdepth}{3} \setlength{\parskip}{0ex plus 0.5ex minus 0.2ex} \tableofcontents \section{Introduction} The documentation below is a medium{}- to low{}-level view of the Sierra IO system targeted at developers who will be adding or modifying the database IO portion of the system. It should give enough detail that a new database type could be added by reading this document and looking at an existing database class. It is also helpful to have the doxygen{}-generated documentation for the Ioss class hierarchy available. The IO Subsystem has been designed to support multiple database formats simultaneously. It is possible to have the finite element model read from an ExodusII database; two results files being written to an ExodusII file with a third results file being written to a CGNS file; and the restart file being written to yet another ExodusII file. Each of these output databases can have a different schedule for when to write and what data is to be written. If there are any questions or corrections, please contact Greg Sjaardema at (505) 844{}-2701 or \href{mailto:gdsjaar@sandia.gov}{\nolinkurl{gdsjaar@sandia.gov}}. \setlength{\parskip}{1ex plus 0.5ex minus 0.2ex} \section{Class Structure Overview} The three main modules or namespaces involved in the IO are: \begin{description} \item [\code{Fmwk}] Main sierra framework namespace. This defines and manages the Sierra applications data structures, algorithms, mechanics, etc. Each \code{Fmwk::Region} owns a \code{Frio::IOBroker} which is its link to all bulk data io. \item [\code{Frio}] The bridge between the Sierra frameworks data structures and the IO databases data structures in the Ioss namespace. Ideally all data transfers between Sierra and the IO databases is under the control of the \code{Frio} classes. However, there are some special cases where an application will directly interact with the Ioss classes. The \code{Frio::IOBroker} manages the Restart, Mesh, Results, and other databases used by a \code{Fmwk::Region.} \item [\code{Ioss}] The \code{Ioss} namespace contains several classes which attempt to provide an abstract interface to multiple concrete database types. It also defines a lightweight generic representation of the finite element model. \end{description} \subsection{The Ioss Module} There are two major pieces of the \code{Ioss} module: The \code{Ioss::DatabaseIO} which is a pure virtual base class from which all concrete IO databases are derived and the \code{Ioss::GroupingEntity} classes which define a lightweight generic model representation. The current inheritance structure of \code{Ioss::DatabaseIO} is shown in Figure~\ref{fig:dbio_inherit}. \begin{figure}[htp] \centering \includegraphics[scale=0.75]{DatabaseIO__inherit__graph.png} \caption{\code{Ioss::DatabaseIO} Inheritance Diagram}\label{fig:dbio_inherit} \end{figure} Currently the concrete database types are ExodusII (\code{Ioex}), CGNS (\code{Iocgns}), and Heartbeat (\code{Iohb}). The \code{Ioss::DatabaseIO} class has a pointer to an \code{Ioss::Region }which is the root of the generic model representation. \subsubsection{Ioss::Region and Ioss::GroupingEntity Classes} An \code{Ioss::Region} is an \code{Ioss:GroupingEntity} and it contains a vector of \code{NodeBlocks}, \code{ElementBlocks}, \code{SideSets}, and \code{CommSets} defining the current model. These are all also \code{Ioss::GroupingEntities}. The \code{Ioss::GroupingEntity} inheritance structure is shown in Figure~\ref{fig:ge_inherit}. \begin{figure}[htp] \centering \includegraphics[scale=0.75]{GroupingEntity__inherit__graph.png} \caption{\code{Ioss::GroupingEntity} Inheritance Diagram}\label{fig:ge_inherit} \end{figure} An \code{Ioss::GroupingEntity} contains an \code{Ioss::PropertyManager} which maintains 1 or more \code{Ioss::Property;} and an \code{Ioss::FieldManager} which maintains 1 or more\code{ Ioss::Field}. It also contains a name and a pointer to the \code{Ioss::DatabaseIO}. There is also an \code{Ioss::EntityBlock} which contains 1 extra member which is the \code{Ioel::ElementTopology} which is the topology of the entities contained in this block. The collaboration diagram for the \code{Ioss::GroupingEntity} is shown in Figure~\ref{fig:ge_collab} \begin{figure}[htp] \centering \includegraphics[scale=0.75]{GroupingEntity__coll__graph.png} \caption{\code{Ioss::GroupingEntity} Collaboration Diagram}\label{fig:ge_collab} \end{figure} A \code{GroupingEntity} is a very lightweight class; it does not contain any bulk data. It represents a portion of the finite element model and can transfer bulk data to/from the database from/to the Sierra datastructures. It also contains metadata about that portion of the finite element model. Each specific \code{GroupingEntity} type has a few required properties and fields and some optional properties and fields. The \code{Ioss::GroupingEntity}'s define the basic finite element model. For example, a model with two element blocks, a nodeset, and two sidesets is shown in Figure~\ref{fig:block}. \begin{figure}[htp] \centering \includegraphics[scale=0.5]{Block_Diagram_Model.png} \caption{Block Diagram of Finite Element Model Structure}\label{fig:block} \end{figure} Note that the \code{SideSets} are further divided into one or more \code{SideBlocks} which contain entities of homogeneous topology. Each of these \code{GroupingEntity's} contain both \code{Properties} and \code{Field} definitions which can be queried by the \code{Frio} classes to define the model. \subsubsection{Ioss::Property and Ioss::PropertyManager} Each \code{Ioss::GroupingEntity} class contains an \code{Ioss::PropertyManager} which manages the \code{Ioss::Property} on this particular entity. An \code{Ioss::Property} is basically a named integer, real, string, or pointer value. The \code{PropertyManager} maintains a list of all \code{Properties} for a specific entity and provides an interface where the \code{Properties} can be located by name. \subsubsection{Ioss::Field and Ioss::FieldManager} Each \code{Ioss::GroupingEntity} class contains an \code{Ioss::FieldManager} which manages the \code{Ioss::Field}s on this particular entity. An \code{Ioss::Field} contains the metadata for the field it describes; it does not hold the bulk data, so an \code{Ioss::Field} is fairly lightweight. Each \code{Ioss::Field} contains: \begin{itemize} \item { std::string \code{name} -- The name of the field.} \item { Int \code{rawCount\_} -- The size of the untransformed field } \item { Int \code{transCount\_} -- The size of the transformed field.} \item { BasicType \code{type\_} -- The basic type of the field (Integer, Real, String)} \item { const VariableType * \code{rawStorage\_} -- The storage type of the untransformed field (Vector, Scalar, Tensor, ....)} \item { const VariableType * \code{transStorage\_} -- The storage type of the transformed field (Vector, Scalar, Tensor, ...)} \item { RoleType \code{role\_} -- The ``role'' of the field. Valid roles are: \code{INTERNAL}, \code{MESH}, \code{ATTRIBUTE}, \code{COMMUNICATION}, \code{INFORMATION}, \code{REDUCTION}, and \code{TRANSIENT}. The most common are \code{MESH} for describing the model geometry/topology, \code{ATTRIBUTE} for block entity attributes (typically \code{ElementBlock} at this time), \code{TRANSIENT} for time{}-dependent results data, and \code{REDUCTION} for ``summary fields'' (see below).} \item { Int \code{size\_} -- The number of bytes required to store the entire \code{Field}. Equal to \code{rawCount\_ * sizeof(type\_) * (rawStorage\_ components)}. } \item { std::vector{\textless} Iotr::Transform * {\textgreater} \code{ransforms\_} -- A list of transforms which are applied to this field. \textit{Note that the transforms are implemented at the Field level, but are not yet functional in Sierra applications.}} \end{itemize} The \code{Ioss::Fields} are used for all bulk data input and output. Each \code{Ioss::GroupingEntity }contains a \code{get\_field\_data} and a \code{put\_field\_data} function. The function signatures are: \begin{source} int get\_field\_data(const std::string \&field\_name, void *data, size\_t data\_size=0) int put\_field\_data(const std::string \&field\_name, void *data, size\_t data\_size=0) \end{source} The functions will get or put the data corresponding to the field named \code{field\_name}; the data will be read from/written to the memory pointed to by \code{data} which is of size \code{data\_size}. If \code{data\_size} is nonzero, then the field checks that there is sufficient space to store the field data; if it is zero, then it is assumed to be large enough. The function returns the number of entities for which the field was read/written. The \code{get\_field\_data} and \code{put\_field\_data} functions are actually at the\code{ Ioss::GroupingEntity} level and they forward the call down to \code{internal\_get\_field\_data} and \code{internal\_put\_field\_data} which are defined on each class derived from \code{Ioss::GroupingEntity.} These in turn call\footnote{The \code{offset} and \code{count} were originally intended to provide partial field input and output, but this became too unwieldy and is not used at this time. Only full field input or output is supported.}: \begin{source} int get\_database(){}-{\textgreater}get\_field(this, field, offset, count, data, data\_size); \end{source} get the concrete database class which is responsible for transferring the bulk data to/from the database to/from the memory pointed to by \code{data} of size \code{data\_size} bytes. \paragraph{Ioss::Field -- Roles} As defined above, there are six values for a fields \code{role}. The three most commonly used are: \begin{description} \item [\code{MESH}] A field which is used to define the basic geometry or topology of the model and is not normally transient in nature. Examples would be element connectivity or nodal coordinates. \item [\code{ATTRIBUTE}] A field which is used to define an attribute on an \code{EntityBlock} derived class. Examples would be thickness of the elements in a shell element block or the radius of particles in a particle element block. \item [\code{TRANSIENT}] A field which is typically calculated at multiple steps or times in an analysis. These are typically ``results'' data. Examples would be nodal displacement or element stress. \item [\code{REDUCTION}] A field which is typically summarizes some transient data about an entity. The size of this field is typically not proportionate to the number of entities in a \code{GroupingEntity}. An example would be average displacement over a group of nodes or the kinetic energy of a model. This data is also transient. \end{description} \paragraph{Ioss::Field -- Storage} The field storage members define the higher{}-order type of the variable such as vector, tensor, etc. The storage class is represented by the \code{Ioss::VariableType} base class which has several derived classes. The four types of derived class are: \begin{itemize} \item { \code{ConstructredVariableType} -- Specify a name and the number of components. This is typically used for user{}-defined storage types such as state variables. A specific variable may have 86 components, so the user would create composite variable type specifying a name and 86 components. By default, the IO system will create the type as \code{Real[component\_count]}, so the above type would be known as ``\code{Real[86]}''.} \item { \code{CompositeVariableType }{--} A combination (or ``composite'') of the other field storage classes. For example, a field defined on the four integration points of a quadrilateral element would be a composite of 4 tensors. Currently the composite can only contain two subfields.} \item { \code{Ioel::ElementVariableType} -- Each element known to the IO system will register a type which is sufficient for storing its nodal connectivity. For example, a 20{}-node hex registers the type \code{Ioel::St\_Hex20} which has the name ``hex20'' and consists of 20 components.} \item { Predefined ``common'' types. This category encompasses many predefined types such as:} \begin{tabular}{rl} Scalar: & \\ Vector: & 2D, 3D \\ Quaternion: & 2D, 3D \\ Full\_Tensor:& 36, 32, 22, 16, 12 \\ Sym\_Tensor: & 33, 31, 21, 13, 11, 10 \\ Asym\_Tensor:& 03, 02, 01 \\ \end{tabular} \end{itemize} Each variable type will return its component count. It also will return an optional suffix for each component. For example, the \code{Sym\_Tensor\_33} will return the suffices XX, YY, ZZ, XY, YZ, ZX. \subsubsection{Defined Properties and Fields} All classes derived from \code{GroupingEntity} provide the property: \begin{quote} \begin{tabular}{ll} \code{name} &(String) \\ \end{tabular} \end{quote} In addition, the \code{NodeSet} class provides the properties: \begin{quote} \begin{tabular}{ll} \code{entity\_count} &(Integer) \\ \code{distribution\_factor\_count} &(Integer) \\ \end{tabular} \end{quote} And the fields: \begin{quote} \begin{tabular}{lll} \code{ids} & (Integer) & scalar \\ \code{distribution\_factors} & (Real) & scalar \\ \end{tabular} \end{quote} The \code{SideSet} class provides the properties: \begin{quote} \begin{tabular}{ll} \code{block\_count} &(Integer) \\ \code{side\_block\_count} &(Integer) \\ \end{tabular} \end{quote} And no additional fields. All classes derived from \code{EntityBlock} provide the properties: \begin{quote} \begin{tabular}{ll} \code{name} &(String) \\ \code{entity\_count} &(Integer) \\ \code{topology\_node\_count} &(Integer) \\ \code{topology\_type} &(String) \\ \code{parent\_topology\_type}&(String) \\ \end{tabular} \end{quote} And the fields: \begin{quote} \begin{tabular}{lll} \code{ids} & (Integer) & scalar \\ \code{connectivity} & (Integer) & topology\_node\_count \\ \end{tabular} \end{quote} The \code{NodeBlock} class provides the additional properties: \begin{quote} \begin{tabular}{ll} \code{component\_degree} &(Integer) \\ \end{tabular} \end{quote} And the additional fields: \begin{quote} \begin{tabular}{lll} \code{mesh\_model\_coordinates} & (Real) & component\_degree \\ \end{tabular} \end{quote} The \code{ElementBlock} class provides the additional properties: \begin{quote} \begin{tabular}{ll} \code{attribute\_count} &(Integer) \\ \end{tabular} \end{quote} If \code{attribute\_count} is greater than zero, then there will be additional fields defined for the attributes. These fields will all have the role \code{Ioss::Field::ATTRIBUTE}. In addition to the individual attribute fields, there will be a single Real field named ``attribute'' which contains a field for each element in the element block; the field will have \code{attribute\_count} components. The \code{SideBlock} class provides the additional properties: \begin{quote} \begin{tabular}{ll} \code{distribution\_factor\_count} &(Integer) \\ \end{tabular} \end{quote} And the field: \begin{quote} \begin{tabular}{lll} \code{element\_side} &(Integer) & 2 (first component is element global id, second is local element ordinal; 1-based \\ \end{tabular} \end{quote} An \code{Ioss::Region} is an \code{Ioss::GroupingEntity}. In addition to the \code{name} property, it also provides properties that describe the overall structure of the model: \begin{quote} \begin{tabular}{ll} \code{entity\_count} &(Integer) \\ \code{node\_block\_count} &(Integer) \\ \code{node\_block\_count} &(Integer) \\ \code{element\_block\_count} &(Integer) \\ \code{side\_set\_count} &(Integer) \\ \code{node\_set\_count} &(Integer) \\ \code{comm\_set\_count} &(Integer) \\ \code{node\_count} &(Integer) \\ \code{element\_count} &(Integer) \\ \code{state\_count} &(Integer) \\ \code{current\_state} &(Integer) \\ \code{database\_name} &(String) \\ \end{tabular} \end{quote} \subsection{Ioss::DatabaseIO Class Hierarchy} The\code{ Ioss::DatabaseIO} class is a virtual base class which defines the interface required of each concrete database format. In addition, it also provides some functionality which is common to all (or many) database types. \subsubsection{Attributes} The \code{Ioss::DatabaseIO }class has the following attributes: \begin{description} \item [\code{std::string DBFilename}] --The name of the file this database is reading or writing. \item [\code{Ioss::State dbState}] --An input database will always be in \code{STATE\_READONLY} which signifies that it cannot be written to or modified. An output database follows a set order of access. The states corresponding to this access are: \begin{center} \begin{tabular}{rp{4in}} \code{STATE\_INVALID}&Error state if something goes wrong.\\ \code{STATE\_UNKNOWN}&Typically used at the very beginning of the databases existence when the class has been created, but no reading or writing has occurred.\\ \code{STATE\_READONLY}&An input database is only in either \code{STATE\_UNKNOWN} or in \code{STATE\_READONLY} which means that it cannot be written to or changed.\\ \code{STATE\_CLOSED}&The states are not nested, so each state must end with a transition to the \code{STATE\_CLOSED} prior to entering the next state.\\ \code{STATE\_DEFINE\_MODEL}&Defining the metadata which defines the model (non{}-transient, geometry and topology).\\ \code{STATE\_MODEL}&Outputting the bulk data (mesh\_model\_coordinates, ids, connectivity) relating to the model portion.\\ \code{STATE\_DEFINE\_TRANSIENT}&Defining the metadata relating to the transient data. For example, the element or nodal fields.\\ \code{STATE\_TRANSIENT}&Outputting the transient bulk data.\\ \end{tabular} \end{center} \item [\code{bool isParallel}]-- true if running in parallel \item [\code{int myProcessor}]-- the processor this instance is running on. \item [\code{TopoContainer faceTopology}]-- There are times when a Sierra application needs to know the topology of the faces in the model. This container contains a list of the face topology types in the model. It is populated by the concrete database types. \item [\code{Ioel::ElementTopology *commonFaceTopology}]-- If there is a single face topology in the model, this is set to that value; otherwise it is \code{NULL}. This is used to speed up some face topology queries. \end{description} \subsubsection{Interface} Each concrete database class must derive from the \code{Ioss::DatabaseIO} class and provide an implementation of several functions. The most obvious of these is the constructor: \begin{source} DatabaseIO (Region *region, const std::string \&filename, Ioss::EventInterest db\_usage) \end{source} The constructor is passed a pointer to an \code{Ioss::Region}, a \code{filename}, and a `\code{db\_usage}' field. The region can be \code{NULL} at this time if the database is an input database. The \code{db\_usage} specifies what this database will be used for and have valid values of \code{WRITE\_RESTART}, \code{READ\_RESTART}, \code{WRITE\_RESULTS}, \code{READ\_MODEL}, \code{WRITE\_HISTORY}, or \code{WRITE\_HEARTBEAT}. If this is an output database, then it should not be opened at this time since the user may have specified the same file for the reading of an initial state or a restart and we don't want to overwrite any data that may be needed. If an input database, then it can be opened to check that it exists, but no major processing of data should occur. For an input database, data processing will occur in the `\code{read\_meta\_data()}' function. When this is called, the database will be in state \code{STATE\_DEFINE\_MODEL}. It should then create all \code{GroupingEntities} and add them to the \code{Region} to create the lightweight \code{Ioss} model. Any properties should be added to the \code{GroupingEntities} at this time also as should transient fields. Basically, the \code{Ioss} system should have a complete metadata representation of the model at this time and should be able to respond to any queries about the model that do not involve bulk data without accessing the database. For an output database, the Sierra application through the \code{Frio} classes will build the \code{Ioss} metadata representation of the model while in \code{STATE\_DEFINE\_MODEL}. When Ioss::DatabaseIO::end() is called to exit from \code{STATE\_DEFINE\_MODEL}, then the metadata model is complete and the database can write whatever non{}-bulk data is needed at that point. The Ioss system will next go into the \code{STATE\_MODEL} at this point and the fields with a role of \code{MODEL} will be written or read via the field interface. As documented earlier, all bulk data reading and writing is done via \code{Ioss::Field}s. The \code{Ioss::DatabaseIO} class has several field input and output functions; one for each \code{GroupingEntity} type: \begin{source} int get\_field (const Region *reg, const Field \&field, void *data, size\_t data\_size) int get\_field (const NodeBlock *nb, const Field \&field, void *data, size\_t data\_size) int get\_field (const ElementBlock *eb, const Field \&field, void *data, size\_t data\_size) int get\_field (const SideBlock *sb, const Field \&field, void *data, size\_t data\_size) int get\_field (const NodeSet *ns, const Field \&field, void *data, size\_t data\_size) int get\_field (const SideSet *ss, const Field \&field, void *data, size\_t data\_size) int get\_field (const CommSet *cs, const Field \&field, void *data, size\_t data\_size) int put\_field (const Region *reg, const Field \&field, void *data, size\_t data\_size) int put\_field (const NodeBlock *nb, const Field \&field, void *data, size\_t data\_size) int put\_field (const ElementBlock *eb, const Field \&field, void *data, size\_t data\_size) int put\_field (const SideBlock *sb, const Field \&field, void *data, size\_t data\_size) int put\_field (const NodeSet *ns, const Field \&field, void *data, size\_t data\_size) int put\_field (const SideSet *ss, const Field \&field, void *data, size\_t data\_size) int put\_field (const CommSet *cs, const Field \&field, void *data, size\_t data\_size) \end{source} The implementation of these functions simply do some optional logging and then call the private pure virtual function \code{put\_field\_internal(...)} with the exact same arguments that they were called with.\footnote{See \url{http://www.gotw.ca/publications/mill18.htm} for a discussion of why (pure) virtual functions should not appear in the public interface. Note that this guideline is both respected and violated in Ioss::DatabaseIO....} The underlying concrete database must either read or write the specified field when the function is called. It is recommended to treat the `\code{data}' field as readonly at this time, so if any reordering or processing of the data is required, a scratch array should be allocated to hold the modified data\footnote{I am considering adding an argument to the field functions which would indicate whether the `data' can be modified or should be treated as readonly. This could help a little with memory use, but most cases don't need to do any modifications of the field....}. One of the first fields read/written from/to most \code{GroupingEntities} is the ``ids'' field. This defines a global id for each entity in the \code{GroupingEntity} and also defines the ordering of all subsequent fields on that entity. For example, if the id field for a nodeblock is \{1,5,2,4,3\}, then a nodal coordinate of \{1.0, 2.0, 3.0, 4.0, 5.0\} would assign 1.0 to node 1, 2.0 to node 5, 3.0 to node 2, 4.0 to node 4 and 5.0 to node 3. There is a caveat to this which is discussed in a later section. Once the \code{STATE\_MODEL} state is finished, the application may enter the \code{STATE\_DEFINE\_TRANSIENT} state for an output database. It will then define what fields are to be written to each of the GroupingEntities. Once these fields are defined, the database will enter \code{STATE\_TRANSIENT}. In \code{STATE\_TRANSIENT}, the transient fields are written. The application will call: \begin{source} Ioss::DatabaseIO::begin\_state(Ioss::Region *region, int state, Real time) \end{source} which tells the database that the application will start writing transient fields applying to time `\code{time}'. The `\code{state}' will be an sequentially increasing counter which indicates how many transient steps have been written. It is 1{}-based. This function can also be called for an input database if the application is going to be reading transient data from the input database. In this case, access to the state can be essentially random; the application does not have to read from step 1...end\_step sequentially. The function: \begin{source} Ioss::DatabaseIO::end\_state(Ioss::Region *region, int state, Real time) \end{source} will be called when all transient fields for this state have been written. At this time, the \code{DatabaseIO} class can do whatever is needed to finalize this particular timestep. \subsubsection{Database Traits Interface} In addition to the model and field{}-related functions defined above, there are a few functions which specify characteristics of the database. They are: \begin{description} \item [\code{virtual bool supports\_nodal\_fields()}]-- true/false depending on whether this database type supports fields written on nodes in nodeblocks. \item [\code{virtual bool supports\_side\_fields()}]-- true/false depending on whether this database type supports fields written on sides in sidesets. \item [\code{virtual bool supports\_element\_fields()}]-- true/false depending on whether this database type supports fields written on elements in element blocks. \item [\code{virtual bool supports\_nodelist\_fields()}]-- true/false depending on whether this database type supports fields written on nodes in nodelists. \item [\code{virtual Int node\_global\_to\_local(Int global, bool must\_exist)}]-- Provides a mapping from a global node id down to the local 1..entity\_count position in the field data. In the `id' and coordinate example above, the local position of global node `5' is `2'. If the boolean `\code{must\_exist}' is true, then it is an error if the specified node does not exist; otherwise if \code{must\_exist} is false, then the function returns `0' if a node with that global id does not exist. \item [\code{virtual int maximum\_symbol\_length()}]-- Return the maximum length of the `field names' that the database can handle; return 0 if it is unlimited. This is used by the restart system which must `mangle' the field names down to the maximum size that can be supported on the database. \end{description} \section{Special Cases --Dynamic Topology} Although the above description is valid for most cases, there are some situations which must be handled to allow the database to be used in all situations. Dynamic topology which include element death, load balancing, and adaptivity cause the topology of the original mesh to change from the originally defined topology. \subsection{Element Death} In the element death case, elements are selectively ``killed'' or inactivated based on some criteria in the application. Most databases continue to output these elements but they are flagged via an element variable as whether they are active or inactive. This shouldn't cause much difficulty, but the Sierra framework reorders the elements when element death occurs, so the output database must map the element fields which are not ordered based on the new element ordering back to the original element ordering so that they can be output correctly. When the element id ordering is changed, the \code{Frio} system will inform the database by calling a \code{put\_field} on the element block entities ``ids'' field with the new ordering. This can be distinguished from the original output of the element ids by the fact that in the original ordering, the database will be in \code{STATE\_DEFINE\_MODEL}; in any subsequent reordering of the element ids, the database will be in \code{STATE\_TRANSIENT}. All subsequent output of element transient fields will then be in the current ordering and will need to be mapped to the original ordering before being written to the database. \subsection{Load Balance} The load balance case involves shuffling the current nodes and elements among processors. One or more nodes and elements will move to a new processor in this case. The current handling of this case is suboptimal and is based on the characteristics of the ExodusII database format which is the primary format currently used in Sierra applications. The ExodusII database creates a file per processor and all communications to these databases are independent of all other databases. At the end of the calculation, a separate program is used to join the per{}-processor databases into a single database. Because the per{}-processor databases are independent, the current handling of a load balance reshuffling is that the current databases are all closed and a new set is opened; a new model is defined; the transient fields are redefined; and then everything continues from that point. This is definitely not the optimal situation and will be changing in the future. It is expected that a new function will be added to the \code{Ioss::DatabaseIO} traits interface which will indicate whether a concrete database instance supports dynamic topology. If it is supported, then the \code{Frio} system will notify the database that shuffling of elements and nodes among processors has occurred and output a new set of node, element, face, and edge orderings. If the database does not support dynamic topology, execution will continue as above. \subsection{Adaptivity} The adaptivity case is handled the same way as the above load balance case. The only difference is if the database is being used to store restart data. In that case, instead of just outputting fields existing on all of the current generation nodes and elements; fields are output on the complete element and node hierarchy including the parent elements which have been adapted. As stated above, the current behavior will be changing in the future. \section{Database Registration} Before a database type can be used in a Sierra application, it must be registered and it must provide a method for creating an instance of itself. This is currently handled via an \code{Ioss::IOFactory} class as shown below: \begin{source} namespace IoXX \{ \ \ const IOFactory* IOFactory::factory()\{ \ \ \ \ static IOFactory registerThis; \ \ \ \ return \®isterThis; \ \ \} \ \ IOFactory::IOFactory() \ \ \ \ : Ioss::IOFactory({\textquotedbl}database\_type{\textquotedbl}) \{ \ \ \ \ Ioss::IOFactory::alias({\textquotedbl}database\_type{\textquotedbl}, {\textquotedbl}alias\_for\_database\_type{\textquotedbl}); \ \ \} \ \ Ioss::DatabaseIO* IOFactory::make\_IO(const std::string\& filename, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Ioss::EventInterest db\_usage) const \ \ \{ return new DatabaseIO(NULL, filename, db\_usage); \} \} \end{source} The database type must also be made known to the parsing system. This is done by editing the file \code{framework/parser/io/Ioss\_MeshInput.xml }and adding an enum\_entry with ``\code{database\_type}'' to the enum \code{DatabaseTypes} near the top of the file. Since Sierra applications are currently statically linked, some function in your database library must be called by an existing piece of Sierra code in order for the linker to pull in your library. If the database is to be available for all codes, the best way to do this is to add a call to your database's \code{IOFactory} to the code in \code{Ioinit::Initializer()} as shown below: \begin{source} namespace Ioinit \{ \ \ Initializer::Initializer() \{ \ \ \ \ Ioex::IOFactory::factory(); \ \ \ // ExodusII \ \ \ \ Iohb::IOFactory::factory(); \ \ // HeartBeat \ \ \ \ IoXX::IOFactory::factory(); \ // My new database type \ \ \} \ \} \end{source} If the database is only available to a few applications, then the call to the \code{factory()} method is usually placed in the applications main routine. If a database has some finalization functions which must be called at the end of execution (this is for a library, not a specific database instance), then a call to that code can be added to the \code{Ioinit::Initializer} destructor. The user can then request this database type in the input file using the line command \cmd{Database Type = database\_type} or \cmd{Database Type = alias\_for\_database\_type} \section{Build System} The \code{io\_system} uses the standard Sierra build system which is currently \code{BJam}. The main documentation page for BJam is \url{http://boost.org/boost-build2/index.html}, and documentation for using BJam with Sierra applications can be reached from the Sierra developers documentation page at \url{http://swi.sandia.gov/developers/develop.php}. The build system must also be notified that the new library is to be built and linked to the other code. The building of the library is controlled in the \file{NbtTools/io\_system/votd/Jamfile} file. If the new database requires any external libraries (which are referred to as ``TPL's'' which means ``third-party libraries'') which provide an API or other functionality, they are specified in the rules for the library. For example, the \code{ioex} library (exodusII) requires the exodusII, nemesis, and netcdf libraries and that is specified as: \begin{source} lib ioex : [ glob $(io_system-root)/src/exodusII/*.C ] ioss /sierra/utility//utility /tpl/exodus//exodus /tpl/nemesis//nemesis /tpl/netcdf-c//netcdf ; \end{source} The specification of what library versions will be used is specified in the \file{Nbtools/Jamroot} file: \begin{source} \ldots register-product tpl : exodus : exodusii : 4.72 ; register-product tpl : nemesis : nemesis : 3.09 ; register-product tpl : netcdf : netcdf : 3.6.2-snl1 ; \ldots \end{source} This specifies that version 4.72 of the ExodusII library, version 3.09 of the Nemesis library, and version 3.6.2-snl1 of the netcdf library will be used. Later on in the file, the Ioex library build instructions are defined as: \begin{source} lib ioex : [ glob $(io_system-root)/src/exodusII/*.C ] ioss /sierra/utility//utility /tpl/exodus//exodus /tpl/nemesis//nemesis /tpl/netcdf-c//netcdf ; \end{source} This specifies that the Ioex library is to be built. The source code for the library is in \file{src/exodusII}. And it consists of all \file{.C} files in that directory. The \code{Ioex} library ``requires'' the \code{ioss} library from the io\_system; the external \code{utility} library from \code{/sierra/utility}; and the \code{exodus}, \code{nemesis}, and \code{netcdf} third-party-libraries (tpl). There are similar sections for the other libraries that are part of the io\_system package. One last piece of information is to tell the build system how to build an application that uses the IO libraries. The build system is told this in the following code block for the \code{io\_shell} executable: \begin{source} exe io_shell : $(io_system-root)/src/main/Main_io_shell.C ioinit ioss iotr iocgns /sierra/utility//utility : @runtest-tag ; \end{source} Using this information, the build system should be able to generate compile flags and correctly order the libraries on the link line for the linker. A new database type can be added by following the patterns for the Ioex, Iocgns, and Iohb database types. \section{Testing} The Sierra unit test system contains an executable called \code{Utst\_io} which can be used to test the various \code{DatabaseIO} classes. This executable is part of the Sierra system, but it is also available in a standalone version which can be built outside of the Sierra tools. The executable reads a database of a specified format and then writes the data to a database of a specified format. This can be used to test your \code{DatabaseIO} class to make sure it reads the data correctly and writes the data correctly. It can also be used as a translator from one database format to another. \section{Summary} The above documentation gives a brief overview of the Sierra IO system. It is not enough detail for the reader to add a completely functioning new database type, but hopefully gives enough detail that the reader can understand existing database IO classes and using those as an example add a new database type. There are some idiosyncrasies in the current Ioss system, but hopefully as they are discovered, they will be removed via refactoring. The current interface is definitely biased towards the ExodusII functionality, but as more database types are added, I will attempt to remove these biases. \end{document}