You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
735 lines
28 KiB
735 lines
28 KiB
<!-------- @HEADER
|
|
!
|
|
! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
!
|
|
! Zoltan Toolkit for Load-balancing, Partitioning, Ordering and Coloring
|
|
! Copyright 2012 Sandia Corporation
|
|
!
|
|
! Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
|
|
! the U.S. Government retains certain rights in this software.
|
|
!
|
|
! Redistribution and use in source and binary forms, with or without
|
|
! modification, are permitted provided that the following conditions are
|
|
! met:
|
|
!
|
|
! 1. Redistributions of source code must retain the above copyright
|
|
! notice, this list of conditions and the following disclaimer.
|
|
!
|
|
! 2. Redistributions in binary form must reproduce the above copyright
|
|
! notice, this list of conditions and the following disclaimer in the
|
|
! documentation and/or other materials provided with the distribution.
|
|
!
|
|
! 3. Neither the name of the Corporation nor the names of the
|
|
! contributors may be used to endorse or promote products derived from
|
|
! this software without specific prior written permission.
|
|
!
|
|
! THIS SOFTWARE IS PROVIDED BY SANDIA CORPORATION "AS IS" AND ANY
|
|
! EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
! IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
|
! PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SANDIA CORPORATION OR THE
|
|
! CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
|
! EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
|
! PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
|
! PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
|
|
! LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
|
|
! NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
|
! SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
!
|
|
! Questions? Contact Karen Devine kddevin@sandia.gov
|
|
! Erik Boman egboman@sandia.gov
|
|
!
|
|
! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
|
|
!
|
|
! @HEADER
|
|
------->
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
|
<meta name="filename" content="Zoltan.html">
|
|
<meta name="review" content="28 May, 1999">
|
|
<meta name="subject" content="Zoltan Home Page">
|
|
<meta name="sandia.approval_type" content="formal">
|
|
<meta name="sandia.approved" content="SAND99-1375">
|
|
<meta name="sandia.create_date" content="05/28/99">
|
|
<meta name="keywords" content="Zoltan, Zoltan Home Page, Zoltan dynamic load balancing library, Zoltan parallel computing">
|
|
<meta name="description" content="Zoltan: Home Page for the Zoltan Library project at Sandia National Laboratories">
|
|
<meta name="GENERATOR" content="Mozilla/4.7 [en] (X11; U; SunOS 5.7 sun4u) [Netscape]">
|
|
<title>Zoltan</title>
|
|
|
|
<!----CHANGE INFORMATION IN AREAS WITH THIS HEADER---->
|
|
<!----SCROLL DOWN TO FIND OTHER AREAS TO BE CHANGED---->
|
|
<!--------CHANGE THE NAME AFTER THE DASH-------->
|
|
<!--------CHANGE THE FILENAME-------->
|
|
<!--------CHANGE THE REVIEW DATE-------->
|
|
<!--------CHANGE THE SUBJECT-------->
|
|
<link rel="schema.sandia" href="https://www.sandia.gov/html_schema.htm">
|
|
<!--------CHANGE THE SAND NUMBER INFO-------->
|
|
<!--------INSERT THE DATE DOCUMENT CREATED-------->
|
|
<!--------CHANGE THE PAGE OWNER AND EMAIL ADDRESS-------->
|
|
<link rev="made" title="name of contact" >
|
|
<!--------CHANGE THE PAGE MAKER AND EMAIL ADDRESS-------->
|
|
<!--------PLACE FIVE KEY WORDS WITHIN THE QUOTES-------->
|
|
<!---------------END OF THIS CHANGE AREA--------------->
|
|
</head>
|
|
<body text="#000000">
|
|
<!-- KDD Turned off alternative link colors in template; the >
|
|
<!-- following line was part of the above body command. >
|
|
<!-- link="#003366" vlink="#cc0033" alink="#000000">
|
|
<a NAME="TOP"></a><!---TOP BANNER AREA STARTS HERE--->
|
|
<table BORDER=0 valign="top" >
|
|
<tr VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="160" BGCOLOR="#003366">
|
|
<table BORDER=0 WIDTH="160" valign="top" >
|
|
<tr VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="160"><!--SANDIA LOGO AT TOP LEFT-->
|
|
<a href="https://www.sandia.gov/Main.html"><img SRC="https://www.sandia.gov/images/snlstkdc.gif" ALT="[Sandia National Laboratories]" BORDER=0 valign="top" height=49 width=126></a>
|
|
<p><img ISMAP SRC="https://www.sandia.gov/images/labelNEW.gif" ALT="[navigation panel]" HSPACE=2 BORDER=0 usemap="#shortMap" height=119 width=111></td>
|
|
|
|
<td><img SRC="https://www.sandia.gov/images/1pixel.gif" BORDER=0 height=1 width=10></td>
|
|
</tr>
|
|
</table>
|
|
|
|
<table BORDER=0 WIDTH="160" valign="top" >
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 0th little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td><b><font face="Verdana, Arial, Helvetica"><a href="Zoltan.html">Zoltan
|
|
Home Page</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 1st little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td><b><font face="Verdana, Arial, Helvetica"><a href="ug_html/ug.html">Zoltan
|
|
User's Guide</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 2nd little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td><b><font face="Verdana, Arial, Helvetica"><a href="dev_html/dev.html">Zoltan
|
|
Developer's Guide</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 2A-nd little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td><b><font face="Verdana, Arial, Helvetica"><a href="Zoltan_FAQ.html">
|
|
Frequently Asked Questions</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 3rd little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td COLSPAN="2"><b><font face="Verdana, Arial, Helvetica"><a href="Zoltan_phil.html">Zoltan
|
|
Project Description</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 4th little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td COLSPAN="2"><b><font face="Verdana, Arial, Helvetica"><a href="Zoltan_pubs.html">Papers
|
|
and Presentations</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 4Ath little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td COLSPAN="2"><b><font face="Verdana, Arial, Helvetica"><a href="Zoltan_cite.html">How to Cite Zoltan</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 5th little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td COLSPAN="2"><b><font face="Verdana, Arial, Helvetica"><a href="https://github.com/sandialabs/Zoltan/releases">Download
|
|
Zoltan</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 6th little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td COLSPAN="2"><b><font face="Verdana, Arial, Helvetica"><a href="Zoltan_bugreport.html">Report a Zoltan Bug</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
|
|
<!-------------------------------------------------------------------------->
|
|
<tr ALIGN=LEFT VALIGN=TOP>
|
|
<td VALIGN=TOP WIDTH="150"><!----------- 7th little turquoise bevel button ------------>
|
|
<table BORDER=0 CELLSPACING=0 CELLPADDING=0 WIDTH="150" BGCOLOR="#00CCFF" >
|
|
<tr ALIGN=CENTER VALIGN=CENTER>
|
|
<td COLSPAN="2"><b><font face="Verdana, Arial, Helvetica">
|
|
<a href="mailto: zoltan-dev@software.sandia.gov">Contact Zoltan Developers</a></font></b></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP WIDTH="20"></td>
|
|
</tr>
|
|
|
|
<tr VALIGN=TOP>
|
|
<td COLSPAN="2"></td>
|
|
</tr>
|
|
<!-------------------------------------------------------------------------->
|
|
</table>
|
|
</td>
|
|
|
|
<td VALIGN=TOP>
|
|
<!--MAIN CONTENT AREA STARTS HERE-->
|
|
<!----------------THIS IS A CHANGE AREA---------------->
|
|
<!------HEADER TEXT SHOULD BE REPLACE THIS TEXT------>
|
|
<b><font face="Verdana, Arial, Helvetica"><font size=+2>
|
|
Zoltan:
|
|
</font></font></b>
|
|
<br>
|
|
<b><font face="Verdana, Arial, Helvetica"><font size=+2>
|
|
Data-Management Services for Parallel Applications
|
|
</font></font></b>
|
|
<p>
|
|
<!---------------END OF THIS CHANGE AREA--------------->
|
|
<!----------------THIS IS A CHANGE AREA---------------->
|
|
<!--MAIN CONTENT SHOULD BE PLACED IN THE AREA BELOW-->
|
|
<!------------------------------------------------------------------------->
|
|
<!------------------------------------------------------------------------->
|
|
<!------------------------------------------------------------------------->
|
|
<b><font face="Verdana, Arial, Helvetica"><font size=+2>
|
|
Project Description
|
|
</font></font></b><br>
|
|
<hr>
|
|
<ul>
|
|
<li>
|
|
<a href="Zoltan_phil.html#Overview">Overview</a></li>
|
|
<li>
|
|
Design Considerations
|
|
<ul>
|
|
<li>
|
|
<a href="Zoltan_phil.html#Design">Design of Toolkits and Libraries</a>
|
|
</li>
|
|
|
|
<li>
|
|
<a href="Zoltan_phil.html#ZDesign">Zoltan's Design</a></li>
|
|
|
|
<li>
|
|
<a href="Zoltan_phil.html#ZExamples">Zoltan Examples</a></li>
|
|
</ul>
|
|
</li>
|
|
|
|
<li>
|
|
Dynamic Load-Balancing and Data Migration Tools
|
|
<ul>
|
|
<li>
|
|
<a href="Zoltan_phil.html#Typical">Typical Approach to Dynamic Load Balancing</a
|
|
></li>
|
|
<li>
|
|
<a href="Zoltan_phil.html#Harder">Why Dynamic Load Balancing is Harder
|
|
than Static Partitioning</a></li>
|
|
<li>
|
|
<a href="Zoltan_phil.html#ZLB">Zoltan's Load-Balancing Suite</a></li>
|
|
<li>
|
|
<a href="Zoltan_phil.html#Mig">Data Migration Tools</a></li>
|
|
</ul>
|
|
</li>
|
|
|
|
<li>
|
|
Communication Tools
|
|
<ul>
|
|
<li>
|
|
<a href="Zoltan_phil.html#UComm">Unstructured Communication Library</a>
|
|
</li>
|
|
<li>
|
|
<a href="Zoltan_phil.html#DD">Distributed Data Directories</a>
|
|
</li>
|
|
</ul>
|
|
</li>
|
|
|
|
<li>
|
|
<a href="Zoltan_phil.html#MemTools">Memory Management Tools</a></li>
|
|
|
|
|
|
</ul>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="Overview"><hr><b>Overview</b><p>
|
|
The Zoltan library is a collection of data management services for parallel,
|
|
unstructured, adaptive, and dynamic applications. It simplifies the
|
|
load-balancing, data movement, unstructured communication, and memory usage
|
|
difficulties that arise in dynamic applications such as adaptive
|
|
finite-element methods, particle methods, and crash simulations.
|
|
Zoltan's data-structure neutral design also lets a wide range of applications
|
|
use it without imposing restrictions on application data structures.
|
|
Its object-based interface provides a simple and inexpensive way for
|
|
application developers to use the library and researchers to make new
|
|
capabilities available under a common interface.
|
|
<p>
|
|
Zoltan provides tools that help developers of parallel applications.
|
|
<ul>
|
|
<li>
|
|
A suite of <a href="ug_html/ug_alg.html">parallel partitioning algorithms</a>
|
|
and <a href="ug_html/ug_interface_mig.html">data migration tools</a>
|
|
dynamically redistribute data to reflect, say, changing processor workloads.
|
|
</li>
|
|
<li>
|
|
<a href="ug_html/ug_util_dd.html">Distributed data directories</a> and
|
|
<a href="ug_html/ug_util_comm.html">unstructured communication</a>
|
|
services let
|
|
applications perform complication communication using only a few simple
|
|
primitives.
|
|
</li>
|
|
<li>
|
|
<a href="ug_html/ug_util_mem.html">Dynamic memory management tools</a>
|
|
enhance common memory allocation functions to
|
|
simplify debugging of dynamic memory usage.
|
|
</li>
|
|
</ul>
|
|
These tools are provided in an easy-to-use toolkit that is callable from C,
|
|
C++, and Fortran90.
|
|
|
|
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="Design"><hr><b>Design of Toolkits and Libraries</b><p>
|
|
Using general-purpose libraries allows
|
|
algorithms to be shared among and compared within many applications. The
|
|
close dependence of libraries on application data, however,
|
|
requires careful design to maintain separation between the
|
|
libraries and application data structures.
|
|
<p>
|
|
|
|
One way to provide this separation is to use object-based software design.
|
|
Instead of requiring the application to build data structures
|
|
required by the library, the application could pass functions that access
|
|
the application data structure to the libraries.
|
|
For example, rather than require an application to build a complicated graph
|
|
description, the library can require an application to provide a
|
|
function returning graph vertices and a function returning edge
|
|
connectivity for a given vertex.
|
|
Using these functions, the
|
|
library can build the data structures it needs.
|
|
<p>
|
|
|
|
This object-based design has a number of advantages.
|
|
<ul>
|
|
<li>
|
|
Changes in the library's data structures need not
|
|
propagate back to the application. As long as the set of required functions
|
|
does not change, the application does not need to change to use new versions
|
|
of the library.
|
|
</li>
|
|
<li>
|
|
Once the set of required functions is implemented, the application can use all
|
|
the algorithms in the library.
|
|
</li>
|
|
|
|
<li>
|
|
The required functions are generally easy for
|
|
an application to implement, as most applications need to
|
|
access their data objects and the interactions between objects
|
|
for their own computations.
|
|
</li>
|
|
<li>
|
|
Memory usage is lower as
|
|
an application does not have to build an intermediate data structure
|
|
that is later converted to appropriate data structures for the library.
|
|
</li>
|
|
<li>
|
|
The constructor for library data structures is called only when it
|
|
is needed, and only the data needed for a particular algorithm is obtained.
|
|
</li>
|
|
</ul>
|
|
<p>
|
|
There are a few disadvantages to this object-based approach as well.
|
|
<ul>
|
|
<li>
|
|
Additional overhead is incurred as the library calls the functions to
|
|
build its data structures.
|
|
In experiments, however, this cost has been very low
|
|
relative to the cost of actual computation in the library.
|
|
</li>
|
|
<li>
|
|
A general-purposes tool can
|
|
provide only limited support for manipulations of application data
|
|
structures (e.g., data movement).
|
|
</li>
|
|
</ul>
|
|
<p>
|
|
For more detailed information, see
|
|
[<a href="http://cs.sandia.gov/pub/papers/kddevin/cmame.ps.gz">Hendrickson
|
|
and Devine</a>].
|
|
<p>
|
|
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="ZDesign"><hr><b>Zoltan's Design</b><p>
|
|
We have chosen an object-based, callback function design. An application
|
|
provides a number of simple callback functions that access the application
|
|
data structures. Zoltan then calls these functions to obtain data it needs.
|
|
Geometric algorithms are
|
|
supported via callback functions returning objects to be balanced and the
|
|
weights and coordinates of those objects.
|
|
Graph-based algorithms are
|
|
supported by callback functions returning objects to be
|
|
balanced, edges between objects, and object and edge weights.
|
|
For refinement-tree algorithms, additional callback functions return
|
|
parent-child relationships.
|
|
<p>
|
|
Support for data migration (the movement of data to establish a new
|
|
decomposition) is also provided through a similar callback
|
|
function interface. An application provides callback functions that pack
|
|
object data into and unpack data from communication buffers provided by
|
|
Zoltan. Zoltan calls the packing function to load communication buffers,
|
|
performs the communication necessary to move the data, and calls the unpacking
|
|
function to unload the communication buffers.
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="ZExamples"><hr><b>Zoltan Examples</b><p>
|
|
Several examples of Zoltan's use can be found in the
|
|
<a href="ug_html/ug.html">Zoltan User's Guide</a>.
|
|
<ul>
|
|
<li>
|
|
<a href="ug_html/ug_intro.html#Load-Balancing Tools">General
|
|
examples of library usage</a>.
|
|
</li>
|
|
<li>
|
|
<a href="ug_html/ug_examples.html">Specific examples</a>
|
|
for each phase of balancing
|
|
(<a href="ug_html/ug_examples_init.html">initialization</a>,
|
|
<a href="ug_html/ug_examples_lb.html">load balancing</a>, and
|
|
<a href="ug_html/ug_examples_mig.html">data migration</a>).
|
|
</li>
|
|
<li>
|
|
<a href="ug_html/ug_examples_query.html">Examples of callback function
|
|
implementations</a>.
|
|
</li>
|
|
</ul>
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="Typical"><hr><b>Typical Approach to Dynamic Load Balancing</b><p>
|
|
Dynamic load balancing has been used in many applications, ranging from
|
|
adaptive mesh refinement to particle methods to contact detection algorithms.
|
|
In most applications using dynamic load balancing, the load-balancing
|
|
algorithm is implemented directly in the application, with close coupling
|
|
between the application's and load-balancing algorithm's data structures.
|
|
This typical approach has two disadvantages.
|
|
<ul>
|
|
<li> It is possible that the application developer did not select the
|
|
best algorithm for the application, but the developer is unable to compare the
|
|
algorithm with others without taking time to implement many algorithms in the
|
|
application.
|
|
</li>
|
|
<li> The close coupling of the algorithm's and application's
|
|
data structures limits the algorithm's use to a single application.
|
|
Developers wanting to use the algorithm in a new application have to re-write
|
|
the algorithm using the new application's data structures.
|
|
</li>
|
|
</ul>
|
|
As a result, research into and use of dynamic load-balancing algorithms are
|
|
severely impaired.
|
|
<p>
|
|
|
|
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="Harder"><hr><b>Why Dynamic Load Balancing is Harder than Static Partitioning</b><p>
|
|
Many high-quality static partitioning tools exist; examples include
|
|
<a href="http://cs.sandia.gov/CRF/chac.html">Chaco</a>,
|
|
<a href="https://www-users.cs.umn.edu/~karypis/metis">METIS</a>,
|
|
<!---
|
|
<a href="https://www.uni-paderborn.de/fachbereich/AG/monien/RESEARCH/PART/party.html">Party</a>,
|
|
-->
|
|
and
|
|
<a href="https://www.labri.fr/perso/pelegrin/scotch/">SCOTCH</a>.
|
|
General-purpose dynamic load-balancing tools are less common, however,
|
|
since they are more difficult to implement. The difficulty arises from
|
|
fundamental algorithmic and software-engineering
|
|
differences between static and dynamic partitioning. These differences are
|
|
summarized in the following table.
|
|
<p>
|
|
|
|
<table border=1>
|
|
<tr>
|
|
<td valign=top><b>Static Partitioning...</b></td>
|
|
<td valign=top><b>Dynamic Load Balancing...</b></td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Generally used as a pre-processor to an application.</td>
|
|
<td valign=top>Runs side-by-side with an application.</td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Can be (and usually is) implemented serially.</td>
|
|
<td valign=top>Must be implemented in parallel.</td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Has only modest concern for execution time.</td>
|
|
<td valign=top>Must run quickly (time to load balance should not exceed time to run in
|
|
an unbalanced state).</td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Has only modest concern for memory usage.</td>
|
|
<td valign=top>Must use little memory (should not affect scalability of
|
|
application).</td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Can use file-based interfaces (read geometry from a file; write partition
|
|
info to a file).</td>
|
|
<td valign=top>Must use function-call interfaces.</td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Has no dependence on an application's data structures.</td>
|
|
<td valign=top>Needs information stored in an application's data structures.</td>
|
|
</tr>
|
|
<tr>
|
|
<td valign=top>Accounts for part sizes and communication costs.</td>
|
|
<td valign=top>Accounts for part sizes, communication costs, and data movement
|
|
costs.</td>
|
|
</tr>
|
|
</table>
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="ZLB"><hr><b>Zoltan's Load-Balancing Suite</b></a>
|
|
<p>
|
|
In our experience, no single partitioning strategy is effective for all
|
|
parallel computations. Some application require partitions based only on the
|
|
problem's workloads and geometry; others benefit from explicit consideration
|
|
of dependencies between data. Some applications require the highest quality
|
|
partitions possible, regardless of the cost to generate them; others can
|
|
sacrifice some quality so long as new partitions can be generated quickly.
|
|
For some applications, the cost to relocate data is prohibitively high, so
|
|
incremental partitioning algorithms are needed; other applications can
|
|
tolerate greater remapping costs. Most important, application developers
|
|
might not know in advance which strategies work best in their applications, so
|
|
the need a convenient means of comparing algorithms.
|
|
<p>
|
|
We provide two classes of parallel partitioning algorithms in the Zoltan
|
|
library:
|
|
<ul>
|
|
<li>Geometric methods
|
|
<blockquote>
|
|
<a href="ug_html/ug_alg_rcb.html">Recursive Coordinate Bisection</a><br>
|
|
<a href="ug_html/ug_alg_rib.html">Recursive Inertial Bisection</a><br>
|
|
<a href="ug_html/ug_alg_hsfc.html">Hilbert Space-Filling Curve Partitioning</a> <br>
|
|
<a href="ug_html/ug_alg_reftree.html">Refinement Tree Partitioning</a>
|
|
</blockquote>
|
|
</li>
|
|
<li>Topology-based methods
|
|
<blockquote>
|
|
<a href="ug_html/ug_alg_phg.htmml">Hypergraph partitioning</a><br>
|
|
<a href="ug_html/ug_alg_phg.htmml">Graph partitioning</a><br>
|
|
Graph partitioning (through interfaces to
|
|
<a href="ug_html/ug_alg_parmetis.html">ParMETIS</a> (U. Minn.) and
|
|
<a href="ug_html/ug_alg_jostle.html">Jostle</a> (U. Greenwich))<br>
|
|
</li>
|
|
</ul>
|
|
|
|
Once the Zoltan callback functions are implemented, an application can switch
|
|
between partitioning algorithms by changing only the
|
|
<a href="ug_html/ug_alg.html#LB_METHOD"><i>LB_METHOD</i></a> parameter
|
|
through a call to
|
|
<a href="ug_html/ug_interface_init.html#Zoltan_Set_Param"><b>Zoltan_Set_Param</b></a>.
|
|
Thus, comparing different algorithms within a single application is easy,
|
|
enabling users to try several algorithms and find
|
|
the best ones for their applications.
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="Mig"><hr><b>Data Migration Tools</b></a>
|
|
<p>
|
|
A complicated part of dynamic repartitioning is the need to move data from old
|
|
processors to new ones. This data migration requires deletions and insertions
|
|
from the application data structures as well as communication between the
|
|
processors.
|
|
<p>
|
|
To help an application with
|
|
<a href="ug_html/ug_interface_mig.html">data migration</a>, Zoltan requires an application to
|
|
supply callback functions that pack data into communication buffers and unpack
|
|
data from communication buffers. Zoltan
|
|
calls the packing function to load communication buffers with objects to be
|
|
exported, performs all communication needed to move the data, and calls the
|
|
unpacking function to load the objects in the data structures on the new
|
|
processors. This mechanism eliminates the need for the application developer
|
|
to implement complicated communication for data migration.
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="UComm"><hr><b>Unstructured Communication Library</b></a>
|
|
<p>
|
|
Unlike static applications, where communication patterns remain fixed
|
|
throughout the computation, dynamic applications can have complicated,
|
|
changing communication patterns. For example:
|
|
<ul>
|
|
<li>After adaptive mesh refinement,
|
|
new communication patterns must reflect dependencies between newly created
|
|
elements.
|
|
</li>
|
|
<li>
|
|
Multiphysics simulations, such as crash simulations, might require complicated
|
|
communication to transfer data between decompositions for different simulation
|
|
phases.
|
|
</li>
|
|
</ul>
|
|
Zoltan provides an
|
|
<a href="ug_html/ug_util_comm.html">unstructured communication package</a>
|
|
to simplify
|
|
communication. The package builds a communication plan, including information
|
|
about both message sends and receives for a given processor. The plan can be
|
|
reused throughout the application, or destroyed and rebuilt when communication
|
|
patterns change. The package also includes simple communication primitives that
|
|
insulate the user from details of message sends and receives.
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="DD"><hr><b>Distributed Data Directories</b></a>
|
|
<p>
|
|
Dynamic applications often need to locate off-processor information. After
|
|
repartitioning, for example, a processor might need to rebuild ghost cells and
|
|
lists of data to be communicated. It might know which data it needs, but not
|
|
where the data are located.
|
|
<p>
|
|
To help locate off-processor data, Zoltan includes a
|
|
<a href="ug_html/ug_util_dd.html">distributed data
|
|
directory</a> tool that is scalable with respect to both memory usage and
|
|
computation time. Processors register their owned objects with the directory.
|
|
Then, through a rendezvous algorithm, other processors can look up the
|
|
location of data they need.
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
<a name="MemTools"><hr><b>Memory Management Tools</b></a>
|
|
<p>
|
|
Dynamic applications rely heavily on the ability to allocate and free memory
|
|
as needed. Memory leaks and invalid memory accesses are common to developing
|
|
software. Although many software development tools let users track memory
|
|
bugs, these tools are often not available on state-of-the-art parallel
|
|
computers.
|
|
<p>
|
|
Zoltan's <a href="ug_html/ug_util_mem.html">memory management package</a>
|
|
provides simple in-application debugging
|
|
tools that are beneficial on state-of-the-art computing platforms. The
|
|
package includes wrappers around malloc and free that record the location of
|
|
all memory
|
|
allocation operations. Thus, tracking memory leaks is simplified, as
|
|
source-code locations of unfreed-memory allocations can be printed.
|
|
Statistics about memory allocations and frees are also available.
|
|
<p>
|
|
<!---------------------------------------------------------------------------->
|
|
|
|
<!---------MAIN CONTENT AREA ENDS HERE---------><!-- CHANGE CONTACT + E-MAIL, NOTE "SUBJECT" IN E-MAIL CODE --></td>
|
|
</tr>
|
|
</table>
|
|
|
|
<hr width="100%">
|
|
<table BORDER=0 WIDTH="100%" >
|
|
<tr ALIGN=CENTER>
|
|
<td VALIGN=TOP WIDTH="140">
|
|
<table BORDER=0 WIDTH="140" >
|
|
<tr>
|
|
<td ALIGN=CENTER VALIGN=TOP WIDTH="120"></td>
|
|
|
|
<td WIDTH="20"></td>
|
|
</tr>
|
|
</table>
|
|
</td>
|
|
|
|
<td ALIGN=CENTER VALIGN=TOP WIDTH="100%"></td>
|
|
</tr>
|
|
</table>
|
|
<!--Image maps below-->
|
|
<map name="shortMap">
|
|
<area shape="rect" coords="2,2,108,14"href="https://www.sandia.gov/about/index.html"></area>
|
|
<area shape="rect" coords="2,19,108,31"href="https://www.sandia.gov/mission/ste/index.html"></area>
|
|
<area shape="rect" coords="2,36,108,48"href="https://www.sandia.gov/mission/index.html"></area>
|
|
<area shape="rect" coords="2,53,108,65"href="https://www.sandia.gov/contact-us/index.html"></area>
|
|
<area shape="rect" coords="2,70,108,82"href="https://www.sandia.gov/news/index.html"></area>
|
|
<area shape="rect" coords="2,87,108,99"href="https://www.sandia.gov/search/index.html"></area>
|
|
<area shape="rect" coords="2,104,108,116"href="https://www.sandia.gov/Main.html"></area>
|
|
</map>
|
|
<!----------------THIS IS A CHANGE AREA---------------->
|
|
<!----NAME AND DATE OF LAST REVISION SHOULD BE HERE---->
|
|
<!---------------END OF THIS CHANGE AREA--------------->
|
|
</body>
|
|
</html>
|
|
|