Visualization on New York Blue - PowerPoint PPT Presentation

1 / 45
About This Presentation

Visualization on New York Blue


New York Blue needs a state-of-art parallel visualization software. Negative features ... Blue Gene/P: 1-PFLOPS, 294,912-processors, 72-racks ... – PowerPoint PPT presentation

Number of Views:68
Avg rating:3.0/5.0
Slides: 46
Provided by: Stony9
Tags: blue | new | visualization | york


Transcript and Presenter's Notes

Title: Visualization on New York Blue

Visualization on New York Blue Roman
Samulyak AMS Department, Stony Brook
University and Computational Science
Center Brookhaven National Laboratory
  • Hardware for visualization
  • Main visualization software development projects
  • Visualization with VisIt
  • Ideas of remote visualization
  • VisIt capabilities
  • Serial and parallel visualization
  • Supported data formats

New York Blue
  • Features of New York Blue
  • IBM Blue Gene
  • 128 teraflops
  • 18-rack configuration
  • 36,864 processors
  • Typical simulation generates large data sets
  • Data rendering / visualization of data is not
    possible in serial
  • New York Blue needs a state-of-art parallel
    visualization software
  • Negative features
  • Trimmed version of operating system
  • Lack of graphics libraries etc.
  • Sophisticated visualization software can not be
    installed on IBM Blue Gene

Visualization Cluster
  • A practical solution suggested by visualization
  • perform visualization on a separate
    visualization cluster
  • cluster must share home directories with the
    main supercomputer
  • fast wide bandwidth connection to the main
  • LLNL visualization solution
  • BlueGeneL 478 TFLOPS sustained and 596 TFLOPS
    peak performance
  • Blue Gene/P 1-PFLOPS, 294,912-processors,
  • Visualization cluster 512 Intel based nodes
  • BNL initial visualization solution
  • 4 node 32 processor cluster, IBM PowerPC, 64 bit
  • 8 processors on a node have shared memory (8 GB)
  • shared home directories, 1GB connection to the
    main machine
  • Name to

ASCI Visualization Software
  • ASCI (Advanced Strategic Computing Initiative)
    was a supercomputing initiative of the Department
    of Energy, created to help the maintenance of the
    United States nuclear arsenal after the 1992
    moratorium on nuclear testing
  • ASCI Supercomputers ASCI White, ASCI Red, ASCI
    Blue Pacific, ASCI Blue Mountain, ASCI Thor's
    Hammer, ASCI Purple, Blue Gene
  • Large parallel visualization research
  • Supported main visualization projects
  • SCIRun, Utah University, targeted shared memory
  • ParaView, Los Alamos and collaborators,
    distributed and shared memory supercomputers
  • VisIt, Lawrence Livermore National Laboratory,
    distributed and shared memory supercomputers
  • ASCI visualization software became publicly

SciRun http//
  • SCIRun is a Problem Solving Environment (PSE),
    for modeling, simulation and visualization of
    scientific problems. It is available for free and
    open source.

ParaView http//
  • The ParaView project started started in 2000 as
    a collaborative effort between Kitware Inc. and
    Los Alamos National Laboratory within ASCI Views
  • Today, ParaView development continues as a
    collaboration between Kitware, Sandia National
    Labs, CSimSoft, Los Alamos National Lab, Army
    Research Lab and others.
  • ParaView is an open-source, multi-platform
    application designed to visualize data sets of
    size varying from small to very large.
  • The goals of the ParaView project include the
  • Develop an open-source, multi-platform
    visualization application.
  • Support distributed computation models to
    process large data sets.
  • Create an open, flexible, and intuitive user
  • Develop an extensible architecture based on open

VisIt http//
  • VisIt is a free software application developed
    at LLNL for visualizing and analyzing terascale
    simulation datasets
  • Successor to MeshTV
  • Similar user interface
  • Many more features
  • Begun Summer 2000
  • Initial version Fall 2002

VTK The Visualization ToolKit (not explicitly
related to ASCI)
  • The Visualization ToolKit (VTK) is an open
    source, freely available software system for 3D
    computer graphics, image processing, and
  • VTK consists of a C class library, and several
    interpreted interface layers including Tcl/Tk,
    Java, and Python.
  • Professional support and products for VTK are
    provided by Kitware, Inc.
  • It may be difficult for some users to write
    parallel visualization codes using VTK libraries

VisIt on New York Blue
VisIt is installed on New York Blue visualization
cluster Directory rosamu/software/Visit contains
necessary packages Mesa, Silo, VTK, Python, Qt,
. and two versions of Visit Serial version
rosamu/software/Visit/visit/bin/visit Parallel
version rosamu/software/Visit/visit_par/bin/visi
VisIt is freely available
Download VisIt from the Web at http//
/visit Binary distributions Windows
2000/XP Linux MacOS X AIX (IBM) IRIX
(SGI) Tru64 (Compac) Same user interface on
each platform
VisIt is well documented and supported
  • Available on the web
  • Documentation
  • VisIt Getting Started Manual (30 pages)
  • VisIt Users Manual (356 pages)
  • Getting Data into VisIt (220 pages)
  • Includes file with example source code
  • VisIt Python Interface Manual (278 pages)
  • Frequently asked questions

VisIt can access your data
  • Reads over two dozen different database formats
  • Silo
  • Exodus
  • PDB (Flash, KullLite, PF3D, LEOS)
  • Mili
  • BoxLib
  • Ensight
  • VTK (FronTier)
  • Vista
  • HDF5
  • Database reader plug-ins can be developed for
    new formats

Variable types Scalar Vector Tensor
Material Species X,Y pairs
Supported mesh types
  • 1D Curves
  • 2D/3D meshes
  • Rectilinear
  • Curvilinear
  • Unstructured
  • Points
  • AMR

VisIt lets you see and examine your data
Plots Manipulate data or create new data
using operators and expressions Develop new
plots and operators as plug-ins VisIt supports
stereo rendering
  • Create derived
  • variables using data
  • from your database
  • Pick
  • Lineout
  • Query

VisIt can handle large datasets
You can run VisIt on your desktop computer and
have VisIt process large data in parallel on a
remote supercomputer Client/Server
architecture Uses fast local graphics
hardware MPI parallel compute engine
Scalable rendering in parallel for largest
VisIt architecture
4 main components Graphical User Interface
(GUI) Viewer Database server Compute
engine GUI and Viewer usually meant to run
locally on your desktop computer Database
server and parallel compute engine can run
on remote computers where the data files are
located and talk to the GUI and viewer running on
your desktop computer
VisIt has multiple interfaces
Use VisIt as an application or a library
C, Python, Java interfaces allow other
applications to control VisIt Interfaces
Graphical user interface Python programming
interface Java programming interface C
programming interface All interfaces send
commands to the viewer and in turn get the latest
state from the viewer Use GUI when interaction
is required Use Python interface to script
actions or use VisIt as a batch mode movie
generation tool
A brief tour
What is a plot?
A plot is a viewable object, created from a
database, that can be displayed in a
visualization window Type of plots
Pseudocolor Mesh FilledBoundary Boundary
Contour Volume Vector Surface Subset
Streamline Curve Histogram Tensor Plots
come from plug-ins so you can extend VisIts
plotting capabilities by writing a new plug-in
What is an operator?
An operator is a filter that is applied to a
database variable before the compute engine uses
that variable to generate a plot Operators
come from plug-ins so you can extend VisIts data
manipulation capabilities by writing a new plug-in
Reflect operator
The Reflect operator reflects database geometry
across one or more axes Use this operator when
your simulation data contains only part of the
geometry and relies on symmetry to recover the
rest of the geometry
Threshold operator
Removes cells whose value is not in the
specified range Use this operator when you only
want to look at cells that have values within an
interesting range
Inverse ghost zone operator
This operator makes ghost zones visible and
real zones invisible Use this operator when you
want to look at your databases ghost zones
Three slice operator
This operator slices plots using three
axisaligned slice planes Use this operator when
you want to see into the interior of 3D plots
Quantitative Analysis Expressions
VisIt allows you to create new derived
variables from values in your database using
VisIts expression language Use expressions to
derive quantities that were not stored in your
database Expressions can operate on scalars,
vectors, tensors, or on meshes VisIt provides
built-in math functions Trigonmetric
functions Logarithmic functions Vector
What is a valid expression?
VisIts expression syntax roughly resembles the
C language, although there are a few differences
Built-in expressions
The table to the right lists some of VisIts
built-in expression functions A more complete
list can be found in the VisIt Users manual or
in the help provided in VisIts Help window
Creating expressions
Databases can define expressions You can
create expressions using the Expression window
Click New button to create new blank expression
Fill in name, definition, variable type Click
Apply button
Remote Visualization
An Overview of Remote Visualization
Simulations are almost always run on a powerful
supercomputer Databases usually reside on the
computer that generated the data Run VisIt on
your local workstation but do the data I/O and
processing on the computer that generated the
data The GUI and viewer run locally while the
database server and parallel compute engine run
on the remote supercomputer Moving data is
not necessary No performance degradation due
to encryption of X11
Distributed mode
To run in Distributed mode, begin by opening
the File Selection Window and type the name of
the computer where the files are stored into the
Host text field You can also select the name
of a known host from the Hosts list Once the
database server is launched on the remote
computer, the files for the remote computer are
listed in the window. Add the files to be
visualized to the Selected files list and dismiss
the window Create plots as usual using a
remote file from the selected files list
Sometimes when you try to access files on a
remote computer, VisIt prompts you for a
password, opening a Password Window When
prompted, enter your password and click the OK
button or press the Enter key to proceed
Useful trick SSH Tunneling
  • Standard login to NYBlue or visualization
  • On your desktop ssh
    (enter password from the cryptocard)
  • On SSH gateway ssh (enter
    passphrase for the gateway)
  • How to login to NYBlue multiple times without
    entering any passwords / passphrases and without
    braking security rules?

SSH Tunneling I
Create config file in .ssh directory on your
desktop Host sshbgl? HostName
cal? User rosamu? LocalForward 30022 Host HostName localhost? User
rosamu? Port 30022? HostKeyAlias localhostvis
SSH Tunneling II
Then on the desktop ssh -N -f -q -X sshbgl enter
password from the cryptocard ssh-agent
SHELL ssh-add enter the local machine
passphrase ssh -X Lets
you in without password or passphrase If your
dont have a passphrase for the local
desktop, run ssh-keygen -t dsa and copy your into authorized_keyes on the
Connection progress dialog
The Connection progress window is opened when a
parallel compute engine cannot be launched after
a few seconds The dialog is visible until a
remote compute engine connects back to the viewer
or the connection is cancelled Clicking the
cancel button after VisIt submits a job to the
batch system does not remove the job On heavily
saturated batch systems, it might be prudent for
you to manually remove your compute engine job
from the batch queue
Host Profiles
When VisIt launches a compute engine, it looks
for something called a host profile Contains
information VisIt uses to launch a compute engine
on a remote computer remote user name number
of processors parallel launch method (many
other options) You may also have multiple host
profiles for any given computer Common to have
separate host profiles for running VisIt in
serial and parallel
Host Profile Window
The Host Profile Window is divided vertically
in two areas Top area contains tabs for host
profiles for a certain computers and controls to
create or delete host profiles The bottom area
of the window displays all attributes of the
selected host file
Host Profile Window, Continued
Setting Parallel Options
In order to make launching compute engines
easier, VisIt hides the details of the parallel
launch program, which may vary from computer to
computer VisIt allows you to set some common
parallel options and it figures out how to launch
the compute engine on the specified computer
Parallel options you can adjust
You may alter various Parallel Option settings
prior to running your job - partition/pool -
number of processors used - number of nodes -
parallel time limit given to run your program
Engine Option Window
This window provides a list of host profiles
from which to choose Once a host profile is
chosen, parallel options can be altered to fine
tune how the compute engine is launched Click
the OK button to confirm your choices or changes
Compute Engine Window
This windows purpose is to display the
progress of a compute engine as it completes a
task, via status bars The window displays
information about the compute engine such as how
many processors and nodes it uses Interrupt a
compute engine Close a compute engineCompute
Engine Window
Thank You
Write a Comment
User Comments (0)