Data Analysis Services Group - January 2012

News and Accomplishments

Version 2.1.0 of VAPOR was released in January.

Wes Jeanette, a grad student at CU Boulder, joined the VAPOR team as a student assistant.

VAPOR Project

Project information is available at: http://www.vapor.ucar.edu

TG GIG PY6 Award:

We continue to make progress toward the first release of the PIOVDC library:

• An IO wrapper was written for netCDF to allow buffered writes for (greatly) improved IO performance.
• PIOVDC was spit into VAPOR and PIO managed solutions. The  PIOVDC/VDF half of project was committed to the VAPOR repository and tested. The PIOVDC/PIO half was integrated into the newest PIO SVN release.
• Tested build system for PIOVDC/VDF
• Began testing build system for PIOVDC/PIO

XD Vis Award:

Collaborative work continued, with U.C. Davis Hank Childs, to build a VDC type 2 plugin reader for VisIt.

Development:

Version 2.1.0 of VAPOR was released in January. The latest release was download by over 500 users in the first two week.

With 2.1 behind us we are now focusing on developing new Drupal-based documentation, based on plans made while Kendall Southwick was working on the website.  This month we are moving all of the content under the Documentation menu over to a Drupal version, which will be indexed and have search capability, making it much easier for users to find answers to questions.  We hope that the reorganized website will be easier to maintain.  Markus Stobbs has been very helpful in showing us how to create the capabilities we want in the website

Administrative:

Interviewing was completed and Wes Jeanette, a grad student at CU Boulder, joined the VAPOR team as a student assistant. We're happy to have Wes aboard.

Education and Outreach:

The WRF Winter tutorial was held the week of January 23.  Alan updated the VAPOR documentation for the tutorial, so that the students would make use of 2.1 code.  We installed VAPOR 2.1 on the WRF lab machines.  The students were very interested in using VAPOR (more so than in any of our previous WRF tutorials; about 1/3 of the teams showed a strong interest.).  Cindy Bruyere indicated that she would like us to expand the tutorial to include a full day of postprocessing tools, including more VAPOR work.  We are planning to meet her in mid February to decide what we need to do.

Niklas Robler of DKRZ is putting together a workshop at EGU 2012 in April that will focus on visualization and analysis of earth science data.  Because of our travel constraints we have asked Rick Brownrigg (from vets) to talk about VAPOR at his presentation there.

Misc

Alan had a useful discussion with Mel Shapiro and Bill Hibbard (the author of vis5d).  Evidently the most important vis5d capabilities that are missing in VAPOR are scripting and isolines.  Mel also indicated that discrete color mapping may be a preliminary (and easier) alternative to providing isolines.

Software Research Projects

The VAPOR team participated in two new funding proposals:

• An NREL LDRD pre-proposal was submitted with former VAPOR student intern, Kenny Gruchalla, leading. The proposed work would extend the VAPOR progressive access data model to support spatially-varying refinement control: more information could be made available in spatial regions of interest.
• An NSF Sustainable Energy Proposal was submitted with Texas Tech U. (lead), NCAR's RAL, and CU. If funded the four-year award would support would support various enhancements to VAPOR aimed at the wind energy research community.

Publications, Papers & Presentations

• Clyne, J., Mininni, P., and Norton, A. "Physically Based Feature Tracking for CFD Data”, IEEE Transactions on Visualization and Computer Graphics (to appear, 2012)
• Lord, J., Rast, M., McKinlay, C., Clyne, J., and Mininni, P. Wavelet decomposition of forced turbulence: applicability of the iterative Donoho-Johnstone threshold. Physics of Fluid, (to appear, 2012)

NWSC Planning & Installation

• Installed CentOS 6.2 (RedHat Enterprise Linux 6.2 clone) on my test system to  start familiarizing myself with changes from RHEL 5.6.  Reviewed the RHEL 6.2
  documentation.  Tested several of the system recovery boot procedures.
• Started looking at the xCat documentation web site.

System Support

Data Analysis & Visualization Clusters

• Installed a newer version of tcsh in /contrib
• Installed the progressbar library for Python.

GLADE Storage Cluster

• Repartitioned the /glade/home backup groups into six subsets to reduce the running time of the full backups.
• Reviewed Bluearc NFS appliance documentation for possible use as a higher performance server for the /glade/home file system.
• After talking with DDN, increased disk timeout settings on the DCS9900 controllers to hopefully prevent the SCSI I/O errors that can cause filesystems to unmount.  We also changed a setting on the NSD servers to allow I/Os up to 2048KB over infiniband to the controllers, which were previously capped at 512KB.

Data Transfer Services

• Still awaiting establishment of a production UCAS authentication based MyProxy server for use with the GLADE GridFTP service.  We have not heard
  of any plans to deploy this yet.
• Examined GridFTP source code for possible support of simple read/write only (no metadata) access to HPSS.  Reviewed the source code for an older version of the NERSC/Argonne HPSS DSI module to see how it used the HPSS CLAPI.  Met to discuss this with other HSS personnel and came to the conclusion that using the HPSS CLAPI would be the best method to provide HPSS access via GridFTP. Will wait to find out about recent developments at NERSC in this area. It looks like file transfers to/from HPSS using GLADE file systems as an intermediate step may still provide higher overall throughput at the cost of having to manage the intermediate step.