Data Analysis Services Group - November 2011

News and Accomplishments

xxx

VAPOR Project

Project information is available at: http://www.vapor.ucar.edu

TG GIG PY6 Award:

Yannick is wrapping up development of the PIOVDC library and getting the code ready for distribution. At the request of John Dennis, who will distribute the code as part of PIO, a parallel VDC readback capability was added: VDC data sets can now be both written and read in parallel on a distributed memory platform. In the process of testing bug was uncovered when handling grids that were not block-aligned. The bug was fixed. Finally, the current version of the code is again experiencing performance issues that are still being investigated.

John resolved a couple of AIX portability bugs in the VDF library.

XD Vis Award:

Yannick continued work  VAPOR's ShaderMgr to provide support for GLSL compilation pre-processing (e.g. #ifdefs).

KISTI Award:

The period of performance for year one of the three-year KISTI award ended on 11/30. Most of the efforts of the group were focused on meeting required deliverables, all of which were met.

Sang Myeong Oh, from Jeju University in Korea, worked with us for most of this month, as part of our collaboration with Kisti.  He created sample data sets, converted them to VAPOR and visualized them, and in the process identified a number of issues that needed to be resolved. On his last day he showed us a number of visualizations he created and discussed some of the valuable capabilities he would like to have.  He did find VAPOR visualization to provide valuable understanding of his data.  In general his visit was quite useful to us and we hope to continue to collaborate now that he has returned to Korea.

Alan developed the momvdfcreate and mom2vdf applications, which we had agreed to implement as the first year deliverables of out Kisti agreement.  These applications convert MOM4 datasets to VAPOR datasets.  They operate much faster (25 times as fast) as the corresponding Python routines that we prototyped last summer.  The Python routines depend on several existing applications, such as CDO and NCL, which were operating too slowly on the data.  For the new code, we accelerated the conversion by reimplementing the code that calculates bilinear weights and remaps the data, going back to the original algorithms provided by Phil Jones of LANL.

Niklas Roeblas of DKRZ provided us with several ocean data sets, output from the MPI-OM ocean model that is used at DKRZ.  We found that such data may be visualized using the same code that we are developing for MOM4 data, with minor modification.

Progress was also made towards year-two deliverables. John refactored vaporgui's internal data model to use the an abstract class representation of a regular grid as the token for data exchange within the application. The RegularGrid class, and it's derivatives (SphericalGrid and LayeredGrid), support missing data values as a first class citizen, in particular, offering reasonable behavior when interpolants are undefined during data reconstruction. Two visualizers, the direct volume renderer (dvr) and the isosurface rendered, were re-written to treat missing values appropriately. Most of the effort involved rewriting the GPU hardware shaders that perform the actual rendering. Numerous pathological cases had to be supported. Additionally, the shaders were modified to interpolate layered (e.g. terrain following) grids on the fly, avoiding the re-gridding step required previously. Finally, a "fast" rendering option was added to the shaders to improve performance during interactivity when low quality rendering is acceptable.

John also re-implemented the memory manager supporting vaporgui's data cache. The new memory manager lazily allocates dynamic memory as needed (as opposed to allocating all memory up front), reducing the virtual memory footprint in most instances.

Finally, the year-end report, and an accompanying power point presentation, were written and submitted to KISTI.

Development:

John worked with CU's Sam Geen to troubleshoot and correct a number of bugs in the Adaptive Mesh Refinement (AMR) support modules. Additionally, a critical AMR access method that was performing poorly when applied to deep hierarchies was re-implemented to optimize performance. Run time on Sam's data sets dropped from hours to seconds.

Administrative:

The hiring process for a new student assistant was started. We hope to start interviewing in January.

Data Analysis & Visualization Lab Projects

File System Space Management Project

Accounting & Statistics Project

System Monitoring Project

CISL Projects

GLADE Project

Data Transfer Services Project

NWSC Planning

System Support

Data Analysis & Visualization Clusters

GLADE Storage Cluster

External Collaborations

Data Transfer Cluster