1 Introduction
This document describes objectives for VAPOR 3.x. It consolidates and replaces material from several other sources that include:
2 Summary
2.1 Usability
Each new release of VAPOR has added new capabilities to the package. Moreover, with little exception the priority of each release has been the addition of new features above all else. A side effect of increased capability has been increased complexity. As a result VAPOR has become more and more difficult to use, particularly for the uninitiated. Usability improvement goals for 3.x include:
- Lowering the barriers to getting started with VAPOR for new users
- Reducing complexity of commonly performed tasks (e.g. preparing data for vaporgui)
- Making it easier for users learn how to perform a task
- Making it easier for users to learn what VAPOR's capabilities exist
- Providing users with up-to-date documentation
Note: much of this activity involves updating and maintaining our Website. The remainder of this document limits discussion to changes to the VAPOR code base.
2.2 Extensibility
The 3.x architecture will facilitate addition of new features by VAPOR staff and 3rd party developers. Specific examples include:
- New importers of foreign data into the DataMgr (high priority)
- New (or enhanced) renderers and associated GUI control
- New data grids
2.3 Limitations
Several limitations in the 2.x implementation exist that need to be addressed to support future needs:
- Single grid: vaporgui expects all data variables in a VDC to be sampled on a single grid
- Regular grid structure: The only grid type supported is a tesselation of 2D or 3D Euclidean space by rectangles or parallelpipeds
- Single data set: only a single VDC can be loaded at one time
- Spherical data: horizontal slices of spherical data must be projected to 2D before they can be operated on
2.4 Maintainability
To facilitate long-term code maintenance a number of areas must be addressed:
- Definition of major components. E.g. what are they, what do they do, what do they not do, how do they interact with other components
- OpenGL state mgt: There is not centralized management of OpenGL state. It's very easy for state to become corrupted. Mixing of fixed function and programmable pipeline may be problematic. Code is based on pre-OpenGL3.0 api
- Code reuse: many common tasks within vaporgui have multiple implementations
- Elimination/reduction of task-level parallelism. It's not needed and unsafe
- Code cleanup: prune dead code, better document surviving code, etc.
3 Refactoring goals
3.1 Improving Usability
Aspects of the 2.x design are responsible for a number of behaviors that can be frustrating to the user. Here we identify a number of areas aimed at improving the user experience.
3.1.1 Reuse principle
The vaporgui application should strive for consistency in functionality and presentation, both internally between different tabs, for example, and with other successful, similar applications. Specific examples where improvements could be made include, but are not limited to:
- The transfer function editor and layout selectors should offer consistent capability, operation and appearance across all tabs
- Parameters taking numerical values with a known valid range should be settable with both scroll bars or typing the value.
- Terminology of common buttons/labels should be consistent. For example, the Image tab has a "fit box to current region" button, while the 2D tab has "fit to current region".
3.1.2 Simplicity principle
Many components of vaporgui have overwhelming complexity. The Flow tab is one of the primary culprits. Too many options are presented in an apparent random fashion that gives the user no clear idea on what needs to be done to perform flow visualization. Options need to presented to the user in a way that is not intimidating and helps guide the user through the steps needed to perform a task.
3.1.3 Error popups
The error handing mechanism needs to be redesigned to limit the cascade of sometimes never ending error message popups that can occur in vaporgui when an error condition is encountered.
3.1.4 Better rendering control
The current implementation does not provide explicit control over rendering. Spurious events may trigger re-renders at any time. When working with very large data this can be quite frustrating.
3.1.5 Startup delays
VDCs that contain numerous time steps and/or variables can result in significant delays then the data set is selected by vaporgui. Lazy evaluation of VDC (or foreign data import) should be used to speed data loading. Lazy evaluation of VDC has been implemented in 2.2. Now when a VDC is opened, only the first and last occurrences of each variable are identified.
3.2 Enabling Extensibility
It should be possible for any experienced C++ programmer, with a general understanding of scientific visualization, but without extensive knowledge of the VAPOR internals, to extend key VAPOR components in the following ways.
3.2.1 Data formats
It should be easy to add data readers for gridded data that conform to supported types (see Removing Limitations), but are stored in a foreign (not presently supported) file format. Data not stored in a VDC may be imported directly into vaporgui (read directly from a file), or may first be translated into a VDC using a data translator. The development of both direct data importers, and data translators should be possible by following well-documented APIs.
3.2.2 Visualizers
Adding new, or extending existing, visualizers should also be facilitated.This would require developing (refactoring) well-defined major architectural components with documented APIs. Additionally, a toolkit of GUI objects should provided to facilitate development. Elements of the toolkit might include "panel" objects such as TF editors and color selectors, as well has "scene" objects such as spatial domain selectors (e.g. axis aligned box, a line segment)
3.2.3 Shaders
OpenGL vertex and fragment shader programs are an integral component of the current direct volume renderer, and ray caster. Furthermore, in an OpenGL 3.X environment fixed functionality rendering is a deprecated capability: we will likely have to migrate all rendering to programmable shaders in the future. While new shaders can easily be loaded at run time, there is no documented interface that would allow the development of a new shader without digging into the compiled code. The interface between the DVR and Iso, and shader programs should be documented to facilitate development of new shaders.
3.2.4 Alternative UIs
The VAPOR architecture should support different configurations of VAPOR components, where different user interfaces (e.g. scripting) may be used to drive a subset of the vapor libraries. For example it should be possible to developing scripting or web based UIs that are independent of Qt.
3.2.5 Vaporgui Toolbox
Enabling developers to extend or add new visualizers would be greatly simplified with the existence of a toolbox of commonly used, well-documented components. For example, the existence of transfer function editor or region selector class object(s) would simplify the process of implementing the GUI "Tab" for a new renderer.
3.3 Removing Limitations
A number limitations exist in the 2.x code that restrict the functionality that VAPOR can provide. These limitations primary concern the VAPOR data model.
3.3.1 Generalized structured grids
The VDC and vaporgui should be extended to support generalized structured grids. In the most general case a grid point coordinate would be specifiable by functions: X(i,j,k,t), Y(i,j,k,t), Z(i,j,k,t), where i, j, and k are spatial grid indices and t indexes a time step. Furthermore, distinctly different spatial coordinate functions should be allowed for each data variable. For example, the X coordinate of the variable temp might be given by, X(i,j,k,l), while the X coordinate of the variable pressure might be given by a different function with different dimensions: X'(i',j',k',l'). Thus data sets are no longer restricted to a single grid, and supported grid types would be generalized to include structured grids.
This generalization of the VDC would allow, for example, the direct support of a large subset of CF compliant grids commonly used in ocean and atmospheric modeling.
3.3.2 Unstructured grids
The 2.x VDC provides no support whatsoever for unstructured grids: tessellations of Euclidean space by simple shapes, such as tetrahedra, in an irregular pattern. Numerical models employing less structured grids are emerging in the earth sciences and VAPOR should be prepared to handle them. Examples of such models include the MPAS, ICON, and HOMME models. As with general structured grids and spherical grids changes would be required of the external VDC format, the internal data model, as well as the visualizers in vaporgui.
3.3.3 Alternate VDC API
The 2.x VDC API has limited capability (e.g. lacks support for metadata attributes) and bears little resemblance to APIs for any of the widely used scientific data format (e.g. netCDF). Thus the VDC's suitability as a more general container for scientific data is limiting. Furthermore, substantial changes may be required to adapt a code currently using a more common data format to use the VDC. It would be desirable to offer an API that more closely resembles the capabilities and appearance of a more widely used file format (e.g. netCDF).
3.3.4 Multiple data sets
Version 2.x permits only a single data set to be active at a time, precluding the capability of performing any kind of simultaneous visualization of variables from different data sets (e.g. combining ocean and atmospheric data). It would be desirable to allow multiple data sets to be loaded in a single session, and allow simultaneous visualization of variables from two or more data sets.
3.4 Improving Maintainability
Similar to the difficulties associated with extensibility the current code base is very messy. Components and interfaces are not well defined, resulting in much legacy code that is neither robust nor easy to fix when something goes wrong.
3.4.1 OpenGL use
OpenGL entry points proliferate the code with no mechanisms to guide or enforce their use in a way that prevents state corruption. A common problem is making OpenGL changes in one segment of the code that then change behavior in an unrelated code section. Application-wide OpenGL state management and usage policies are required.
3.4.2 Code reuse
Major components of the code should be identified and isolated, specifications for their functionality and their interfaces defined and documented, and implemented. In numerous instances the same basic functionality is reimplemented in different files over and over again. For example, the 2.x code base contains multiple basic linear algebra implementations. Such instances should be identified and eliminated.
3.4.3 Task level parallelism elimination
Task level parallelism is currently employed to provided functionality that could easily be implemented with serial methods (e.g. animation and error handling). The usage of task level parallelism should be evaluated and if it's use not justified it should be replaced with more reliable, deterministic, and easily understood serial methods.
3.4.4 Code cleanup
Dead code should be identified and removed. Surviving code should be reviewed for adherence to best software engineering practices. The use of global variables should be eliminated.