Summary of Supported Data Formats

NOTE: this page is currently a bit out of date. I'll update with more detailed information as I prepare the final documentation for the Extended Flight Level Dataset.

  1. USAFR ASCII text formats
    1. C130-E/H *.ten format (2004 and previous)
    2. ARWO Software Version 15.400.21.1 (first used in 2004) - fully supported
    3. ARWO Software Version 15.400.21.4 (circa 2005) - fully supported
    4. ARWO Software Version 15.400.21.7 (circa 2006) - fully supported
    5. ARWO Software Version 15.400.22.2 (circa 2007 - 2008) - fully supported
    6. ARWO Software Version 15.400.23.0 (circa 2008 - used for Western Pacific) - fully supported
    7. ARWO Software Version 15.400.23.1 - fully supported
    8. ARWO Software Version 15.400.23.2 - fully supported
    9. ARWO Software Version 15.400.24.0 (circa 2009) - fully supported
    10. ARWO Software Version 15.400.25.0 (circa 2010-2011) - fully supported
    11. ARWO Software Version 16.0.1.3 (circa 2012)  - fully supported
    12. ARWO Software Version 16.0.2.3 (circa 2013)
  2. HRD ASCII text formats 
    1. format used circa 2005-2006 - fully supported
    2. format used circa 2007-2012 - fully supported
  3. AOC NetCDF formats 
    1. Type 1 - fully supported
    2. Type 2 - fully supported
    3. Type 3 - fully supported
    4. Notes on each year:
      1. 1998 format
      2. 1998 format
      3. 1999 format
      4. 2000 format
      5. 2001 format
      6. 2002 format
      7. 2003 format - last instance of the Type 2 variant
      8. 2004 format - fully supported
      9. 2005 format - fully supported     (missing radar altitude)
      10. 2006 format - fully supported     (missing radar altitude)
      11. 2007 format - fully supported     (missing radar altitude)
      12. 2008 format - fully supported     (missing radar altitude)
      13. 2009 format - fully supported     (missing radar altitude)
      14. 2010 format - fully supported
      15. 2011 format - fully supported
      16. 2012 format - fully supported
  4. NOAA standard tape format (mostly applicable prior to 2005) - not supported

 

Notes on AFRES ASCII formats:

From Colonel Eric Dutton (retired AFRES director):

The .min and .tsc files were binary files used by the WC-130E/H.  The .tsc file was the 10-sec data, the .min file changed for the 1-min record data (20 min hdobs) to 30-sec record data (10 min hdobs) around the 1996 or 1997 time frame.  The .SEC and .TEN have been the 1-sec and 10-sec binary data, respectively, used by the WC-130J since the beginning.  The .TXT was used by the WC-130J up through 2008 for the 10-sec text file, after that the J model used .01.txt, .10.txt, .30.txt and .60.txt for text files for 1-sec, 10-sec, 30-sec and 60-sec data, respectively.  I cannot address the .1sec, .10sec, .one, .ONE or .ten file types, although the identifiers appear to be self evident.  The .ten may be the same as the .TEN, but of the few records I have, the 10-sec binary file always had a .TEN extension. 

 

Notes on AOC NetCDF formats:

As per guidance from Barry Damiano and Neal Dorst, the various NOAA aircraft data formats are prioritized in the following order:

  •    *_cXC.nc          highest priority - QC'd, post-season processed netCDF file with modified parameters from AOC (where 'c' is 'A', 'B', 'C', etc.). These usually combine all the data for a given flight.
  •    *_cC.nc            next highest priority - QC'd, post-season processed netCDF without any modified parameters. These usually combine all the data from a given flight.
  •    *_c.nc               intermediate priority - initial netCDF file produced within a couple days of flight, no QC. There can be multiple files for each flight if they had to restart.
  •    *.dat                 lower priority - standard tape format, which is basically a copy of all the parameters in the netCDF into the historic standard tape format
  •    ASCII text file    lowest priority - the ASCII text files are the farthest removed from the original data.

From Barry Damiano (AOC Flight Director):

Prior to 2011, all missing values are indicated by a "nan" or -32767 for P3 data sets.

Prior to October 2011, the old names were used for parameters (e.g. lati1, lati2, latnvl, etc.). After this change over, the new AAMPS parameter names are used in the NetCDF files.

The new airborne data collection system (AAMPS) was put into production starting in the 2012 season.


From Sonia Otero (AOC software engineer and AAMPS developer):

This web page explains the NetCDF conventions followed by AOC with the
advent of the new airborne data collection system put in production in
2012 (AAMPS).

[http://www.eol.ucar.edu/raf/Software/netCDF.html|http://www.eol.ucar.edu/raf/Software/netCDF.html]

These conventions are likely to remain pretty stable.  If Chris Webster
(NCAR) decides to tweak things, AOC is likely to follow on, in order to
stay current with the latest tools of ncdump, ncplot, also maintained by
Chris Webster.

The new NetCDF files contain a 'Time' variable (not a global attribute),
holding the amount of seconds since the start of the flight.

The new files contain these global attributes:

// global attributes:
         :StartTime = 1320988978 ;
         :FlightDate = "11/11/2011" ;
         :TimeInterval = "05:22:58-10:38:36" ;

StartTime is the Unix epoch time (seconds since Jan. 1, 1970 00z).

During the 2012 hurricane season, it was detected that the Time variable
was incorrectly written.  That was corrected starting on Sep. 20, 2012.
  However, in previous files, one can grab the Unlimited Index variable,
whose values should coincide with those of 'Time'.

  • No labels