Every data value in the IDV must be mapped to latitude and longitude, with optional depth or altitude, and
optional time.
A very common use of the IDV is working with regular grids in latitude - longitude - depth.
Continuous fields are generally represented by grids.
Regular grids have a single list of latitudes on a north-south axis, and
a single list of longitudes along
an east west axis.
The spacings in latitude and longitude are uniform.
The grid locations are the combinations of those points along the axis, like a chess board.
For example you could have longitudes of 0.0, 1.0, 2.0, and 3.0, and
latitudes of 40.0, 41.0, 42.0, and 43.0. This makes a regular grid of 16
locations, at (0.0E 40.0N), (1.0E 40.0N), ... (3.0E 43.0N).
There is a list of 16 values for each parameter, one value
for each parameter at each location.
One depth value applies to the grid (if depth is appropriate).
A 3D grid is in effect a collection of these 2D grids,
one 2D lat-lon grid for each depth in the 3D grid.
The GIAG Format - UNAVCO's GEON IDV ASCII Grid Format Description
GIAG (GEON IDV ASCII Grid) is an ASCII format is for these kinds of grids.
Several variables (parameters, fields) can be stored in one grid, with
one or more parameter values at each grid location (a "missing value" is available for gaps in data).
The goal of the GIAP ASCII format is to simplifiy
converting existing research data for the GEON IDV.
This format lets investigators quickly convert their existing data in row and column format
to the NetCDF format used by the IDV for gridded data,
and save them the trouble of writing code to create
NetCDF data files directly.
This document describes how to add headers to your ASCII files with columns
of lat-lon-variable ASCII data so that the GIAG to NetCDF converter can handle it.
UNAVCO makes a converter available to convert GIAG files to NetCDF. You can
download this converter program and use it yourself to make NetCDF fiules that work in the GEON IDV.
It is described in the next section on this page.
A GIAG formatted file is for one 2D grid of one or more variable, on a lat-lon grid; depth and time also
may be specified. Multiple
GIAG files can be used for multiple depths, making a set of files used as input to the UNAVCO GIAG converter,
to make one NetCDF file of a 3D grid of one or more variable parameters on that 3D lat-lon-depth grid.
A coming improvement may allow multiple time steps in the grids.
Converting 40 megabytes of ascii data files in the GIAG format to NetCDF will reduce storage space
by about a factor of eight.
NetCDF has many advantages. Read more about NetCDF for the GEON IDV.
********************* Quick Guide to GIAG file format **************************
filename of form sourcename.depthvalue.timevalue, like this filename aT_intp.2616.450
file contents: lines beginning with "#" are header info; i
each header line starts with keyword, one of "source", "depth", "time",
"missing_value", and "var1", "var2" etc for each variable
example: DO NOT INCLUDE MATTER IN square brackets [ ]
# source Mantle temperature and Vp model 22B
# depth 2616 kilometer down
# time -450 Myr
# missing_value -999
# var1 Trcb "temperature frac of core boundary temp, 1 to 0" none
[shortname, long name, unit name]
# var2 T "mantle temperature" kelvin
# var3 Pv "P wave velocity" km s-1
# lon lat var1 var2 var3
[this header line labels the columns]
0.000 -8.700 .682 4568.2 7.546
1.000 -8.700 .731 4573.1 7.556
2.000 -8.700 .756 4575.6 7.499
3.000 -8.700 .773 4577.3 7.887
4.000 -8.700 .790 4579.0 7.678
and so on to create a 2D long-lat grid of values at depth 2616 for time 450 mybp.
********************* End of Quick Guide to GIAG file format **************************
Data File Naming, and Data Sets to make a 3D Grid
The data files are ASCII text files, and must be named something like dataID.050.120.
The period "." is used to separate the file names into 3 parts.
dataID: a name for the entire data set, composed of characters and numbers
with no spaces, ending with "." This name is the same for ALL files in the entire
collection of files.
The depth 050 is a sample depth indicator, normally the depth number for data in this particular file, in whatever depth units you use.
All files of the same depth have the same depth indicator value.
The depth indicator is really just an ordering number for file processing; the real depth value comes from
the header line described below. It is handy to use a depth value here to help recognize files by name.
Be sure that all depth indicator numbers are sorted into the order wanted in your folder or directory: zero pad to fill.
"50" follows "100" in a directory list, so use "050" in this case.
The time 120 is a time indicator. All files of the same time have the same time indicator. At the moment the
converter program only processes files for the same time indicator. In future multiple times will be
processed. Zero pad time values like the depth values to give proper order in a folder or directory.
The time values are actually found in each file headers but the file name parts are used to sort the data in correct depth and time order.
Each file contains one 2D lon-lat grid of values for one time and one depth.
A set of files for four depths for one time is like:
aPvel_interpd.10.10
aPvel_interpd.20.10
aPvel_interpd.30.10
aPvel_interpd.40.10
In this example the time value is the same, "10". If no time applies to your data just use some number like 1. The converter will make one 3D grid at one (or no) time.
File Contents and Header Information
The data columns are preceeded by several lines of header or metadata information.
All headers lines begin with the "#" character. Each header line then
has a space, a keyword and data or information, as described here for each header line.
Example of a complete header:
# source Mantle temperature model 22B
# depth 2616 kilometer down
# time -450 Myr
# missing_value 100000.
# var1 Trcb "temperature fraction of core boundary temp, 1 to 0" none
# var2 T "mantle temperature" kelvin
# var3 Pv "P wave velocity" km s-1
# lon lat var1 var2 var3
Every file of the same data set has the same header lines, except for the depth and time value.
Source header
Descriptive source of the data, with keyword "source"
example:
# source mantle stress model A05
Whatever follows " source " to the end of line is used as the description. Quotations are not required.
Depth header
keyword " depth "; numerical depth value; units of depth; keyword for if positive Z is "down" or "up"
(see below for accepted units' names)
example
# depth 23.654 kilometer down
Note that the surface of the earth is at 0 depth.
Time header [optional]
keyword " time ", its numeric value, and units
example
# time -165 Myear
Ma or Myr or Myear is the unit for million years. Use negative values for "years before
present," so the IDV shows time animations loops in the normal sense.
Use positive time values if you have a sequence starting at some arbitraty
times. There are UD Units for actual times too, precise to any degree of
accuracy you want; see Units below.
About time, the IDV plots data from one file with the same times in one display. If all times in your
file are the same, there is one display.
Missing value header [optional]
The number used in your columns of variable data to indicate no data value is known at that grid point.
keyword " missing_value "; the numerical value
example:
# missing_value -9999
The IDV will not plot data which is the missing value, or use it in
computations. If you do not use a missing value number, omit this line entirely.
Parameter variable metadata header lines
One header line for each variable:
The keywords are " var1 ", " var2 ", etc., followed by
the short name, long name, and units name or abbreviation.
The short name is a character string, with no spaces. Fewer than about 10 characters are recommended.
the "short name" is in the IDV Displays legends.
The long name is in quotes. The "long name" will appear as the variable name in the IDV "Fields" panel;
Unit name is last. enter "none" if no unit applies.
example:
# var1 SvrDG "Shear wave velocity relative to 1D model DG" kilometer/second
The UNAVCO web page
NetCDF Data Format
has a list of "Variable names for the GEON IDV" that the
GEON IDV recognizes for special plots, such as focal mechanisms.
These variable names, such as ve and eps2, should be used in the ASCII files as the "short name"
where appropriate.
Column label header line
Last, the header line that labels columns to give column order by column identifiers, such as
# lon lat var1 var2 var3
or
# lat lon var1
The first column (latitude or longitude data) should vary fastest.
Data values
The data is in space-separated columns of latitude, longitude, and variable data values, as shown below.
The actual data are in columns separated by white space:
example for lat lon var1 var2 var3:
89.0 0.0 45.77 65.89 0.087
88.0 0.0 45.77 65.89 0.087
87.0 0.0 45.77 65.59 0.087
86.0 0.0 45.77 65.99 0.037
85.0 0.0 45.55 66.07 0.098
...
89.0 1.0 45.77 62.89 0.087
88.0 1.0 45.47 64.89 0.047
87.0 1.0 45.77 63.89 0.057
86.0 1.0 45.77 67.89 0.097
85.0 1.0 45.55 65.67 0.068
...
89.0 180.0 43.77 69.89 0.047
88.0 180.0 42.77 66.89 0.037
87.0 180.0 42.77 67.89 0.027
86.0 180.0 48.77 61.89 0.097
85.0 180.0 49.55 64.67 0.068
The first coordinate, either lat or lon, must vary fastest.
Do not supply values at the poles (latitude +/- 90.000), or make them all the missing value.
The longitudes and latitudes are in geographic degrees. The IDV uses positive east from 0 longitude,
positive up from 0 degrees latitude.
Negative values, such as -105.342 for Denver's longitude, are useable.
All variable values can be in decimal (123.45) or in scientific notation (1.2345e+02; -3e-9).
The file must end on the last data line. Do not have any extra lines.
Units and unit names
For accepted variable units and names, use the UD Units conventions.
UD Units is a very complete software implementation for recognizing SI and
related units. The IDV uses UD Units. See the list of recognized unit names in
http://my.unidata.ucar.edu/content/software/udunits/udunits.txt .
The UD Units home page is
http://my.unidata.ucar.edu/content/software/udunits; see also
http://my.unidata.ucar.edu/content/software/netcdf/docs/netcdf.html#Units.
For SI units, see also
http://physics.nist.gov/cuu/Units/index.html, and
http://physics.nist.gov/cuu/Units/units.html.
When the IDV is used to make computations involving two or more data sets which
have different units, the unit conversions are automatically done for you, if
all the data has units with a recognized name.
Examples of unit names recognized by UD Units:
1300 meter or 1300 m
250 kilometer or 250 km
12 sec, 12 s
-165 Ma, -165 Myr, -165 Myear
Celsius [not C which means Coulomb]; kelvin
gram [not g which is accel of gravity], kilogram, kg does mean kilogram
degF
17.7 meter second-1, 17 m/s; 17 m s-1
17.7 km/s, 17.7 km s-1
9.80665 meter2 second-2
10 kilogram.meters/seconds2
10 kg-m/sec2
10 kg m/s2
(PI radian)2
100 rpm
geopotential meters
24598 seconds since 1992-12-31 12:34:0.1 -7:00
For pure numbers or unitless data omit the unit name or enter "none."
For percentages you may omit a unit name, or enter "none."
Or use unit name "percent" and in that case use data value 1.0 for 1 %, and
calculations using the values in IDV will multiply your percent values by 0.01
where appropriate.
UNAVCO GIAG to NetCDF Converter
The format converter program to convert GIAG ASCII files can be downloaded here.
You need Java to run the converter, and to run the IDV.
If you have installled the IDV, you have Java.
Linux Format Converter -
For the Linux GIAG converter, download GIAGconverter.jar
and ncgen for Linux into the same directory.
Make sure ncgen has execute privilege.
To run the converter, in its directory do a command like
java -Xmx512m -jar GIAGconverter.jar /fullpath/mydatasetname.22.333
/fullpath/mydatasetname.22.333 stands for the complete path and filename for one file, from all your
GIAG format files. The data files must be in one directory
but need not be in the same directory as the converter program.
(If java is not found, add the location of java to your PATH environment variable, or provide the
full path for the java program, such as /usr/java/java1.4.09/bin/java/).
After the program runs you will find in the directory with your GIAG files two new files,
such as mydatasetname333.cdl and mydatasetname333.nc. The "nc" file will work in the GEON IDV.
Windows Converter -
Download and use the Java program GIAGconverter.jar described above for Linux; since
it is a Java program it runs on Windows as well.
You will need to obtain the NetCDF utility program ncgen for Windows from the
NetCDF package.
Make sure ncgen is in the same folder as the converter program.
Solaris Sun systems -
Download and use the Java program GIAGconverter.jar described above for Linux; since
it is a Java program it runs on Solaris as well.
You will need to obtain the NetCDF utility program ncgen for Solaris from the
NetCDF package.
Make sure ncgen is in the same directory as the converter program.
unavco.org).