What is the File Converter?
Installing & Running the File Converter
Make a Configuration File
Adding and Changing Attributes
Area subsetting and reducing resolution
Converting a File
Modifying a Configuration File
Frequently Asked Questions
Description of File Types
What File Type is Mine?
How to add a new File Type to the Converter
Contact Us

This converter takes ASCII data files of earth science data (lat-lon(-depth) files of scalar values) and converts them to Unidata's NetCDF file format. The IDV uses NetCDF format files for 2D and 3D (and 4D - with time) grids. This converter uses a graphical user interface with user-entered values to describe how the input files are formatted and the extent of the output file's geographic area in the output NetCDF file.

The way the conversion process works is that first, a "configuration file" is created. This contains all the data that the program needs to read and convert the file. The configuration file is stored on disk so it can be reused, even if you exit the program. Then you "execute" that file, which means actually converting the data. The program helps you create the configuration file (just fill in the information it asks for). You then choose to execute that file and it will convert the data for you.

All sources must have longitude and latitude (2D grid); it can also have depth (3D grid) and time. And then, of course, it also has the data itself. The data can be anything, temperature, density, any scalar value.

Installing and Running the Converter

Download the file Converter.zip. Unzip it and find the file Coverter.jar in a directory (folder) called NetCDF_Creator.

You need "Java" installed on your computer, since the converter is a java program. You can download all Java software at Sun's Java website. To run the converter you only need the "JRE" or Java Runtime Environment. With the JRE on your system, you can run any java program no matter what kind of computer you are using (at least for the most common new computers.)

Once you have installed the Java JRE, and you have the Coverter.jar file, run Coverter.jar with Java. If you're running Windows simply double click the Converter.jar file to start the program.

If you're on Linux or Unix, in the appropriate directory (with Converter.jar) the following line should run the program:
java -Xmx512m -jar Converter.jar
Do not omit the final m in -Xmx512m. "512m" tells the program to use up to 512 megs of hard memory for data. If you have lots of hard memory, you can use a larger number and you may see a speed improvement.

Example Use

Here is an example of using the format converter.

The data file is a row-column file of a 3D grid in longitude-latitude-depth-variable value. Here are a few of many data rows. Note that this does not have exact column alignment; the space separation is all that is required.

115.5   47      2       2.2     2.245
115.5   47.5    2       2.18    2.184
115.5   48      2       2.18    2.184
115.5   48.5    2       2.18    2.184
115.5   49      2       2.18    2.184
115.5   49.5    2       2.026   2.063
115.5   50      2       2.026   2.063
116     20      2       2.026   2.02
116     20.5    2       2.026   2.026
116     21      2       0.001   2.0
116     21.5    2       2.114   2.4
116     22      2       2.3     2.4
116     22.5    2       2.3     2.4

Each row in one grid point in a 3D grid. Columns 4 and 5 are two different Vs velocity values at each grid point. Note the correspondence of this file format and the configuration values in the window shown below.

Make a Configuration File

The configuration file describes the format of your input ASCII data file, and may also apply geographic limits to data for the output file (subsetting).

When you start the converter program, this window will pop-up:

Click on the "Create a Configuration File" button. A new window pops up that looks something like the new picture below.
Here you enter information about your ASCII input data file.

Along the top is a series of tabs. Each tab represents the interface for an individual "file type." See File Types and what is a a File Type. Click on the appropriate tab for your file type. The row-column file we are using is a "Column File."

Enter the 8 data values which describe your file.

Adding and Changing Attributes

A NetCDF file includes metadata about the information in the file. At a minimum for IDV use, every data value must have a geographic location and optionally a time. Data units may be provided, in which case the data is more useful in the IDV. The IDV does automatic unit conversions in calculations between differing data sets, for example, but only if the units are given.

All data in a NetCDF file, including the location values, are called variables. Variables have "attributes." Attributes are metadata about the variables, such as the units for depth or velocity.

Variables have names. The IDV recognizes the special position variables with the names "lat," "lon," "depth," and "timeObs." Your data values must have a variable name as well.

Your data likely needs one or more attributes. To add a new attribute, click on the Open Attribute Window "Open Window" button." A new window titled "Attributes for List file" appears.

Click on the "Add an attribute" button. In the Variable pull down menu, select "data." For "Title of Attribute" enter, for example, "units" and for "Text of Attribute" (its value) enter for example "km", "km/s", "m/s", "meter", and so on. Then click the Add Attribute button.

So far your variable is named "var." Variable names are displayed in IDV labels. To change the name of a variable from "var" to a better name click on the "Variable namning window" button. A new window appears. Change "var" to anything you like, such as "Vsh."

In the same window, change all "Data Type" choives from double to float, unless you need more than 7 decimals of precision in your data file. Floats in place of doubles halves the final NetCDF file size.

For topography data you want to display as 3D relief, change the name of the variable for topography (elevation) from "var" to "Altitude" which is required for the IDV to make 3D relief plots.

The file converter defaults to units for depth in kilometers, latitude in degrees north ('¿½degrees_noh'¿½), longitude in degrees et ('degrees_east'), and time in 'seconds since 1970-01-01 00 UTC'. Any of these may be changed by the user, though you should take care to use units from the UD Units package which covers most anything, such as 'degrees_west.'

Adding more attributes - optional

It is very simple to add an attribute. Click on the Add an Attribute button, a window should pop up that looks like this:

Some common attributes used in NetCDF and the IDV are
units A character string that specifies the units used for the variable's data. Unidata has developed a freely-available library of routines to convert between character string and binary forms of unit specifications and to perform various useful operations on the binary forms. This library is used in some netCDF applications including the IDV. Using the recommended units syntax permits data represented in conformable units to be automatically converted to common units for arithmetic operations. For more information see UD Units.
long_name A long descriptive name. This could be used for labeling plots, for example. If a variable has no long_name attribute assigned, the variable name should be used as a default.
missing_value Your special value entered at grid points where you have no data.

First, select the variable you wish to add the attribute to. Your options for variables are: Longitude, Latitude, Depth, Time, and Data. (Global will be explained below.) If your file is temperatures in the mantle, an example of adding an attribute would be to select Data as your variable, then type in long_name for the title of the attribute, and mantle temperature as your text. Or perhaps select Data, title of units, and text of Celsius.

As an example, another way of using the attributes page is to change the units for depth. They're automatically set to kilometers, but perhaps yours are in meters. To fix this, first go to the basic attribute panel and click Remove Attribute on the button next to the depth units attribute. This will remove that attribute from the file. Then click on Add an Attribute and fill out a title of units and text of meters for the Depth variable. Click on Add Attribute, and now Depth will be read in meters instead of kilometers.

The Global attributes apply to the entire file, such as data source, time of creation, author, etc. You may want to put in the file information detailing where it came from, this is a good use of the Global attribute. You would click “Add an Attribute”, the variable will automatically be selected as Global, then enter a title of “source” and a text of “My Name created this file on August 23, 2004”. Then add that attribute and your file will automatically contain the creation data within itself.

Done with configuration

Your configuration is complete. Click on Save Configuration As button. A file browser will pop up and ask you for a filename for a configuration file. The extension ".conf" will automatically be appended.

If you want to stop creating a configuration file and don't want to save the one you're working on, click on 'Exit File Creator'.

Subsetting data areas, and reducing resolution of grids - Optional

The file converter, besides converting data into netcdf format, will also allow you to subset the data area and reduce its resolution. click on the open window button next to the words modify file window. when you do, a window should appear that looks like this:

The latitude and longitude limits are the widest possible range of values to keep from accidentally removing any data you doni't want removed. You may change any or all of these values, which will tell the program what limits you want to put on latitude, longitude, depth and time. The depth and time limit selection can be found on the second panel (click on the tab "Time/Depth" to get to this panel).

The other modification is to lower the resolution. The lat/lon resolution is kept separate from the depth/time resolution (you could lower the resolution on the lat/lon grid by a factor of 2, and keep all of the depth and time resolution as is). The number you enter in the "resolution" field represents the factor by which you wish to reduce the resolution. A factor of 1 will leave the resolution unchanged (and is the default value). A factor of 2 will skip every other datum on the lat/lon grid (or depth/time, if you're modifying the resolution in that pane). You can enter any number, 1 or greater. since anything less than one would be a meaningless resolution factor, the program will not accept negatives or zero.

To exit the modification window press the 'exit file modifier' button in either pane. This will automatically save the changes you made and return you to the file configuration editor.

Executing the program - Converting a File

To "execute the file" you just created -- make the conversion, exit the configuration file creator if open. In the original screen, click "Execute a Configuration File. Then use the file browser that pops up to select the configuration file you just created (or any other configuration file you would like to execute).

Once you select the file, the program will begin converting the data. This may take a while. A bar will pop up alerting you that the file is being converted, it will disappear when the process is over.

No message appears if the process succeeds. Look for the Netcdf file you named.

Modifying a Configuration File

You can choose to modify a previous configuration file rather than create it from scratch. Simply click on “Modify a Configuration File”, then select the file you would like to modify. The file configuration creator will open with your file loaded into the appropriate tab.

Improvements

We plan to revise the converter to allow use of more than one data variable, which is now the case. A NetCDF file can have many data parameters for one grid.

We plan to use more time units such as "yyyy-MM-dd HH:mm:ss"

Back to Top

( NGDC topography ascii format files are row-column files, each row for one latitude. This is a "List File" as far as the Converter is converned. For NGDC ascii topography data the starting latitude is the highest one, and the starting longitude is the leftmost (western or lowest one) files. Spacing between latitudes is -0.03333333 for 2 minute spacing data; -0.06666666 for 4 minute data, etc. Note the negative. Spacing between longs is the same size but positive (increasing eastward) for the NGDC ascii topography data. For that data enter NO for "does latitude change faster than longitude." )