The Interactive ASCII to NetCDF file converter

What is the File Converter?
Installing & Running the File Converter
Make a Configuration File
Adding and Changing Attributes
Area subsetting and reducing resolution
Converting a File
Modifying a Configuration File
Frequently Asked Questions
Description of File Types
What File Type is Mine?
How to add a new File Type to the Converter
Contact Us

This converter takes single-parameter, 2D, ASCII data files of earth science data (lat-lon grids of scalar values) and converts them to Unidata's NetCDF file format. The intent is to provide mediation between scientists who want to use their ASCII data files in the GEON IDV, and the GEON IDV, which uses NetCDF formatted files This converter uses a graphical user interface with user-entered values to describe how the input files are formatted and the extent of the output file's geographic area in the output NetCDF file.

(There are also UNAVCO ASCII to NetCDF converters to include conversion in automatic creation of ASCII files, and for post-conversion, for mutltiple parameters at each point in 3D and 4D grids, that can handle multiple parameters per file. )

NetCDF format files store metadata in the same file as the data, making every NetCDF file self-describing. NetCDF (network Common Data Form) is both a way of formating data and software interface for array-oriented data access and a library that provides an implementation of the interface. NetCDF is a machine-independent format. See more at The NetCDF Format for the IDV.

This conversion program is primarily designed to help investigators and students with solid earth science data, using the UNAVCO's GEON IDV implementation of Unidata's IDV, an earth-science data analysis and visualization (2D and 3D) program. The IDV usually requires data to be mapped on the earth's surface, that is, each data value be associated with a latitude-longitude position, optionally with time and depth or elevation.

The way the conversion process works is that first, a "configuration file" is created. This contains all the data that the program needs to read and convert the file. The configuration file is stored on disk so it can be reused, even if you exit the program. Then you "execute" that file, which means actually converting the data. The program helps you create the configuration file (just fill in the information it asks for). You then choose to execute that file and it will convert the data for you.

Since the results are for IDV use, it only accepts certain data. All data must have longitude and latitude, it can also have depth and time. And then, of course, it also has the data itself. The data can be anything, temperature, density, any scalar value.

Installing and Running the Converter

If everything works as usual, you will be able to download and run and use the converter program in less than 10 minutes, unless you have a very slow internet connection.

Download the file Converter.zip. (On Netscape, right click on this link and choose "Save link target as." On Windows, use save link as).

The first step in running the file converter is unpacking it. In Linux or Unix you should be able to use the "unzip" command to do this. In Windows, right click on the file (Converter.zip) and then select "Extract All"; on Linux use the command "unzip Converter.zip" in the directory where that file is found.

No matter the operating system, you need "Java" installed on your computer, since the converter is a java program. You can download all Java software at Sun's Java website. To run the converter you only need the "JRE" or Java Runtime Environment. With the JRE on your system, you can run any java program no matter what kind of computer you are using (at least for the most common new computers.)

Once you have installed the Java JRE, and you have downloaded and unzipped the Coverter.zip file, if you're running Windows simply double click the Converter.jar file to start the program.

If you're on Linux or Unix, in the appropriate directory (with Converter.jar) the following line should run the program: java -Xmx512m -jar Converter.jar. Do not omit the final m in -Xmx512m. The 512m part tells the program to use up to 512 megs of hard memory for data. If you have lots of hard memory, you can use a larger number and you may see a speed improvement.

Make a Configuration File

The configuration file describes the format of your input ASCII data file, and may also apply geographic limits to data for the output file (subsetting).

When you start the Converter program, a window will pop-up that looks like this:

Click on the "Create a Configuration File" button. A new window pops up that looks like this.
Here you enter information about the input data file.

Along the top is a series of tabs. Each tab represents the interface for an individual File Type. Click on the appropriate tab for your file type. For example, NGDC topography ascii format files are row-column files, each row for one latitude. This is a "List File" as far as the Converter is converned.

Add in the data that describes your file. Remember that "west" longitudes are negative, as are south latitudes.

For NGDC ascii topography data the starting latitude is the highest one, and the starting longitude is the leftmost (western or lowest one) files. Spacing between latitudes is -0.03333333 for 2 minute spacing data; -0.06666666 for 4 minute data, etc. Note the negative. Spacing between longs is the same size but positive (increasing eastward) for the NGDC ascii topography data. For that data enter NO for "does latitude change faster than longitude."

Since some files can have depth and time data and some do not, the zeros are the default values for depth and time. If you leave the zeros there, it will assume you have neither depth nor time in your file.

Adding and Changing Attributes

Data in a NetCDF are called "variables." Variables have attributes. "Attributes" are metadata about the variables, such as the units for the depth.

Your data likely needs one or more attributes. To add a new attribute, click on Open Attribute Window "Open Window" button." A new window titled "Attributes for List file" appears. Click on the "Add an attribute" button. In the Variable pull down menu, select "data." For "Title of Attribute" enter, for example, "units" and for "Text of Attribute" (its value) enter for example "kilometer". Ten click the Add Attribute button.

So far your variable is named "var." Variable names are saved in NetCDF files and displayed in IDV labels. To change the name of a variable from "var" to Click on the "Variable namning window" button. A new window appears. Change "var" to anything you like, such as "depth to moho."

In the same window you can change all real data types from double to float, unless you need more than 7 decimals of precision in your data file. This halves the final NetCDF file size.

For topography data you want to display as 3D relief, change the name of the variable for topography (elevation) from "var" to "Altitude" which is required for the IDV to make 3D relief plots.

Click on "Save configuration file as" and enter a new file name.

You configuration file is complete. Click on Save Configuration As button. Click on "Execute a configuration file" to make the conversion.

More details

Some common attributes used in NetCDF and the IDV are
units A character string that specifies the units used for the variable's data. Unidata has developed a freely-available library of routines to convert between character string and binary forms of unit specifications and to perform various useful operations on the binary forms. This library is used in some netCDF applications including the IDV. Using the recommended units syntax permits data represented in conformable units to be automatically converted to common units for arithmetic operations. For more information see UD Units.
long_name A long descriptive name. This could be used for labeling plots, for example. If a variable has no long_name attribute assigned, the variable name should be used as a default.
missing_value Your special value entered at grid points where you have no data.

Take care changing anything having to do with locations, variable lat, lon, and depth. These are attributes used by all files, included automatically in the configuration file.

The file converter defaults to units for depth in kilometers, latitude in degrees north ('¿½degrees_nort¿½), longitude in degrees eas('degrees_east'), and time in 'seconds since 1970-01-01 00 UTC'. Any of these may be changed by the user, though you should take care to use units from the UD Units package which covers most anything, such as 'degrees_west.'

It is perfectly acceptable to add on as many of your own attributes as you want. To do so, click on the Open Attribute Window button at the bottom of the configuration page. A window should pop up that looks like this:

(If you've already modified this file (see below for modifying a configuration file), there may be different attributes there.) If you want to revert to the default attributes, click on Set Attributes to Default, but be careful, this will erase all the changes youve made, permanently. If you see that there are attributes for time and depth when your file doesnt have those variables, don’t worry, they will automatically be removed when the file is created (though you can manually remove them yourself if you wish to). To exit out of the screen simply click on Return to Config File Creator.

It is very simple to add an attribute. Click on the Add an Attribute button, a window should pop up that looks like this:

First, select the variable you wish to add the attribute to. Your options for variables are: Longitude, Latitude, Depth, Time, and Data. (Global will be explained below.) If your file is temperatures in the mantle, an example of adding an attribute would be to select Data as your variable, then type in long_name for the title of the attribute, and mantle temperature as your text. Or perhaps select Data, title of units, and text of Celsius.

As an example, another way of using the attributes page is to change the units for depth. They're automatically set to kilometers, but perhaps yours are in meters. To fix this, first go to the basic attribute panel and click Remove Attribute on the button next to the depth units attribute. This will remove that attribute from the file. Then click on Add an Attribute and fill out a title of units and text of meters for the Depth variable. Click on Add Attribute, and now Depth will be read in meters instead of kilometers.

The Global attributes apply to the entire file, such as data source, time of creation, author, etc. You may want to put in the file information detailing where it came from, this is a good use of the Global attribute. You would click “Add an Attribute”, the variable will automatically be selected as Global, then enter a title of “source” and a text of “My Name created this file on August 23, 2004”. Then add that attribute and your file will automatically contain the creation data within itself.

Subsetting data areas, and reducing resolution of grids

The file converter, besides converting data into netcdf format, will also allow you to subset the data area and reduce its resolution. click on the open window button next to the words modify file window. when you do, a window should appear that looks like this:

The latitude and longitude limits are the widest possible range of values to keep from accidentally removing any data you doni't want removed. You may change any or all of these values, which will tell the program what limits you want to put on latitude, longitude, depth and time. The depth and time limit selection can be found on the second panel (click on the tab "Time/Depth" to get to this panel).

The other modification is to lower the resolution. The lat/lon resolution is kept separate from the depth/time resolution (you could lower the resolution on the lat/lon grid by a factor of 2, and keep all of the depth and time resolution as is). The number you enter in the "resolution" field represents the factor by which you wish to reduce the resolution. A factor of 1 will leave the resolution unchanged (and is the default value). A factor of 2 will skip every other datum on the lat/lon grid (or depth/time, if you're modifying the resolution in that pane). You can enter any number, 1 or greater. since anything less than one would be a meaningless resolution factor, the program will not accept negatives or zero.

To exit the modification window press the 'exit file modifier' button in either pane. This will automatically save the changes you made and return you to the file configuration editor.

Executing the program - Converting a File

Once you've filled out the data (for an explanation of each file type and what the data means, go here) click on “Save Configuration File As”, if you want to stop creating a configuration file and don't want to save the one you're working on, click on 'Exit File Creator'.

If you do save your file, a file browser will pop up and ask you for the file name. The extension “.conf” will automatically be appended onto your file choice. When the file browser exits, the file has been saved and you're returned to the configuration file creator. To execute the file you just created, exit the configuration file creator. The original screen has a button labeled “Execute a Configuration File”, click on that. Then use the file browser that pops up to select the configuration file you just created (or any other configuration file you would like to execute).

Once you select the file, the program will begin converting the data. This may take some time, please be patient. A bar will pop up alerting you that the file is being converted, it will disappear when the process is over. To leave the program click the “Exit” button at the bottom of the original window.

Modifying a Configuration File

You can choose to modify a previous configuration file rather than create it from scratch. Simply click on “Modify a Configuration File”, then select the file you would like to modify. The file configuration creator will open with your file loaded into the appropriate tab.

Contact

Email Stuart Wier at wier@unavco.org.

Back to Top