URGENT Air Data Files: Structure and Format


Contents

  1. Foreword to data providers
  2. Data and metadata
  3. Storage and file structure
  4. Format


1. Foreword to data providers

Data providers, please note that, unless the nature of the data make it impossible, files submitted to BADC in the framework of the URGENT programme (Air Data) must fall into one of the three categories described in this note (Numbers, Images or Software) and that all data must be accompanied by relevant information (metadata.). Data and metadata should meet the specifications formulated below, in terms of storage, file names and format. In particular, data suppliers unfamiliar with NASA Ames Format should definitely consult Gaines & Hipskind, 1998 BEFORE formatting their data (see also Section 4.1.1 below).

2. Data and metadata

URGENT air data held at the BADC, whether issued from observation or produced by a model, are of three types: All forms of data are accompanied by relevant information (or metadata) needed to understand or interpret the data. Metadata are stored as text.

Data may exist in the form of NASA Ames formatted fields and have an image representation as well. In such a case, there is a separate metadata set for each version of the data (numerical and graphical).

3. Storage and file structure

Please note that information on file names is provided in a separate document (see URGENT File Names).

3.1 Numbers

Files including NASA Ames formatted fields are divided into a header containing the metadata and a body containing the data themselves.

3.2 Images

Pictures of the Earth surface are stored in binary files. Every image file is accompanied by a separate text file containing the corresponding metadata and located in the same directory. If the same images are stored in several formats, the image files share the same metadata file.

3.3 Software

Files containing software may include the metadata as a commented header. Alternatively, metadata can be supplied in a separate file.

4. Format

4.1 Numbers

Fields of numbers are formatted according to the NASA Ames specifications. Several documents are available at BADC to assist data providers in formatting data and interpreting possible error messages returned by the format checking program. Users are also advised to refer to this documentation to fully understand the data. In addition to NASA Ames rules, metadata in the framework of the URGENT programme also obey a small number of specific rules.

4.1.1 NASA Ames general rules

Please refer to the BADC NASA Ames Web page.

4.1.2 URGENT specific rules

Specific rules apply to URGENT data presentation. They regard the spatiotemporal units and the comment fields in the metadata.

A. Spatiotemporal independent variables

a). System of reference and units

For reasons of uniformity and compatibility with graphic software, participants to the URGENT Thematic Programme have agreed on a common convention regarding the expression of time and the three components of space, whether appearing among the independent or among the auxiliary variables, or in the comment fields.

Time should be expressed as Universal Time elapsed from the 1st of January 1901 at zero hour (i.e. the 31st of December 1900 at midnight), in days.
A spatial location is defined by its (longitude, latitude, altitude) coordinates.
Longitude is expressed in degrees (decimal value) from -360 to 360, starting from Greenwich Meridian eastwards. The overlap allows to avoid discontinuities when crossing the 0o meridian.
Latitude is expressed in degrees (decimal value) from -90 (South Pole) to 90 (North Pole).
Altitude is expressed in metres above sea level.

Examples: the 15th of June 1999 at 3h 37min p.m. (Greenwich time) is expressed as
(74 x 365 + 24 x 366 + 165) days + 15 hours + 37 minutes = 35959.650694 days;
35o 20' 46'' West can be expressed (in decimal degrees) as -35.3461 or 324.6539;
20o 12' South is expressed (in decimal degrees) as –20.2;
the altitude of 5.6 km above sea level is expressed (in metres) as 5600;
the altitude of 10 m above Cambridge Botanical Garden (elevation: 12 m) is expressed as 22.

b). Remark on constant intervals

The choice of a specific unit for any of the independent variables is restrictive, preventing the utilisation of the NASA Ames provision for a constant interval (non-zero DX) when the decimal equivalent of this interval in the selected unit is unlimited.

For example, if observations or calculations are performed at a regular time interval of 1 minute, since the time unit must be the day, DX cannot be set to a number with a limited decimal development (Delta t = 0.000694444... day). The only way to turn around this obstacle is to set the value of DX to 0 (convention for a non-uniform interval) and include a line in one of the comment fields specifying the exact value of the (uniform) time interval. In the above example, this line could be Regular time step of 1 minute (i.e., 0.000694444... days).

When the interval is constant and its decimal writing is limited, DX can of course be set to the interval value. In this case, no comment is needed.

For example, if the considered independent variable is time, DX = 0.00625 for Delta t = 9 minutes (= 0.00625 day), DX = 1.5 for Delta t = 36 hours (= 1.5 day), etc.

When the interval is not constant, DX must be set to 0 (no comment needed).

B. Mandatory comment fields

NASA Ames format makes provision for two comment fields (normal and special) structured in identical ways (i.e., 1st record: number of comment lines, including 1st record; subsequent records: actual comments). In the case of URGENT data files, some comments are mandatory. They may appear in either the normal or special comments, depending on whether they apply to the entire data set or only to the considered data file (see NASA Ames general rules for the difference between the two sets of comments). These comments are:

and This geographical bounding box may of course be completed by the altitude range where relevant. Regarding longitude, latitude and altitude units, please refer to Spatiotemporal independent variables.

4.2 Images

4.2.1 Image file format

Photographic images of the Earth surface are provided in usual digital formats used for pictures (JPEG, GIF, ...).

4.2.2 Metadata file format

The format adopted for metadata associated to images is inspired from the NASA Ames standard for fields of numbers. Each metadata file includes the following lines in the following order. (The nomenclature is borrowed from the NASA Ames Standard.) Each line must not be more than 132 characters long.

NLHEAD Total number of lines in metadata file (NLHEAD=NSCOML+NNCOML+9).
ONAME List of authors (Lastname, Firstname) separated by an arbitrary character (for example, a hyphen or a semi-colon).
ORG Affiliation (university, institute, etc). May include address and phone/fax number(s).
SNAME Source of visual data (e.g. aircraft name, instrument name, name of technique used, wavelength channel).
MNAME Programme and project. Example: URGENT Project GST/02/2225.
IVOL NVOL FNAME NVOL = Total number of files belonging to the considered set of images.
IVOL = Number of the considered image file in the above series.
FNAME = Name of the latter, including file name extension.
See Note 1 below.
DATE RDATE DATE = Date when picture was taken.
RDATE = Date when picture was lastly updated.
Dates obey the format YYYY MM DD (1 blank space between the year, month and day fields). If the first digit of MM or DD is 0, it may be replaced by a blank but its space may not be deleted.
See Note 1 below.
NSCOML Number of special comment lines (can be 0).
SCOM(k) Special comment lines, if any (k=1,...,NSCOML unless NSCOML=0). Special comments are relative to the considered image only.
See Notes 2 & 3 below.
NNCOML Number of normal comment lines (can be 0).
NCOM(k) Normal comment lines, if any (k=1,...,NNCOML unless NNCOML=0). Normal comments are relative to the entire set of images.
See Notes 2 & 3 below.

Notes

1. There may be any number of blank spaces between IVOL & NVOL, NVOL & FNAME and DATE & RDATE.

2. Comments should include the following:

3. The following comment lines are mandatory and may appear in either the normal or special comment fields depending on whether they apply to the entire set of images or only the considered image file.

4.3 Software

Software may be provided in any format chosen by the data provider (e.g. source code, executable version). Metadata should be supplied as text.