UTLS-OZONE Suggested Metadata Standards

This is a draft document, and will change with time.



In order to help the automatic cataloguing of data files, to ensure the consistency of the data archive, and to extend the usability of the data archive it is important to adopt certain metadata standards. This means that names of institutes, names of compounds, time labels etc. are applied consistently both within files of similar type and across the data archive as a whole. By adopting the NASA-Ames format for Data Exchange as the format for field observation data certain metadata standards are imposed on UTLS-OZONE. On top of these imposed standards other standards can be chosen. For instance in ACSOE the use of KEYWORDS introduced consistent metadata into the files. This document makes explicit the metadata standards that are part of the NASA-Ames file format and suggests other standards that could be adopted for UTLS-OZONE.

This document is separate to the description of the file format: the meta data standards can be applied to file formats other than NASA Ames. This document has been written with NASA Ames Format for Data Exchange in mind however, and so makes most sense if the reader is already familiar with the NASA Ames format. A short introduction to the NASA Ames format can be found here.

Metadata from the NASA-Ames Format for Data Exchange

The definition of the NASA-Ames Format, and in particular the header, automatically defines some of the metadata content, and in some cases the metadata format. The table outlines the content, and where defined, the format for metadata that is part of the NASA-Ames format Standard.

MetadataNASA-Ames imposed FormatUTLS-OZONE FormatExample
Originator of fileSurname, Other names. One line less than 132 charactersSurname, InitialsBloggs, J.
Originators AddressOne line less than 132 charactersAgreed Postal Address
Source NameOne line less than 132 charactersPlatform / Instrumentc-130 / UV meter
Mission NameOne line less than 132 charactersProject / UTLS-OZONEACTO / UTLS-OZONE
Start date of observationsyyyy mm ddyyyy mm dd1999 01 01
Date of data reductionyyyy mm ddyyyy mm dd1999 01 01
Variable Names Name (units)UV Flux (W/m2)
Missing valuesLarger than any good valueLarger than any good value999.999

There are no conventions placed on variable names by the standard. For UTLS-OZONE it is suggested that an agreed set of variable names will be used, to ensure consistency across data files, and in particular files from different instruments which measure the same quantity.

Time stamps

Many of the instruments have time as the independent variable. Where possible it is suggested that this time is measured in seconds from 0UTC on the start date and labelled Seconds from 0UTC on start date. Other time units may be used if necessary, but please check with the BADC to get a name that is consistent with usage across the archive.

Where data has been averaged or collected over a time interval then the centre of the averaging or collection time should be given in the file. If this is not possible then this should be made clear in the normal comments section of the file.

Error Flags

It is suggested that all observational data submitted to the archive should have an indication of the uncertainty limits of the individual observations. Where possible a quantitative error (absolute or relative) should be given. Failing this use the qualitative error flags that have been adopted by the MRF and used during ACSOE.

1Reduced quality
2Present, but likely to be wrong or instrument not in measurement mode
3Completely invalid or missing data

It is suggested that a description of these flags be included as part of the normal comments in the NASA-Ames file. The description can be expanded upon where necessary, for instance, give the reason for thinking the data is of reduced quality, or the algorithm by which the quality of the data has been determined.

Keywords used for data management

In previous campaigns, such as ACSOE, keywords have been included as part of the comments in the file header. For UTLS-OZONE there are two compulsory keywords: the e-mail address of the originator and the file version. Other keywords are optional. Keywords can easily lead to inconsistencies in the dataset if not used correctly and so it is better to omit the optional keywords if you cannot guarantee their correct use. The keywords should be included in the normal comments section of the file.

THIS-FILE-NAMEoptional (omit)
THIS-FILE-ENDS1999 01 02use if different from start date

At some time the BADC may take over the implementation of version number recording and updating. Until they do you should try and keep track of your own version numbers.

Other Information

Other information can be given in the normal comments section of the file. The normal comments section can be used to clarify the instrument type or version, its operation mode, retrieval algorithms and the nature of any averaging that has been done. Normal comments need not be verbose, but can used to supply information that will greatly enhance the integrity and usability of the data set. The following list is suggested for possible inclusion in the normal comments section of the file:

The extra information can be included in a keyword like way, for instance using the keyword FLIGHT-NUMBER, but there are no plans as yet to use these for cataloguing the data and so their format is much less constrained.

Special Comments

NASA-Ames format for data exchange provides a section for special comments. These can be used to describe any special problems or circumstances associated with the data. If the file is resubmitted to the archive then the special comments should outline any changes from the previous version.