J. A. Kettleborough, L. J. Gray, and S. R. Williams. British Atmospheric Data Centre, Rutherford Appleton Laboratory.

• UTLS-OZONE funds a diverse set of projects encompassing field observations, data analysis, model studies and chemical kinetics studies. The data requirements of the different types of project are clearly different.
• Field observation projects will measure a large range of chemical and meteorological parameters from a range of platforms such as the MRF C-130 aircraft, the Egrett aircraft, balloon based observations, and ground based LIDAR. They are the most data intensive projects requiring data dissemination and long term archive facilities. The adopted file format for data exchange and archive will be NASA-Ames format.
• Model and data analysis projects potentially use and produce large volumes of data. Continuous model development and improvement means it is not appropriate to keep this data in a long-term archive. There is a case, however, for storing some chemical fields to act as reference fields. The adopted file format for model data will be NetCDF.
• Chemical kinetics studies produce small amounts of data that is readily disseminated via published papers. This data is most usefully archived along with other chemical kinetic data rather than in isolation.
• There are significant demands for 3^rd party data sets to aid the planning and analysis of field observation campaigns, to establish the climatology and variability of the UTLS, and to evaluate the performance of numerical models. These data sets include ECMWF analyses, UKMO Unified Model output, and value-added products such as trajectories.
• Many of the UTLS-OZONE projects involve collaboration with EU and US funded programs. This presents some challenges for dissemination and archiving the data: how much of the data should be stored at the BADC, and what are the most effective ways to maintain references between data centres?

The British Atmospheric Data Centre (BADC) has been designated by NERC as the data centre for archiving data collected or produced as part of the UTLS-OZONE thematic program. All projects funded by NERC under the UTLS-OZONE thematic program have a commitment to archive the data they produce as part of the program at the BADC, if appropriate. The BADC will then be responsible for disseminating the data and provide a long-term archive.

The general objective of the scoping study was to determine the data requirements of the program. Having determined these requirements the study can be used to identify and prioritise the tasks needed to ensure that these requirements are met efficiently and effectively.

The specific objectives of the scoping study were to:

1. Determine the type of data to be collected.
2. Determine the 3^rd party data sets needed by the projects.
3. Determine a timetable for data collection.
4. Determine the most useful file formats for the data.
5. Determine the structure of the Data Archive.
6. Determine the most appropriate keywords for the data set catalogue.
7. Highlight any areas of data management that are uncertain at present.

Most of the information used in compiling this study was acquired using a data questionnaire sent to all round 1 and round 2 funded projects. A copy of the data questionnaire used in the data scoping study can be found in appendix A. Copies of the individual returns to the questionnaire can be obtained from the BADC on request. The returns to the questionnaire have been supplemented by e-mail contact with researchers in the individual projects.

One of the challenges of data management for the UTLS project is the diversity of the data sets produced as part of the project. For the purposes of data management the projects can be characterised as one of three types.

1. Field observation (FO): using instruments employed to observe geophysical parameters.
2. Model studies (MS) and data analysis (DA): using mainly computational and numerical techniques.
3. Chemical kinetics (CK): using laboratory based instruments.

Clearly some projects encompass more than one type. For instance field observation projects always include an element of data analysis. The division of projects into the different types is not intended to completely define the aims of any one project: it is intended to act as a simple categorisation to infer where the emphasis for data management tasks will be.

The data management demands for the three types of project are clearly different. Field observations, satellite data apart, tend to produce small data sets confined in time and space. The field observations need both to be disseminated rapidly and to be archived for the long term. Model studies and data analysis often use and produce large data sets. For these projects the 3^rd party data requirements, such as meteorological analyses, are often of a higher priority than long term archiving. Long term archiving of the data sets produced by MS and DA projects is of low priority since they have a limited life span as the models evolve and improve. The results of chemical kinetics experiments are very small data sets that can be disseminated either simply by the originating investigator or in published papers. The archiving of chemical kinetics results is obviously important, but is usually achieved by results appearing in published papers.

FOs are the most demanding of the types of project for data management. The BADC has gained experience of dealing with field observations through the ACSOE project. Although it has to be noted that not all of the field observation projects funded under UTLS will fit the ACSOE model. Many of the FO projects are funded under UTLS, but form part of larger field campaigns funded by other bodies such as the EU and NASA. These projects may have commitments to other funding bodies: these might include a commitment to archive data somewhere other than the BADC. The issues raised by the collaborations will be covered later in this document.

Table 1 lists the projects, with a classification of the project type, and any collaborations and/or prior data commitments. Of the thirteen FO projects nine involve collaborations with external projects, or have prior data commitments. Of the eight MS/DA projects three have the specific task of providing modeling support for non-UTLS projects.

Table 2 shows the main times when data will be collected. For projects funded in rounds one and two, most of the data will be collected during the first 3 years of funding. This implies that the initial emphasis for data management should be development and population of the archive. Development of value added products will be delayed until later in the program.

The data sets to be collected by UTLS-OZONE projects are listed in Table 3. The list is, at present, incomplete. More details of the various data sets will be added, as the information becomes available.

As noted earlier, many of the UTLS-OZONE projects are part of collaborations with NASA and EU funded projects. The collaborations raise several questions for the dissemination and archiving of the data collected as part of these projects.

If the collaborating project has its own designated data centre, then, presumably, that data centre will be responsible for the dissemination of data during field campaigns. The main question is then how much of the data archive will subsequently be mirrored at the BADC. Although there is a commitment to archive all UTLS-OZONE funded data at the BADC it may be that it is simply not appropriate, and possibly meaningless, to archive the UTLS-OZONE project data in isolation from the collaborating project data. In these cases it may be that the BADC could act as a mirror to the primary archive, although this obviously would entail negotiation with the data centre of the collaborating project. Alternatively, and probably more practically, the BADC could maintain references to the collaborating project data centre. This would pass responsibility for the maintenance of some of the UTLS-OZONE data archive to another data centre, which may have its own problems.

If the collaborating project does not have its own designated data centre, then the BADC could act as the data centre for the project, if required. The dissemination of UTLS data to other researchers in the collaborating projects can be facilitated by the BADC. Collaborating researchers would be required to sign a data agreement, in a similar way to ACSOE. It would have to be decided, probably on a case by case basis, how much of the UTLS-OZONE archive the external researchers would have access too. Presumably collaborating project researchers would only need access to the data sets produced by the UTLS-OZONE project with which they are collaborating. If the BADC is to facilitate the dissemination of data to collaborating projects, it will be necessary to set up the relevant mechanism in the archive at an early stage.

Table 4 lists the 3^rd party data requirements of the various UTLS-OZONE projects. The column labelled status indicates whether the project already has access to the required data set. If the data set is already available the column labeled source indicates the source of the data. It should be noted that some data sets have separate agreements and are not necessarily available to the all UTLS projects. These include the UKMO/ECMWF forecasts used during APE-THESEO (McKenzie), and, at least initially, the new 40-year ECMWF-ERA data (Lahoz et al).

Forecast data is required for mission planning during field campaigns. ECMWF forecast data can be obtained through the BADC, but requires an independent letter of application to go to the ECMWF.

Some of the 3^rd Party data requirements are already being met by the BADC. These include the ECMWF analyses and UKMO UARS assimilated data. Plans are already underway to meet the UTLS requirement for access to the UKMO Unified Model data. No doubt other 3^rd party data requirements will become evident throughout the period of the UTLS-OZONE program.

There is some demand for value added products such as trajectories, meteorological variables on isentropic surfaces, and meteorological variables along flight tracks. The BADC has developed a WWW interface to a trajectory model to help fulfil the demand for trajectory data by the UTLS-OZONE projects. The BADC will also investigate the possibility of adding tools to the BADC archive to calculate other value-added products.

The adoption of common, standard file formats both eases the dissemination of results and helps ensure the long-term integrity of a data set. Standard formats reduce any ambiguity in a data set and simplify the maintenance of reading and writing software and associated documentation. A good standard data format should be:

1. Portable across computer platforms.
2. Self-describing: there is enough information in the file to determine the contents of that file.

As well as being portable across platforms a good standard format will have potential for use in many analysis packages. In deciding the standard file format to be adopted it is important to consider users previous experience and the resources available for analysis.

Figure 1 indicates the experience of researchers funded by UTLS with various established data formats. NASA-Ames Format for Data Exchange is the format that has had most previous usage. This is an ASCII based format ideal for exchange of FO data. Each NASA Ames file consists of a header, which describes the contents of the file, followed by the data.

A binary format is often more useful for storage of large gridded data sets. Although researchers have about equal experience of GRIB and NetCDF format some thought that NetCDF was easier to use. NetCDF is a portable binary format, with access routines in FORTRAN, C, C++, perl, java, and IDL.

Figure 2 shows the analysis/display methods used by the researchers. IDL is the most widely used, although clearly a significant number use Excel and MATLAB.

Given the previous experience of data formats and analysis/plotting packages used UTLS Ozone will adopt NASA-Ames as the standard data format. Large gridded data sets will use NetCDF. NASA-Ames files are readable by IDL, Excel, and MATLAB. NetCDF files are supported by IDL, and some freely available software packages for plotting NetCDF files, such as FERRET. LiveAcess server can also be used to give WWW access to NetCDF files.

2.1 What data sets are you likely to produce as part of your UTLS-Ozone funded work? Please give some indication of the size (to an approximate order of magnitude) of each of these data sets.

2.2 Which of these data sets do you consider might be worth archiving in the long-term?

2.3 What will be the primary times of your data collection and/or model result production?

2.4 Do you think that other projects within UTLS-Ozone would find it useful to have access to your data?

2.5 Will you need to share data with other institutes?

If so, please give details:

Do you already have plans for doing this, or would you like assistance from the BADC?

2.6 When do you anticipate submitting data to the BADC archive?

3.1 If you anticipate using any third party data (e.g. Met. data or trajectories), please tell us what they are.

3.2 Do you already have access to these data sets? If not how do you anticipate getting them?

3.3 Are there any issues of intellectual property rights that will restrict the availability of these data to other UTLS projects or that will prevent them from being added to the "public" UTLS data sets at the end of the UTLS project?

4.1 Which of these standard data formats do you have experience of working with?

Please list any other formats that you might like to use:

Would you be prepared to write your data to standard data format(s), if assistance was available from the BADC to help with this job?

4.2 What computer hardware are you likely to use to work on your data? Please tick any appropriate boxes:

Other - please specify:

4.3 If you intend to use commercial software to manipulate your data, for example, IDL, PV-Wave or Excel, please list them in the space below:

Are there any other UTLS-Ozone data issues with which the BADC could help you? Do you have any other comments about the UTLS-Ozone data management? Are you happy with the UTLS-Ozone data protocol as it stands at present?