Integration Plan:
Data Standards

Data Exchange Formats

For the data exchange between the project participants, the following data standards are recommended:

• Text files:
  • plain ascii
  • HTML
  • Word (*.doc files): uuencoded, as mail attachments, or by binary ftp transfer
  • RTF files (ascii)
• Imagery and graphics:
  • GIF files (please note that larger files should NOT be sent as mail attachments, but transferred by ftp;
  • TIFF (5.0, compressed) for 24-bit data
• GIS data:
  • Arc/Info export format
  • DLG (USGS Digital Line Graph format)
  • DXF (least desirable)
  • satellite imagery and raster data in any convenient format INCLUDING a format description.
• Model Data:
  • ascii data (with an appropriate format description, and where appropriate with the necessary reading routines)
  • binary data: together with description and the appropriate reading routines;

Client-Server Data Exchange

Client-server data exchange is based on the http protocol. The Demonstrator uses initially simple ascii files based on the protocol specifications for model scenarios and model result transfer. (see also: Architecture Specifications

This uses a simple protocol of keyword:value pairs, for example:

        URL:first.gmd.de/cgi-bin/model.start
        PN:ozone
        PN:NOx
        LL:layer1
        MetScenario:mild_weather
        StartTime:199710250000 # yyyymmddhhmm 
        TimeStep:1480    # in minutes
        RunLength:96     # in time steps
        EmScenario:name  
        NoxIndustry:1.2  # multipliers for data matrices
        NoxTraffic:1.0
        Noxhouseholds:0.9
        Noxairport:2.0
        VOCtotal:1.5

Meta-Data: The HDF Hierarchical Data Format

HDF is a library and platform independent data format for the storage and exchange of scientific data. It includes Fortran and C calling interfaces, and utilities for analyzing and converting HDF data files.

HDF is developed and supported by NCSA, and is available in the public domain. HDF is used world-wide in many fields, including Environmental Science, Neutron Scattering, Non-Destructive Testing, and Aerospace, to name a few. Scientific projects that use HDF include NASA's Mission to Planet Earth, and the DOE's Accelerated Strategic Computing Initiative. A detailed description of HDF is available from the NCSA's web server .

The primary intended application of HDF in ECOSIM is for the storage and Internet publishing of model scenarios

The main relevant features are:

• HDF stands for Hierarchical Data Format. It is a multi-object file format for the transfer of graphical and numerical data between machines.

• HDF is a versatile file format. It supports six different data models. Each data model defines a specific type of data and provides a convenient interface for reading, writing, and organizing a unique set of data elements.

• HDF is a self-describing format, allowing an application to interpret the structure and contents of a file without any outside information.

• HDF is a flexible file format. With HDF, you can group sets of related objects together and then access them as a group or as individual objects. There are pre-defined sets for raster images and floating point multidimensional arrays. Users can also create their own grouping structures using an HDF feature called vgroups.

• HDF is an extensible file format. It can easily accommodate new data models, regardless of whether they are added by the HDF development team or by HDF users.

• HDF is a portable file format. HDF files can be shared across platforms. An HDF file created on one computer, say a Cray supercomputer, can be read on another system, say IBM PC, without modification.

• HDF is available in the public domain.