Home » IOOS: The System
Format for printing
Printer Friendly

IOOS Data Management & Communications Subsystem

Conceptually, the IOOS consists of 3 efficiently linked subsystems: Observing System(in situ measurements, remote sensing, and data telemetry), a Data Management and Communications(DMAC) subsystem and a Data Analysis and Modeling (DAM) subsystem

Central to the success of Integrated Ocean Observing System (IOOS) and other regional, national, and international ocean and coastal observing systems, is the presence of a Data Management and Communications (DMAC) Subsystem capable of delivering: real-time and delayed-mode observations to a wide variety of users: data streams to modeling centers; model-generated forecasts to users; distributed biological measurements to scientists, educators, and planners; and all forms of data to and from secure archive facilities. The needs of end users must be a part of the implementation and operation of the subsystem, both as sources of specifications for subsystem design, and as agents of change to keep the delivery of products from IOOS-relevant to national interests.

The relationship among the DMAC Subsystem and other IOOS components and partners is depicted in the figure to the right. Data flow within IOOS begins with the Observing Subsystem. Raw measurements from the Observing Subsystem elements are processed at various Primary Data Assembly and Quality Control sites to make them available to the uniform DMAC Data Communications Infrastructure. The infrastructure consists of standards and protocols to support:

  • IOOS-wide descriptions of data sets (Metadata);
  • the ability to search for and find data sets of interest (Data Discovery);
  • the ability to access the data in an interoperable manner from client applications (Data Transport)
  • the ability to evaluate the character of the data through common Web Browsers (Uniform On-line Browse); and
  • the ability to securely archive data and metadata and retrieve them on demand (Data Archive).

 

Schematic of the DMAC subsystem. Solid outlines indicate the elements of the IOOS Data Communications framework, which are detailed in the DMAC Implementation Plan. The arrows flowing outward from the users indicate the feedback and control mechanisms through which users ultimately direct the functioning of all parts of the system. Note that the National Data Management Systems are included in the concept of Primary Data Assembly and Quality Control.

 

A community-based process is underway by the DMAC Steering Team (appointed by Ocean.US in spring 2005) to identify, evaluate, and adopt existing data and metadata standards (protocols, interfaces, formats, ....) and best practices relevant to IOOS DMAC, and to propose development of new standards and tools. A description of this process, and the latest version of the resulting document, Guidance to IOOS Data Providers, can be found at http://dmac.ocean.us/dacsc/guidance02.jsp.

The information technology required to meet most of the needs of DMAC, while challenging, can be developed from existing capabilities through relatively straightforward software engineering. The greatest challenge facing DMAC is one of coordination and cooperation among IOOS partners and user communities. DMAC can succeed only if the participants actively use the data and metadata standards, communications protocols, software and policies that will knit IOOS into an integrated whole. The creation of a successful IOOS DMAC will require a sustained effort, a commitment across the U.S. marine community, and continual coordination with our international counterparts.

DMAC Guiding Principles
Interoperability:
 DMAC will serve as a framework for interoperability among heterogeneous cooperating systems. The cooperating systems will be free to evolve independently to address the needs of their target users. Software and standards needed to participate in DMAC will be available directly to partners, or provided through commercial and non-commercial sources. DMAC will also be interoperable with systems outside of the marine community that manage atmospheric and terrestrial data.
Open, easy access and discovery: DMAC will enable users from all over the globe to easily locate, access, and use the diverse distributed forms of marine data and their associated metadata and documentation in a variety of computer applications (e.g., Geographic Information Systems [GIS], and scientific analysis applications). Users will be unencumbered by traditional barriers such as data formats, volumes and distributed locations. DMAC will integrate cooperating systems so that data discovery will be seamless, and multiple versions will be easily tracked. There will be a “free market” of ocean sciences information, including officially sanctioned IOOS data sets, as well as data and products from other sources.

 
Reliable, sustained, efficient operations: DMAC will provide high reliability with 24/7 delivery of real-time data streams from measurement subsystems to operational modeling centers and users with time-critical requirements. It will provide high reliability in the delivery of computer-generated forecasts, estimates of state, and delayed-mode and real-time data to end users. DMAC will require sufficient bandwidth and adequate carrying capacity to support large exchanges of raw data and model outputs among high-volume users. DMAC will offer techniques that reduce the need for large data transfers, such as server-side subsetting and computation, to allow users with limited bandwidth to enjoy the benefits of IOOS. Feedback mechanisms will be built into the technical design of DMAC to ensure that problems are detected and rapidly addressed.
Effective user feedback: IOOS will provide a continuous, vigorous outreach process addressing all levels of users of marine data, emphasizing the benefits of participation in IOOS/DMAC, and helping to identify and remedy difficulties encountered by those who are participating. In addition, this process will identify and address changing user requirements that drive the development and growth of DMAC.
Open design and standards process: DMAC will commit to an open software design. All standards and protocol definitions will be openly published so that participating organizations may create functioning DMAC components based on these specifications. The standards development process will be open and inclusive, so that it fosters buy-in by all stakeholders. Existing information technology and scientific standards will be used in preference to development of new solutions, whenever suitable standards exist. The standards and protocols will be of sufficient breadth and quality to guarantee interoperability of all observations and products. Institutions participating in IOOS will ensure that the data they contribute comply with these standards and protocols.
Preservation of data and products: Irreplaceable observations, data products of lasting value, and associated metadata will be archived for posterity in an efficient and automated manner.
To learn about the DMAC planning and coordination activates and access DMAC plans and other guidance documents, go to: http://dmac.ocean.us/index.jsp

 


Last update: 21 November 2006 - 2:10pm