The provision of Open Access to Public meteorological Data and Development of Shared federated Data Infrastructure for the Development of information Products and Services

Date: 2024-11-01

Version: 0.1.0

Document location: TBD

Document status: DRAFT

RODEO project partners[1]

Copyright © 2024 RODEO project

i. Abstract

The Open Geospatial Consortium (OGC) Environmental Data Retrieval (EDR) API provides a family of lightweight interfaces to access Environmental Data resources. Each resource addressed by an EDR API maps to a defined query pattern. OGC API - EDR identifies resources, captures compliance classes, and specifies requirements which are applicable to OGC Environmental Data Retrieval API’s. The specification addresses two fundamental operations; discovery and query. Discovery operations allow the API to be interrogated to determine its capabilities and retrieve information (metadata) about this distribution of a resource. This includes the API definition of the server as well as metadata about the Environmental Data resources provided by the server. Query operations allow Environmental Data resources to be retrieved from the underlying data store based upon simple selection criteria, defined by this standard and selected by the client.

The flexibility of EDR allows for implementation in various environmental domains and communities of practice.

This document defines an EDR profile in support of providing access to meteorological datasets in the RODEO project. This includes, but is not limited to, conventions and constraints on identifiers, metadata parameters and encodings/formats and their related definitions and extensions.

ii. Keywords

The following are keywords to be used by search engines and document catalogues.

rodeo, eumetnet, ogc, api, edr, weather, meteorology, high value datasets

iii. Security Considerations

No additional security considerations have been made for this standard.

iii. Submitters

All questions regarding this submission should be directed to the editor or the submitters:

Name

Affiliation

Håvard Futsæter (editor)

MET Norway

Tom Kralidis (editor)

Meteorological Service of Canada

1. Scope

This document defines the conventions, constraints and extensions of EDR in the context of the RODEO project.

The RODEO EDR Profile defined herein is an extension of the International Standard OGC API - Environmental Data Retrieval - Part 1: Core.

This specification defines the conformance requirements for the RODEO EDR Profile. Annex A defines the abstract test suite. Annex B provides normative information on schemas. Annex C provides informative examples.

2. Conformance

Conformance with this standard shall be checked using the tests specified in Annex A (normative) of this document.

OGC API - Environmental Data Retrieval (EDR) provides a family of lightweight interfaces to access Environmental Data resources. Each resource addressed by an EDR API maps to a defined query pattern. This standard is a profile/extension of EDR. Conformance to this standard requires demonstrated conformance to the applicable Conformance Classes of OGC API - Environmental Data Retrieval - Part 1: Core.

Implementors of EDR API within the RODEO project are required to comply with the RODEO EDR Profile. A RODEO EDR API shall therefore be compliant with OGC API - Environmental Data Retrieval - Part 1: Core.

This standard identifies the following Requirements Classes which define the functional requirements.

  • "Core": baseline requirements class required by all other requirements classes in this document

  • "Observations": TBD

  • "Radar": TBD

Compliant implementation of this profile requires conformance to at least the "Core" Requirements Class.

3. References

  • OGC: OGC 19-086r6, OGC API - Environmental Data Retrieval Standard - Part 1: Core (2023) [2]

  • OGC: OGC 21-069r2, OGC CoverageJSON Community Standard (2023) [3]

  • IETF: RFC-7946 The GeoJSON Format (2016) [4]

  • IETF: RFC-8259 The JavaScript Object Notation (JSON) Data Interchange Format (2017) [5]

  • W3C/OGC: Spatial Data on the Web Best Practices, W3C Working Group Note (2017) [6]

  • W3C: Data on the Web Best Practices, W3C Recommendation (2017) [7]

  • IANA: Link Relation Types (2020) [8]

  • IANA: Media Types (2023) [9]

  • IETF: JSON Schema (2022) [10]

  • OpenAPI Specification 3.1.0 (2022) [11]

4. Terms and definitions

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “SHALL” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this Standard and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the 'ModSpec'. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

The following additional terms and definitions also apply.

4.1. Abbreviated terms

Table 1. Symbols and abbreviated terms
Abbreviation Term

API

Application Programming Interface

EU

European Union

EUMETNET

European National Meteorological and Hydrological Services

FAIR

Findable, Accessible, Interoperable, Reusable

FEMDI

Federated European Meteo-hydrological Data Infrastructure

GIS

Geographic Information System

HVD

High Value Datasets

HTML

Hypertext Markup Language

HTTP

Hypertext Transfer Protocol

HTTPS

Hypertext Transfer Protocol Secure

IANA

Internet Assigned Numbers Authority

IETF

Internet Engineering Task Force

ISO

International Organization for Standardization

JSON

JavaScript Object Notation

MIME

Multipurpose Internet Mail Extensions

NMHS

National Meteorological and Hydrological Service

NWP

Numerical Weather Prediction

OGC

Open Geospatial Consortium

REST

Representational State Transfer

ROA

Resource-oriented architecture

RODEO

The provision of open access to public meteorological data and development of shared federated data infrastructure for the development of information products and services

S3

Simple Storage Service

SEO

Search engine optimization

SOA

Service-oriented architecture

URI

Uniform Resource Identifier

URL

Uniform Resource Locator

W3C

World Wide Web Consortium

XML

eXtensible Markup Language

5. Conventions

This section provides details and examples for any conventions used in the document. Examples of conventions are symbols, abbreviations, use of JSON and / or JSON schema, or special notes regarding how to read the document.

5.1. Identifiers

The normative provisions in this Standard are denoted by the URI:

https://rodeo-project.eu/spec/rodeo-edr-profile/1

All requirements and conformance tests that appear in this document are denoted by partial URIs which are relative to this base.

5.2. Examples

Examples provided in this specification are encoded as JSON, GeoJSON or CoverageJSON.

5.3. Schema representation

JSON Schema [12] objects are used throughout this standard to define the structure of metadata records. These schema objects are also typically represented using YAML [13]. YAML is a superset of JSON, and in this standard are regarded as equivalent.

5.4. Use of HTTPS

For simplicity, this document only refers to the HTTP protocol. This is not meant to exclude the use of HTTPS and simply is a shorthand notation for "HTTP or HTTPS." In fact, most servers are expected to use HTTPS, not HTTP.

6. Introduction

RODEO, the Provision of Open Access to Public Meteorological Data and Development of Shared Federated Data Infrastructure for the Development of Information Products and Services, is a joint effort by eleven European National Meteorological and Hydrological Services (NMHS), the European Centre for Medium-Range Weather Forecasts (ECMWF) and the network of 31 European National Meteorological and Hydrological Services (EUMETNET). The Project Partners are listed in the page Partners [14].

The RODEO project develops a user interface and Application Programming Interfaces (API) for accessing meteorological datasets declared as High Value Datasets (HVD) by the EU Implementing Regulation (EU) 2023/138 under the EU Open Data Directive (EU) 2019/1024. The project also fosters the engagement between data providers and data users for enhancing the understanding of technical solutions being available for sharing and accessing the HVD datasets.

Surface weather observations, climate data, weather warnings, radar data and numerical weather prediction (NWP) data are defined as meteorological high value datasets. The distribution of these datasets is going to be made available under an open licence, in machine-readable formats using APIs and bulk downloads following the FAIR principles (usability, accessibility, interoperability and reusability).

The project is funded by the EU Digital Europe Program (DIGITAL) and EUMETNET.

6.1. Project Aims & Goals

The project makes meteorological High Value Datasets easily available with an aim to bring new data to businesses, public administrations and citizens. It reinforces the European public meteorological data infrastructure and enhances the providers’ digital capacity to share data. Furthermore, the project also fosters the engagement between data providers and data users for enhancing the understanding of technical solutions to be available for sharing and accessing the datasets.

The project strengthens the capacity of the European meteorological data providers to comply with the HVD Implementing Regulation by:

  • Developing a user interface

  • Developing APIs for accessing weather observation data, climate data, weather radar data, warnings, and Artificial Intelligence (AI) datasets

  • Developing a data catalogue for making data discoverable

  • Engaging with the data owners and user communities

  • Supporting the deployment of national data portals and APIs

  • Making HVDs available from the project partners

6.2. Impact on users

The increased availability of data boosts entrepreneurship and potentially results in the creation of new companies. New open datasets are an important resource for small and medium-sized enterprises to develop new digital products and services. Reuse of data opens business opportunities for several sectors ultimately leads attracting more investors. By making data much easier to use, reseach, academia, AI applications and many other application areas can utilize the data more efficiently.

Overall, better data availability leads to better warnings, forecasts, and services to the public and weather-critical industries, and contributes to the safe and efficient functioning of society with multiple benefits across the European economy, industry, and society.

7. Core

7.1. Requirements Class "Core"

URI

https://rodeo-project.eu/spec/rodeo-edr-profile/1/req/core

Target type

Web API

Dependency

OGC API - Environmental Data Retrieval Standard - Part 1: Core (2023) [15]

Pre-conditions

The API conforms to OGC API - Environmental Data Retrieval - Part 1: Core, Requirements Class: Core and Requirements Class: Collections

7.2. OpenAPI

An OpenAPI specification provides a machine-readable description of the API interface

Requirement 1

/req/core/openapi

A

An API definition SHALL be described using an OpenAPI document (version 3.1 or higher).

B

The OpenAPI document SHALL be encoded as JSON.

C

The OpenAPI document SHALL be made available in the API Landing Page as a link object with a relation type of service-desc.

D

API documentation SHALL be made available in the API Landing Page as a link object with a relation type of service-doc and content type of text/html.

7.3. Collection identifier

A collection identifier provides a mechanism to uniquely identify a given collection in an OGC API.

Requirement 2

/req/core/collection_identifier

A

A collection identifier SHALL NOT be used to convey structured metadata.

B

A collection identifier SHOULD use the following list of values, each suitable for a specific data type: insitu-observations, climate_data, radar_observations, weather_warnings, weather_forecast.

C

The identifier string MAY if needed include a postfix to the values listed above.

7.4. Collection title

A collection title provides a human readable short description of the given collection.

Requirement 3

/req/core/collection_title

A

A title SHALL be set for all collections.

B

A title SHALL NOT have more than 50 characters.

C

A title SHOULD be written in English.

D

A title SHOULD have the most important information first, guarding against truncated presentation of the value.

E

A title SHOULD describe the collection in a way understandable also for non-experts. Usually mention both topic/domain and geographical area.

7.5. Collection license

A license describes the usage permissions for the data in a collection.

Requirement 4

/req/core/collection_license

A

The links property in a collection SHALL contain a link to a license of its data.

B

The license link SHALL set the rel property to license.

C

The type property SHALL be set to text/html.

D

If the data is open data, the license SHALL be CC BY 4.0 and the href property set to https://creativecommons.org/licenses/by/4.0/.

8. Insitu observations

8.1. Requirements Class "Insitu observations"

URI

https://rodeo-project.eu/spec/rodeo-edr-profile/1/req/insitu-observations

Target type

Web API

Dependency

Core

8.2. Collection data queries

The types of data queries implemented in a collection.

Requirement 5

/req/insitu-observations/collection_data_queries

A

A collection SHALL support locations, area and radius.

Annex A: Conformance Class Abstract Test Suite (Normative)

A.1. Conformance Class: Core

label

https://rodeo-project.eu/spec/rodeo-edr-profile/1/conf/core

subject

Requirements Class "core"

classification

Target Type:Web API

A.1.1. OpenAPI

label

/conf/core/openapi

subject

/req/core/openapi

test-purpose

Validate that a RODEO EDR Profile provides an API definition using an OpenAPI document

Construct a path for a landing page.

Issue an HTTP GET request on that path.

Check that the value of the returned HTTP status header is 200.

In the links array in the landing page response, find the object with rel set to service-desc and type set to application/vnd.oai.openapi+json;version=3.1.

Issue an HTTP GET request using the value of href for that object.

Check that the value of the returned HTTP status header is 200 and check that the response body is a valid OpenAPI document of version 3.1 or higher.

In the links array in the landing page response, find the object with rel set to service-doc.

Issue an HTTP GET request using the value of href for that object.

Check that the value of the returned HTTP status header is 200 and that the HTTP response header Content-Type contains text/html.

A.1.2. Collection identifier

label

/conf/core/collection_identifier

subject

/req/core/collection_identifier

test-purpose

Validate that a RODEO EDR Profile API provides a valid collection identifier

This requirement is not applicable to ATS testing.

A.1.3. Collection title

label

/conf/core/collection_title

subject

/req/core/collection_title

test-purpose

Validate that a RODEO EDR Profile provides a valid collection title

Issue an HTTP GET request to path /collections.

Check that the value of the returned HTTP status header is 200.

In the collections array, check that the title property is present for all array elements.

For each title value, check that the value is less than or equal to 50 characters.

A.1.4. Collection license

label

/conf/core/collection_license

subject

/req/core/collection_license

test-purpose

Validate that a RODEO EDR Profile API provides a collection links array with a license link.

Issue an HTTP GET request to path /collections.

Check that the value of the returned HTTP status header is 200.

In the links array, check that there exists one link object with rel set to license.

Issue a HTTP GET request to the url in href for link with rel license.

Check that the value of the returned HTTP status header is 200 and the HTTP response header Content-Type is text/html.

A.2. Conformance Class: Insitu observations

label

https://rodeo-project.eu/spec/rodeo-edr-profile/1/conf/insitu-observations

subject

Requirements Class "Insitu observations"

classification

Target Type:Web API

A.2.1. Collection data queries

label

/conf/insitu-observations/collection_data_queries

subject

/req/insitu-observations/collection_data_queries

test-purpose

Validate that a RODEO EDR Insitu observations Profile API has implemented the mandatory data queries.

Issue an HTTP GET request to path /collections.

Check that the value of the returned HTTP status header is 200.

In the collections array in the returned document, check that each collection has a data_queries array containing at least area, locations and radius.

Annex B: Schemas (Normative)

Annex C: Examples (Informative)

C.1. Examples

Annex D: Bibliography

Annex E: Revision History

Date Release Editor Primary clauses modified Description

2024-09-16

Template

Tom Kralidis

all

initial template
1. https://rodeo-project.eu/partners/#project-partners
2. https://docs.ogc.org/is/19-086r6/19-086r6.html
3. https://docs.ogc.org/cs/21-069r2/21-069r2.html
4. https://datatracker.ietf.org/doc/html/rfc7946
5. https://datatracker.ietf.org/doc/html/rfc8259
6. https://www.w3.org/TR/sdw-bp
7. https://www.w3.org/TR/dwbp
8. https://www.iana.org/assignments/link-relations/link-relations.xml
9. https://www.iana.org/assignments/media-types/media-types.xhtml
10. https://json-schema.org
11. https://github.com/OAI/OpenAPI-Specification/blob/3.1.0/versions/3.1.0.md
12. https://json-schema.org
13. https://en.wikipedia.org/wiki/YAML
14. https://eurodeo.github.io/partners
15. https://docs.ogc.org/is/19-086r6/19-086r6.html