GRIB: writing data into FDB

import os

import earthkit.data as ekd

This example demonstrates how to write earthkit-data GRIB fields into an FDB.

FDB (Fields DataBase) is a domain-specific object store developed at ECMWF for storing, indexing and retrieving GRIB data. For more information on FBD please consult the following pages:

• FDB

• pyfdb

Setting up the target FDB

In this example we will create an FDB in the current folder using the schema taken from the pyfdb test suite. To do so first we need to ensure the directory exists. Next, we have to specify the configuration.

fdb_schema = "../default_fdb_schema"
fdb_dir = "./_fdb"
os.makedirs(fdb_dir, exist_ok=True)

config = {
    "type": "local",
    "engine": "toc",
    "schema": fdb_schema,
    "spaces": [{"handler": "Default", "roots": [{"path": fdb_dir}]}],
}

Archiving GRIB into the target FDB

First, we read temperature and wind GRIB data on pressure levels from a file.

ekd.download_example_file("tuv_pl.grib")
ds = ekd.from_source("file", "tuv_pl.grib").to_fieldlist()
ds.head()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	t	2018-08-01 12:00:00	2018-08-01 12:00:00	0 days	1000	pressure	0	regular_ll
1	u	2018-08-01 12:00:00	2018-08-01 12:00:00	0 days	1000	pressure	0	regular_ll
2	v	2018-08-01 12:00:00	2018-08-01 12:00:00	0 days	1000	pressure	0	regular_ll
3	t	2018-08-01 12:00:00	2018-08-01 12:00:00	0 days	850	pressure	0	regular_ll
4	u	2018-08-01 12:00:00	2018-08-01 12:00:00	0 days	850	pressure	0	regular_ll

Next, we write the whole fieldlist into the FDB.

ds.to_target("fdb", config=config)

Testing our FDB

We test our FDB by retrieving some fields from it using from_source().

request = {
    "class": "od",
    "expver": "0001",
    "stream": "oper",
    "date": "20180801",
    "time": 1200,
    "domain": "g",
    "type": "an",
    "levtype": "pl",
    "levelist": 500,
    "step": 0,
    "param": [131, 132],
}

# read fields from fdb as a stream
for f in ekd.from_source("fdb", request, config=config).to_fieldlist():
    print(f)

Field(u, 2018-08-01 12:00:00, 2018-08-01 12:00:00, 0:00:00, 500, pressure, 0, regular_ll)
Field(v, 2018-08-01 12:00:00, 2018-08-01 12:00:00, 0:00:00, 500, pressure, 0, regular_ll)