GRIB encoder

An encoder is used to generate data in a suitable format that can be written/added to a target. Encoders are typically used implicitly via to_target() but we can also instantiate an object and work with it directly.

import earthkit.data as ekd

# get some input GRIB data
ds = ekd.from_source("sample", "test.grib").to_fieldlist()
ds[0].ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2020-05-13 12:00:00	2020-05-13 12:00:00	0 days	0	surface	0	regular_ll

# create a GribEncoder
encoder = ekd.create_encoder("grib")
encoder

<earthkit.data.encoders.grib.GribEncoder at 0x13549e7b0>

The method to call is encode(). The template argument can be a GRIB field. In the example below encode() will simply create a copy (clone) of the underlying GRIB handle in the field.

r = encoder.encode(template=ds[0])
r

<earthkit.data.encoders.grib.GribEncodedData at 0x1071e0440>

The resulting object can be used in various ways.

# get message as bytes
r.to_bytes()[:10]

b'GRIB\x00\x02\x0e\x01\x00\x00'

# convert to field
f = r.to_field()
f.ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2020-05-13 12:00:00	2020-05-13 12:00:00	0 days	0	surface	0	regular_ll

# write into a file
with open("_res_encoded.grib", "wb") as out:
    r.to_file(out)

Specifying metadata

When we specify GRIB metadata it will be written into the resulting GRIB fields.

r = encoder.encode(template=ds[0], metadata={"date": 20210514})
r.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2021-05-14 12:00:00	2021-05-14 12:00:00	0 days	0	surface	0	regular_ll

r = encoder.encode(template=ds[0], date=20210514)
r.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2021-05-14 12:00:00	2021-05-14 12:00:00	0 days	0	surface	0	regular_ll

Specifying new values

To replace the values in the resulting fields we need to use the values keyword argument.

vals = ds[0].values
r = encoder.encode(values=vals + 1, template=ds[0])
ds[0].values.max(), r.to_field().values.max()

(np.float64(315.4599609375), np.float64(316.4599609375))

Specifying a field

When a field is specified as the data it is used as a template.

r = encoder.encode(data=ds[1], metadata={"date": 20210514})
r.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	msl	2021-05-14 12:00:00	2021-05-14 12:00:00	0 days	0	surface	0	regular_ll

When both data and template are specified, the values from the field in data will be copied into the GRIB message created from the template.

r = encoder.encode(data=ds[1], template=ds[0], metadata={"date": 20210514})
r.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2021-05-14 12:00:00	2021-05-14 12:00:00	0 days	0	surface	0	regular_ll

We cannot use data, values and template together.

try:
    r = encoder.encode(data=ds[1], template=ds[0], values=vals, metadata={"date": 20210514})
except Exception as e:
    print(e)

Cannot provide data, values and template together

Encoding without a template

It is possible to encode GRIB data without providing a template using only values and metadata. This is an experimental feature and only works for certain metadata keys and the grid has to be either global lat-lon or reduced Gaussian grid. The geography is inferred from the shape of the specified values.

# global 1x1 degree data
import numpy as np

vals = np.random.normal(0, 1, (181, 360))
r = encoder.encode(values=vals, date=20250108, param="2t", time=12, step=24, edition=2)
r.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2025-01-09 12:00:00	2025-01-08 12:00:00	1 days	2	height_above_ground_level	None	regular_ll

vals.max(), r.to_field().values.max()

(np.float64(4.244827050861318), np.float64(4.244751930236816))

# encode as a pressure level field
r = encoder.encode(values=vals, date=20250108, param="t", level=700, levtype="pl", time=12, step=24, edition=2)
r.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	t	2025-01-09 12:00:00	2025-01-08 12:00:00	1 days	700	pressure	None	regular_ll

# global O96 data
vals = np.random.normal(0, 1, 40320)
r = encoder.encode(values=vals, date=20250108, param="2t", time=12, step=24, edition=2)
r.to_field().ls(extra_keys=["metadata.isOctahedral", "metadata.N"])

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type	metadata.isOctahedral	metadata.N
0	2t	2025-01-09 12:00:00	2025-01-08 12:00:00	1 days	2	height_above_ground_level	None	reduced_gg	1	96

Using preset options

# create a GribEncoder with preset options
encoder = ekd.create_encoder("grib", date=20250108, template=ds[0])

d1 = encoder.encode(step=12)
d2 = encoder.encode(step=24)

d1.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2025-01-09	2025-01-08 12:00:00	0 days 12:00:00	0	surface	0	regular_ll

d2.to_field().ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	2t	2025-01-09 12:00:00	2025-01-08 12:00:00	1 days	0	surface	0	regular_ll

Working with fieldlists

# create a GribEncoder
encoder = ekd.create_encoder("grib")

# encode now returns a generator
for d in encoder.encode(data=ds, step=18):
    print(d.to_field().ls(keys=["metadata.shortName", "metadata.step"]))

  metadata.shortName  metadata.step
0                 2t             18
  metadata.shortName  metadata.step
0                msl             18