GRIB encoder

An encoder is used to generate data in a suitable format that can be written/added to a target. Encoders are typically used implicitly via to_target() but we can also instantiate an object and work with it directly.

import earthkit.data as ekd

# get some input GRIB data
ds = ekd.from_source("file", "test.grib")
ds[0].ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20200513	1200	0	an	0	regular_ll

# create a GribEncoder
encoder = ekd.create_encoder("grib")
encoder

<earthkit.data.encoders.grib.GribEncoder at 0x1076ed7b0>

The method to call is GribEncoder.encode(). The template argument can be a GRIB field. In the example below GribEncoder.encode() will simply create a copy (clone) of the underlying GRIB handle in the field.

r = encoder.encode(template=ds[0])
r

<earthkit.data.encoders.grib.GribEncodedData at 0x107875de0>

The resulting object can be used in various ways.

r.to_bytes()[:10]

b'GRIB\x00\x02\x0e\x01\x00\x00'

f = r.to_field()
f.ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20200513	1200	0	an	0	regular_ll

with open("_res_encoded.grib", "wb") as out:
    r.to_file(out)

Specifying metadata

When we specify metadata it will be written into the resulting GRIB fields.

r = encoder.encode(template=ds[0], metadata={"date": 20210514})
r.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20210514	1200	0	an	0	regular_ll

r = encoder.encode(template=ds[0], date=20210514)
r.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20210514	1200	0	an	0	regular_ll

Specifying new values

To replace the values in the resulting fields we need to use the values keyword argument.

vals = ds[0].values
r = encoder.encode(values=vals + 1, template=ds[0])
ds[0].values.max(), r.to_field().values.max()

(315.4599609375, 316.4599609375)

Specifying a field

When a field is specified as the data it is used as a template.

r = encoder.encode(data=ds[1], metadata={"date": 20210514})
r.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	msl	surface	0	20210514	1200	0	an	0	regular_ll

When both data and template are specified, the values from the field in data will be copied into the GRIB message created from the template.

r = encoder.encode(data=ds[1], template=ds[0], metadata={"date": 20210514})
r.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20210514	1200	0	an	0	regular_ll

We cannot use data, values and template together.

try:
    r = encoder.encode(data=ds[1], template=ds[0], values=vals, metadata={"date": 20210514})
except Exception as e:
    print(e)

Cannot provide data, values and template together

Encoding without a template

It is possible to encode GRIB data without providing a template using only values and metadata. This is an experimental feature and only works for certain metadata keys and the grid has to be either global lat-lon or reduced Gaussian grid. The geography is inferred from the shape of the specified values.

# global 1x1 degree data
import numpy as np

vals = np.random.normal(0, 1, (181, 360))
r = encoder.encode(values=vals, date=20250108, param="2t", time=12, step=24, edition=2)
r.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	heightAboveGround	2	20250108	1200	24	af	None	regular_ll

vals.max(), r.to_field().values.max()

(4.697071658422638, 4.696983814239502)

# encode as a pressure level field
r = encoder.encode(values=vals, date=20250108, param="t", level=700, levtype="pl", time=12, step=24, edition=2)
r.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	t	isobaricInhPa	700	20250108	1200	24	af	None	regular_ll

# global O96 data
vals = np.random.normal(0, 1, 40320)
r = encoder.encode(values=vals, date=20250108, param="2t", time=12, step=24, edition=2)
r.to_field().ls(extra_keys=["isOctahedral","N"])

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType	isOctahedral	N
0	ecmf	2t	heightAboveGround	2	20250108	1200	24	af	None	reduced_gg	1	96

Using preset options

# create a GribEncoder with preset options
encoder = ekd.create_encoder("grib", date=20250108, template=ds[0])

d1 = encoder.encode(step=12)
d2 = encoder.encode(step=24)

d1.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20250108	1200	12	fc	0	regular_ll

d2.to_field().ls()

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	2t	surface	0	20250108	1200	24	fc	0	regular_ll

Working with fieldlists

# create a GribEncoder
encoder = ekd.create_encoder("grib")

# encode now returns a generator
for d in encoder.encode(data=ds, step=18):
    print(d.to_field().ls(keys=["shortName", "step"]))

  shortName  step
0        2t    18
  shortName  step
0       msl    18