GRIB: create fieldlist from array and metadata

In this notebook we will show how to do some computations with GRIB data and generate a array fieldlist from the results.

First we read some GRIB data containing pressure level fields.

[1]:

import earthkit.data as ekd
from earthkit.data import FieldList
import numpy as np

ekd.download_example_file("tuv_pl.grib")
ds = ekd.from_source("file", "tuv_pl.grib")

We will use the following method to compute the potential temperature:

[2]:

def potential_temperature(t, p):
    # t: temperature in K
    # p: pressure in Pa
    return t*(100000./p)**0.285611

Working with a single field

In this example we compute the potential temperature for the 850 hPa level using the 4th field from the fieldlist.

[3]:

f = ds[3]
f, f.metadata("units")

[3]:

(GribField(t,850,20180801,1200,0,0), 'K')

The computations are done with numpy arrays:

[4]:

t = f.values
print("typeOfLevel=", f.metadata("typeOfLevel"))
p = f.metadata("level")*100. #hPa -> Pa
t_new = potential_temperature(t, p)
t_new[:10]

typeOfLevel= isobaricInhPa

[4]:

array([285.48786915, 285.48786915, 285.48786915, 285.48786915,
       285.48786915, 285.48786915, 285.48786915, 285.48786915,
       285.48786915, 285.48786915])

We create a new GribMetadata object from the source field’s metadata() using override(). The original metadata remains the same.

[5]:

md_new = f.metadata().override(shortName="pt")
print(md_new["shortName"])
print(f.metadata("shortName"))

pt
t

A new FieldList (type of SimpleFieldList) can be created from the resulting ndarray and the modified metadata. It will store a list of ArrayField fields each containing a values array and a metadata object entirely in memory. This fieldlist behaves as if it were a GribFieldList.

[6]:

ds_new = FieldList.from_array(t_new, md_new)
ds_new.ls()

[6]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	pt	isobaricInhPa	850	20180801	1200	0	an	0	regular_ll

[7]:

ds_new[0]

[7]:

ArrayField(pt,850,20180801,1200,0,0)

The new fieldlist can be saved into a GRIB file:

[8]:

path = "_pt_single.grib"
ds_new.to_target("file", path)

# read file back and check content
ds1 = ekd.from_source("file",path)
ds1.ls()

[8]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	stepRange	dataType	number	gridType
0	ecmf	pt	isobaricInhPa	850	20180801	1200	0	an	0	regular_ll

Working with multiple fields

In this example we compute the potential temperature for 3 pressure levels.

[9]:

fs = ds.sel(shortName="t", level=[850, 700, 500])
fs.ls(extra_keys=["units"])

[9]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	dataType	gridType	units
0	ecmf	t	isobaricInhPa	850	20180801	1200	an	regular_ll	K
1	ecmf	t	isobaricInhPa	700	20180801	1200	an	regular_ll	K
2	ecmf	t	isobaricInhPa	500	20180801	1200	an	regular_ll	K

[10]:

p = np.asarray(fs.metadata("level")).reshape(-1, 1)*100. # hPa -> Pa
t_new = potential_temperature(fs.values, p)
t_new.shape

[10]:

(3, 84)

We create an array fieldlist from the resulting ndarray and the modified metadata.

[11]:

md_new = [f.metadata().override(shortName="pt") for f in fs]
ds_new = FieldList.from_array(t_new, md_new)
ds_new.ls()

[11]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	dataType	gridType
0	ecmf	pt	isobaricInhPa	850	20180801	1200	an	regular_ll
1	ecmf	pt	isobaricInhPa	700	20180801	1200	an	regular_ll
2	ecmf	pt	isobaricInhPa	500	20180801	1200	an	regular_ll

The new fieldlist can be saved into a GRIB file:

[12]:

path = "_pt_multi.grib"
ds_new.to_target("file", path)

# read file back and check content
ds1 = ekd.from_source("file", path)
ds1.ls()

[12]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	dataType	gridType
0	ecmf	pt	isobaricInhPa	850	20180801	1200	an	regular_ll
1	ecmf	pt	isobaricInhPa	700	20180801	1200	an	regular_ll
2	ecmf	pt	isobaricInhPa	500	20180801	1200	an	regular_ll

Performing the computations in a loop

In this example we create an empty FieldList and add the results of the computations to it in a loop.

[13]:

fs = ds.sel(shortName="t", level=[850, 700, 500])

# create an empty fieldlist
ds_r = FieldList()

for f in fs:
    p = f.metadata("level")*100. # hPa -> Pa
    t_new = potential_temperature(f.values, p)
    md_new = f.metadata().override(shortName="pt")

    # create new numpy fieldlist with a single field
    ds_new = FieldList.from_array(t_new, md_new)

    # add it to the resulting fieldlist
    ds_r += ds_new

[14]:

ds_r.ls()

[14]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	dataType	gridType
0	ecmf	pt	isobaricInhPa	850	20180801	1200	an	regular_ll
1	ecmf	pt	isobaricInhPa	700	20180801	1200	an	regular_ll
2	ecmf	pt	isobaricInhPa	500	20180801	1200	an	regular_ll

The new fieldlist can be saved into a GRIB file:

[15]:

path = "_pt_from_loop.grib"
ds_r.to_target("file", path)

# read file back and check content
ds1 = ekd.from_source("file", path)
ds1.ls()

[15]:

	centre	shortName	typeOfLevel	level	dataDate	dataTime	dataType	gridType
0	ecmf	pt	isobaricInhPa	850	20180801	1200	an	regular_ll
1	ecmf	pt	isobaricInhPa	700	20180801	1200	an	regular_ll
2	ecmf	pt	isobaricInhPa	500	20180801	1200	an	regular_ll

[ ]: