list-of-dicts: defining geography

This example demonstrates the various ways to define the values and the geography for list-of-dicts fieldlists.

First, we create a helper method to generate the input data.

import numpy as np

import earthkit.data as ekd


def build_lod(data):
    prototype = {
        **data,
        "time": {"valid_datetime": "2018-08-01T09:00:00Z"},
        "parameter": {"variable": "t"},
    }

    lod = [
        {"vertical": {"level": 500}, **prototype},
        {"vertical": {"level": 850}, **prototype},
    ]
    return lod

Disctinct latitudes-longitudes

For structured grids, the latitudes and longitudes can be specified as distinct values, the 2D field structure will be automatically built from them.

data = {
    "geography": {"latitudes": np.array([10.0, 0.0, -10.0]), "longitudes": np.array([20, 40.0])},
    "values": np.array([1, 2, 3, 4, 5, 6]),
}

ds = ekd.from_source("list-of-dicts", build_lod(data)).to_fieldlist()
ds.ls()

	parameter.variable	time.valid_datetime	time.base_datetime	time.step	vertical.level	vertical.level_type	ensemble.member	geography.grid_type
0	t	2018-08-01 09:00:00+00:00	2018-08-01 09:00:00+00:00	0 days	500	unknown	None	_distinct_ll
1	t	2018-08-01 09:00:00+00:00	2018-08-01 09:00:00+00:00	0 days	850	unknown	None	_distinct_ll

ds[0].shape, ds[0].geography.latlons(), ds[0].to_numpy()

((3, 2),
 (array([[ 10.,  10.],
         [  0.,   0.],
         [-10., -10.]]),
  array([[20., 40.],
         [20., 40.],
         [20., 40.]])),
 array([[1, 2],
        [3, 4],
        [5, 6]]))

We can define the same data by using the actual shape for the “values”.

data = {
    "geography": {"latitudes": np.array([10.0, 0.0, -10.0]), "longitudes": np.array([20, 40.0])},
    "values": np.array([[1, 2], [3, 4], [5, 6]]),
}

ds = ekd.from_source("list-of-dicts", build_lod(data)).to_fieldlist()
ds[0].shape, ds[0].geography.latlons(), ds[0].to_numpy()

((3, 2),
 (array([[ 10.,  10.],
         [  0.,   0.],
         [-10., -10.]]),
  array([[20., 40.],
         [20., 40.],
         [20., 40.]])),
 array([[1, 2],
        [3, 4],
        [5, 6]]))

Alternatively, the “disctinctLatitudes” and “distinctLongitudes” keys can be used. The result will be the same.

data = {
    "geography": {"distinctLatitudes": np.array([10.0, 0.0, -10.0]), "distinctLongitudes": np.array([20, 40.0])},
    "values": np.array([1, 2, 3, 4, 5, 6]),
}

ds = ekd.from_source("list-of-dicts", build_lod(data)).to_fieldlist()
ds[0].shape, ds[0].geography.latlons(), ds[0].to_numpy()

((6,), (None, None), array([1, 2, 3, 4, 5, 6]))

Full latitudes and longitudes

The other supported case is when the “latitudes” and “longitudes” contain all the points.

This example is the same as the distinct case above but the “latitudes” and “longitudes” now contain all the grid points.

data = {
    "geography": {
        "latitudes": np.array([10.0, 10.0, 0.0, 0.0, -10.0, -10.0]),
        "longitudes": np.array([20, 40.0, 20.0, 40.0, 20.0, 40.0]),
    },
    "values": np.array([1, 2, 3, 4, 5, 6]),
}

ds = ekd.from_source("list-of-dicts", build_lod(data)).to_fieldlist()
ds[0].shape, ds[0].geography.latlons(), ds[0].to_numpy()

((6,),
 (array([ 10.,  10.,   0.,   0., -10., -10.]),
  array([20., 40., 20., 40., 20., 40.])),
 array([1, 2, 3, 4, 5, 6]))

We can introduce the 2D structure by using a proper shape either in the “latitudes”/”longitudes” or the “values”. The field’s shape will be the one with the higher dimensionality.

data = {
    "geography": {
        "latitudes": np.array([[10.0, 10.0], [0.0, 0.0], [-10.0, -10.0]]),
        "longitudes": np.array([[20.0, 40.0], [20.0, 40.0], [20.0, 40.0]]),
    },
    "values": np.array([1, 2, 3, 4, 5, 6]),
}

ds = ekd.from_source("list-of-dicts", build_lod(data)).to_fieldlist()
ds[0].shape, ds[0].geography.latlons(), ds[0].to_numpy()

((3, 2),
 (array([[ 10.,  10.],
         [  0.,   0.],
         [-10., -10.]]),
  array([[20., 40.],
         [20., 40.],
         [20., 40.]])),
 array([[1, 2],
        [3, 4],
        [5, 6]]))

In this mode we are free to use arbitrary “latitudes” and “longitudes” without having any structure.

data = {
    "geography": {
        "latitudes": np.array([10.0, 10.0, 6.0, 6.0, 6.0]),
        "longitudes": np.array([20, 40.0, -40.0, 0.0, 40.0]),
    },
    "values": np.array([1, 2, 3, 4, 5]),
}

ds = ekd.from_source("list-of-dicts", build_lod(data)).to_fieldlist()
ds[0].shape, ds[0].geography.latlons(), ds[0].to_numpy()

((5,),
 (array([10., 10.,  6.,  6.,  6.]), array([ 20.,  40., -40.,   0.,  40.])),
 array([1, 2, 3, 4, 5]))