data.core.fieldlist

Classes

Field	Represent a Field.
FieldList	Represent a list of Field s.

Module Contents

class data.core.fieldlist.Field

Bases: earthkit.data.core.Base

Represent a Field.

an_datetime()

property array_backend

Return the array namespace of the field.

Type:

ArrayBackend

property array_namespace

Return the array namespace of the field.

Type:

ArrayBackend

base_datetime()

batched(*args)

Return iterator for batches of data

bounding_box()

Return the bounding box of the field.

Return type:

BoundingBox

abstract clone(*, values=None, metadata=None, **kwargs)

Create a new ClonedField the with updated values and/or metadata.

Parameters:

• values (array-like or None) – The values to be stored in the new ClonedField. When it is None the resulting ClonedField will access the values from the original field.

• metadata (dict, Metadata or None) – If it is a dictionary, it is merged with **kwargs and interpreted in the same way as **kwargs. If it is a Metadata object, it is used as the new metadata. In this case **kwargs cannot be used.

• **kwargs (dict, optional) – Keys and values to update the metadata with. Metadata values can also be callables with the following positional arguments: original_field, key, original_metadata. The new ClonedField will contain a reference to the original metadata object and keys not present in kwargs will be accessed from the original field.

Returns:

The new field with updated values and/or metadata keeping a reference to the original field.

Return type:

ClonedField

Raises:

ValueError – If metadata is a Metadata object and **kwargs is not empty.

copy(*, values=None, flatten=False, dtype=None, array_backend=None, array_namespace=None, device=None, metadata=None)

Create a new ArrayField by copying the values and metadata.

Parameters:

• values (array-like or None) – The values to be stored in the new Field. When it is None the values extracted from the original field by using to_array with flatten, dtype and array_backend and copied to the new field.

• flatten (bool) – Control the shape of the values when they are extracted from the original field. When True, flatten the array, otherwise the field’s shape is kept. Only used when values is not provided.

• dtype (str, array.dtype or None) – Control the typecode or data-type of the values when they are extracted from the original field. If None, the default type used by the underlying data accessor is used. For GRIB it is float64. Only used when values is not provided.

• array_backend (str, array_namespace or None) – Control the array namespace of the values when they are extracted from the original field. If None, the underlying array format of the field is used. Only used when values is not provided. Deprecated since version 0.19.0. Use array_namespace instead. In versions before 0.19.0 an ArrayBackend was also accepted here, which is no longer the case.

• array_namespace (str, array_namespace or None) – Control the array namespace of the values when they are extracted from the original field. When it is None the underlying array format of the field is used. New in version 0.19.0.

• device (str or None) – The device where the array will be allocated. When it is None the default device is used.

• metadata (Metadata or None) – The metadata to be stored in the new Field. When it is None a copy of the metadata of the original field is used.

Return type:

ArrayField

data(keys=('lat', 'lon', 'value'), flatten=False, dtype=None, index=None)

Return the values and/or the geographical coordinates for each grid point.

Parameters:

• keys (str, list or tuple) – Specifies the type of data to be returned. Any combination of “lat”, “lon” and “value” is allowed here.

• flatten (bool) – When it is True a flat array per key is returned. Otherwise an array with the field’s shape is returned for each key.

• dtype (str, array.dtype or None) – Typecode or data-type of the arrays. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.

• index (array indexing object, optional) – The index of the values and or the latitudes/longitudes to be extracted. When it is None all the values and/or coordinates are extracted.

Returns:

An multi-dimensional array containing one array per key is returned (following the order in keys). The underlying array format of the field is used. When keys is a single value only the array belonging to the key is returned.

Return type:

array-like

Examples

• /examples/grib_lat_lon_value.ipynb

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test6.grib")
>>> d = ds[0].data()
>>> d.shape
(3, 7, 12)
>>> d[0, 0, 0]  # first latitude
90.0
>>> d[1, 0, 0]  # first longitude
0.0
>>> d[2, 0, 0]  # first value
272.56417847
>>> d = ds[0].data(keys="lon")
>>> d.shape
(7, 12)
>>> d[0, 0]  # first longitude
0.0

See also

to_latlon, to_points, to_numpy, values

datetime()

Return the date and time of the field.

Returns:

Dict with items “base_time” and “valid_time”.

Return type:

dict of datatime.datetime

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "tests/data/t_time_series.grib")
>>> ds[4].datetime()
{'base_time': datetime.datetime(2020, 12, 21, 12, 0),
'valid_time': datetime.datetime(2020, 12, 21, 18, 0)}

default_encoder()

describe(*args, **kwargs)

Generate a summary of the Field.

dump(namespace=all, **kwargs)

Generate dump with all the metadata keys belonging to namespace.

In a Jupyter notebook it is represented as a tabbed interface.

Parameters:

• namespace (str, list, tuple, None or all) –

  The namespace to dump. The following namespace values have a special meaning:

  • all: all the available namespaces will be used.

  • None or empty str: all the available keys will be used

    (without a namespace qualifier)

• **kwargs (dict, optional) – Other keyword arguments used for testing only

Returns:

Dict-like object with one item per namespace. In a Jupyter notebook represented as a tabbed interface to browse the dump contents.

Return type:

NamespaceDump

Examples

GRIB: inspecting contents

grid_points()

grid_points_unrotated()

group_by(*args)

Return iterator for batches of data grouped by metadata keys

h_datetime()

indexing_datetime()

abstract isel(*args, **kwargs)

ls(*args, **kwargs)

Generate a list like summary using a set of metadata keys.

Parameters:

• *args (tuple) – Positional arguments passed to FieldList.ls.

• **kwargs (dict, optional) – Other keyword arguments passed to FieldList.ls.

Returns:

DataFrame with one row.

Return type:

Pandas DataFrame

property mars_area

property mars_grid

classmethod merge(*args, **kwargs)

Merge the object with other ones.

abstract message()

Return a buffer containing the encoded message for message based formats (e.g. GRIB).

Return type:

bytes

metadata(*keys, astype=None, remapping=None, patches=None, **kwargs)

Return metadata values from the field.

When called without any arguments returns a Metadata object.

Parameters:

• *keys (tuple) – Positional arguments specifying metadata keys. Can be empty, in this case all the keys from the specified namespace will be used. (See examples below).

• astype (type name, list or tuple) – Return types for keys. A single value is accepted and applied to all the keys. Otherwise, must have same the number of elements as keys. Only used when keys is not empty.

• remapping (dict, optional) –

  Creates new metadata keys from existing ones that we can refer to in *args and **kwargs. E.g. to define a new key “param_level” as the concatenated value of the “param” and “level” keys use:

  remapping={"param_level": "{param}{level}"}
  

• **kwargs (dict, optional) –

  Other keyword arguments:

  • namespace: str, list, tuple, None or all

    The namespace to choose the keys from. When keys is empty and namespace is all all the available namespaces will be used. When keys is non empty namespace cannot specify multiple values and it cannot be all. When namespace is None or empty str all the available keys will be used (without a namespace qualifier).

  • default: value, optional

    Specifies the same default value for all the keys specified. When default is not present and a key is not found or its value is a missing value metadata will raise KeyError.

Returns:

• when called without any arguments returns a Metadata object

• when keys is not empty:

  • returns single value when keys is a str

  • otherwise returns the same type as that of keys (list or tuple)

• when keys is empty:

  • when namespace is None or an empty str returns a dict with all the available keys and values

  • when namespace is str returns a dict with the keys and values in that namespace

  • otherwise returns a dict with one item per namespace (dict of dict)

Return type:

single value, list, tuple, dict or Metadata

Raises:

KeyError – If no default is set and a key is not found in the message or it has a missing value.

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test.grib")

Calling without arguments:

>>> r = ds[0].metadata()
>>> r
<earthkit.data.readers.grib.metadata.GribMetadata object at 0x164ace170>
>>> r["name"]
'2 metre temperature'

Getting keys with their native type:

>>> ds[0].metadata("param")
'2t'
>>> ds[0].metadata("param", "units")
('2t', 'K')
>>> ds[0].metadata(("param", "units"))
('2t', 'K')
>>> ds[0].metadata(["param", "units"])
['2t', 'K']
>>> ds[0].metadata(["param"])
['2t']
>>> ds[0].metadata("badkey")
KeyError: 'badkey'
>>> ds[0].metadata("badkey", default=None)

Prescribing key types:

>>> ds[0].metadata("centre", astype=int)
98
>>> ds[0].metadata(["paramId", "centre"], astype=int)
[167, 98]
>>> ds[0].metadata(["centre", "centre"], astype=[int, str])
[98, 'ecmf']

Using namespaces:

>>> ds[0].metadata(namespace="parameter")
{'centre': 'ecmf', 'paramId': 167, 'units': 'K', 'name': '2 metre temperature', 'shortName': '2t'}
>>> ds[0].metadata(namespace=["parameter", "vertical"])
{'parameter': {'centre': 'ecmf', 'paramId': 167, 'units': 'K', 'name': '2 metre temperature',
 'shortName': '2t'},
 'vertical': {'typeOfLevel': 'surface', 'level': 0}}
>>> r = ds[0].metadata(namespace=all)
>>> r.keys()
dict_keys(['default', 'ls', 'geography', 'mars', 'parameter', 'statistics', 'time', 'vertical'])
>>> r = ds[0].metadata(namespace=None)
>>> len(r)
186
>>> r["name"]
'2 metre temperature'

mutate()

abstract order_by(*args, **kwargs)

Reorder the elements of the object.

projection()

Return information about the projection.

Return type:

Projection

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test.grib")
>>> ds.projection()
<Projected CRS: +proj=eqc +ellps=WGS84 +a=6378137.0 +lon_0=0.0 +to ...>
Name: unknown
Axis Info [cartesian]:
- E[east]: Easting (unknown)
- N[north]: Northing (unknown)
- h[up]: Ellipsoidal height (metre)
Area of Use:
- undefined
Coordinate Operation:
- name: unknown
- method: Equidistant Cylindrical
Datum: Unknown based on WGS 84 ellipsoid
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich
>>> ds.projection().to_proj_string()
'+proj=eqc +ellps=WGS84 +a=6378137.0 +lon_0=0.0 +to_meter=111319.4907932736 +no_defs +type=crs'

property resolution

property rotation

save(filename, append=False, **kwargs)

Write the field into a file.

Parameters:

• filename (str, optional) – The target file path, if not defined attempts will be made to detect the filename

• append (bool, optional) – When it is true append data to the target file. Otherwise the target file be overwritten if already exists. Default is False

• **kwargs (dict, optional) – Other keyword arguments passed to write.

abstract sel(*args, **kwargs)

Filter the object based on metadata.

property shape

Get the shape of the field.

For structured grids the shape is a tuple in the form of (Nj, Ni) where:

• ni: the number of gridpoints in i direction (longitude for a regular latitude-longitude grid)

• nj: the number of gridpoints in j direction (latitude for a regular latitude-longitude grid)

For other grid types the number of gridpoints is returned as (num,)

Type:

tuple

to_array(flatten=False, dtype=None, array_backend=None, array_namespace=None, device=None, index=None)

Return the values stored in the field.

Parameters:

• flatten (bool) – When it is True a flat array is returned. Otherwise an array with the field’s shape is returned.

• dtype (str, array.dtype or None) – Typecode or data-type of the array. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.

• array_backend (str, array_namespace or None) – The array namespace to be used. When it is None the underlying array format of the field is used. Deprecated since version 0.19.0. Use array_namespace instead. In versions before 0.19.0 an ArrayBackend was also accepted here, which is no longer the case.

• array_namespace (str, array_namespace or None) – The array namespace to be used. When it is None the underlying array format of the field is used. New in version 0.19.0.

• device (str or None) – The device where the array will be allocated. When it is None the default device is used.

• index (array indexing object, optional) – The index of the values and to be extracted. When it is None all the values are extracted

Returns:

Field values.

Return type:

array-array

to_latlon(flatten=False, dtype=None, index=None)

Return the latitudes/longitudes of all the gridpoints in the field.

Parameters:

• flatten (bool) – When it is True 1D arrays are returned. Otherwise arrays with the field’s shape are returned.

• dtype (str, array.dtype or None) – Typecode or data-type of the arrays. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.

• index (array indexing object, optional) – The index of the latitudes/longitudes to be extracted. When it is None all the values are extracted.

Returns:

Dictionary with items “lat” and “lon”, containing the arrays of the latitudes and longitudes, respectively. The underlying array format of the field is used.

Return type:

dict

See also

to_points

to_numpy(flatten=False, dtype=None, index=None)

Return the values stored in the field as an ndarray.

Parameters:

• flatten (bool) – When it is True a flat ndarray is returned. Otherwise an ndarray with the field’s shape is returned.

• dtype (str, numpy.dtype or None) – Typecode or data-type of the array. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.

• index (ndarray indexing object, optional) – The index of the values and to be extracted. When it is None all the values are extracted

Returns:

Field values

Return type:

ndarray

abstract to_pandas(**kwargs)

Convert into a pandas dataframe

to_points(flatten=False, dtype=None, index=None)

Return the geographical coordinates in the data’s original Coordinate Reference System (CRS).

Parameters:

• flatten (bool) – When it is True 1D arrays are returned. Otherwise arrays with the field’s shape are returned.

• dtype (str, array.dtype or None) – Typecode or data-type of the arrays. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.

• index (array indexing object, optional) – The index of the coordinates to be extracted. When it is None all the values are extracted.

Returns:

Dictionary with items “x” and “y”, containing the arrays of the x and y coordinates, respectively. The underlying array format of the field is used.

Return type:

dict

Raises:

ValueError – When the coordinates in the data’s original CRS are not available.

See also

to_latlon

to_target(target, *args, **kwargs)

Write the field into a target object.

Parameters:

• target (object) – The target object to write the field into.

• *args (tuple) – Positional arguments used to specify the target object.

• **kwargs (dict, optional) – Other keyword arguments used to write the field into the target object.

to_xarray(*args, **kwargs)

Convert the Field into an Xarray Dataset.

Parameters:

• *args (tuple) – Positional arguments passed to FieldList.to_xarray.

• **kwargs (dict, optional) – Other keyword arguments passed to FieldList.to_xarray.

Return type:

Xarray Dataset

unique_values(*coords, remapping=None, patches=None, progress_bar=False)

Given a list of metadata attributes, such as date, param, levels, returns the list of unique values for each attributes

valid_datetime()

property values

Get the values stored in the field as a 1D array.

Type:

array-like

write(f, **kwargs)

class data.core.fieldlist.FieldList(**kwargs)

Bases: earthkit.data.core.index.Index

Represent a list of Field s.

batched(n)

Iterate through the object in batches of n.

Parameters:

n (int) – Batch size.

Returns:

Returns an iterator yielding batches of n elements. Each batch is a new object containing a view to the data in the original object, so no data is copied. The last batch may contain fewer than n elements.

Return type:

object

bounding_box()

Return the bounding box for each field.

Returns:

List with one BoundingBox per Field

Return type:

list

config(name)

cube(*args, **kwargs)

data(keys=('lat', 'lon', 'value'), flatten=False, dtype=None, index=None)

Return the values and/or the geographical coordinates.

Only works when all the fields have the same grid geometry.

Parameters:

• keys (str, list or tuple) – Specifies the type of data to be returned. Any combination of “lat”, “lon” and “value” is allowed here.

• flatten (bool) – When it is True the “lat”, “lon” arrays and the “value” arrays per field will all be flattened. Otherwise they will preserve the field’s shape.

• dtype (str, array.dtype or None) – Typecode or data-type of the arrays. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.

• index (array indexing object, optional) – The index of the values to be extracted from each field. When it is None all the values are extracted.

Returns:

The elements of the array (in the order of the keys) are as follows:

• the latitudes array from the first field when “lat” is in keys

• the longitudes array from the first field when “lon” is in keys

• a values array per field when “values” is in keys

Return type:

array-like

Raises:

ValueError – When not all the fields have the same grid geometry.

Examples

• /examples/grib_lat_lon_value.ipynb

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test6.grib")
>>> len(ds)
6
>>> d = ds.data()
>>> d.shape
(8, 7, 12)
>>> d[0, 0, 0]  # first latitude
90.0
>>> d[1, 0, 0]  # first longitude
0.0
>>> d[2:, 0, 0]  # first value per field
array([272.56417847,  -6.28688049,   7.83348083, 272.53916931,
        -4.89837646,   8.66096497])
>>> d = ds.data(keys="lon")
>>> d.shape
(1, 7, 12)
>>> d[0, 0, 0]  # first longitude
0.0

See also

to_latlon, to_points, to_numpy, values

datetime()

Return the unique, sorted list of dates and times in the fieldlist.

Returns:

Dict with items “base_time” and “valid_time”.

Return type:

dict of datatime.datetime

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "tests/data/t_time_series.grib")
>>> ds.datetime()
{'base_time': [datetime.datetime(2020, 12, 21, 12, 0)],
'valid_time': [
    datetime.datetime(2020, 12, 21, 12, 0),
    datetime.datetime(2020, 12, 21, 15, 0),
    datetime.datetime(2020, 12, 21, 18, 0),
    datetime.datetime(2020, 12, 21, 21, 0),
    datetime.datetime(2020, 12, 23, 12, 0)]}

default_encoder()

describe(*args, **kwargs)

Generate a summary of the fieldlist.

static from_array(array, metadata)

Create an SimpleFieldList.

Parameters:

• array (array-like, list) – The fields’ values. When it is a list it must contain one array per field.

• metadata (list, Metadata) – The fields’ metadata. Must contain one Metadata object per field. Or it can be a single Metadata object when all the fields have the same metadata.

:param In the generated SimpleFieldList: :param each field is represented by an array: :param storing the field values and a MetaData object holding: :param the field metadata. The shape and dtype of the array is controlled by the kwargs.:

static from_fields(fields)

Create a SimpleFieldList.

Parameters:

fields (iterable) – Iterable of Field objects.

Return type:

SimpleFieldList

static from_numpy(array, metadata)

group_by(*keys, sort=True)

Iterate through the object in groups defined by metadata keys.

Parameters:

• *keys (tuple) – Positional arguments specifying the metadata keys to group by. Keys can be a single or multiple str, or a list or tuple of str.

• sort (bool, optional) – If True (default), the object is sorted by the metadata keys before grouping. Sorting is only applied if the object is supporting the sorting operation.

Returns:

Returns an iterator yielding batches of elements grouped by the metadata keys. Each batch is a new object containing a view to the data in the original object, so no data is copied. It generates a new group every time the value of the keys change.

Return type:

object

head(n=5, **kwargs)

Generate a list like summary of the first n Fields. Same as calling ls with n.

Parameters:

• n (int, None) – The number of messages (n > 0) to be printed from the front.

• **kwargs (dict, optional) – Other keyword arguments passed to ls.

Returns:

See ls.

Return type:

Pandas DataFrame

See also

ls, tail

Notes

The following calls are equivalent:

ds.head()
ds.head(5)
ds.head(n=5)
ds.ls(5)
ds.ls(n=5)

index(key)

Return the unique, sorted values of the specified metadata key from all the fields. key will be automatically added to the keys returned by indices.

Parameters:

key (str) – Metadata key.

Returns:

Unique, sorted values of key from all the Fields.

Return type:

list

See also

index

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/tuv_pl.grib")
>>> ds.index("level")
[300, 400, 500, 700, 850, 1000]

indices(squeeze=False)

Return the unique, sorted values for a set of metadata keys (see below) from all the fields. Individual keys can be also queried by index.

Parameters:

squeeze (False) – When True only returns the metadata keys that have more than one values.

Returns:

Unique, sorted metadata values from all the Fields.

Return type:

dict

See also

index

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/tuv_pl.grib")
>>> ds.indices()
{'class': ['od'], 'stream': ['oper'], 'levtype': ['pl'], 'type': ['an'],
'expver': ['0001'], 'date': [20180801], 'time': [1200], 'domain': ['g'],
'number': [0], 'levelist': [300, 400, 500, 700, 850, 1000],
'param': ['t', 'u', 'v']}
>>> ds.indices(squeeze=True)
{'levelist': [300, 400, 500, 700, 850, 1000], 'param': ['t', 'u', 'v']}
>>> ds.index("level")
[300, 400, 500, 700, 850, 1000]

By default indices uses the keys from the “mars” ecCodes namespace. Keys with no valid values are not included. Keys that index is called with are automatically added to the original set of keys used in indices.

isel(*args, **kwargs)

Uses metadata value indices to select a subset of the elements from a fieldlist-like object.

Parameters:

• *args (tuple) – Positional arguments specifying the filter conditions. (See below for details).

• **kwargs (dict, optional) – Other keyword arguments specifying the metadata keys to perform the filtering on. (See below for details).

Returns:

Returns a new object with the filtered elements. It contains a view to the data in the original object, so no data is copied.

Return type:

object

Notes

isel works similarly to sel but conditions are specified by indices of metadata keys. A metadata index stores the unique, sorted values of the corresponding metadata key from all the fields in the input data. If the object is a obj:FieldList <data.readers.grib.index.FieldList> to list the indices that have more than one values use FieldList.indices(), or to find out the values of a specific index use FieldList.index().

Filter conditions are specified by a set of metadata keys either by a dictionary (in *args) or a set of **kwargs. Both single or multiple keys are allowed to use and each can specify the following type of filter values:

• single index:

  ds.sel(param=1)
  

• list of indices:

  ds.sel(param=[1, 3])
  

• slice of values (behaves like normal Python indexing, stop value not included):

  # filter levels on level indices 1 and 2
  ds.sel(level=slice(1,3))
  

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/tuv_pl.grib")

>>> len(ds)
18
>>> ds.indices
{'levelist': (1000, 850, 700, 500, 400, 300), 'param': ('t', 'u', 'v')}

>>> subset = ds.isel(param=0)
>>> len(ds)
6

>>> for f in subset:
...     print(f)
...
GribField(t,1000,20180801,1200,0,0)
GribField(t,850,20180801,1200,0,0)
GribField(t,700,20180801,1200,0,0)
GribField(t,500,20180801,1200,0,0)
GribField(t,400,20180801,1200,0,0)
GribField(t,300,20180801,1200,0,0)

>>> subset = ds.isel(param=[1, 2], level=slice(2, 4))
>>> len(subset)
4

>>> for f in subset:
...     print(f)
...
GribField(u,700,20180801,1200,0,0)
GribField(v,700,20180801,1200,0,0)
GribField(u,500,20180801,1200,0,0)
GribField(v,500,20180801,1200,0,0)

ls(n=None, keys=None, extra_keys=None, namespace=None)

Generate a list like summary using a set of metadata keys.

Parameters:

• n (int, None) – The number of Fields to be listed. None means all the messages, n > 0 means fields from the front, while n < 0 means fields from the back of the fieldlist.

• keys (list of str, dict, None) – Metadata keys. If it is None the following default set of keys will be used: “centre”, “shortName”, “typeOfLevel”, “level”, “dataDate”, “dataTime”, “stepRange”, “dataType”, “number”, “gridType”. To specify a column title for each key in the output use a dict.

• extra_keys (list of str, dict, None) – List of additional keys to keys. To specify a column title for each key in the output use a dict.

• namespace (str, None) – The namespace to choose the keys from. When it is set keys and extra_keys are omitted.

Returns:

DataFrame with one row per Field.

Return type:

Pandas DataFrame

See also

head, tail

metadata(*args, **kwargs)

Return the metadata values for each field.

Parameters:

• *args (tuple) – Positional arguments defining the metadata keys. Passed to GribField.metadata()

• **kwargs (dict, optional) – Keyword arguments passed to GribField.metadata()

Returns:

List with one item per GribField

Return type:

list

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test.grib")
>>> ds.metadata("param")
['2t', 'msl']
>>> ds.metadata("param", "units")
[('2t', 'K'), ('msl', 'Pa')]
>>> ds.metadata(["param", "units"])
[['2t', 'K'], ['msl', 'Pa']]

order_by(*args, remapping=None, patches=None, **kwargs)

Changes the order of the elements in a fieldlist-like object.

Parameters:

• *args (tuple) – Positional arguments specifying the metadata keys to perform the ordering on. (See below for details)

• remapping (dict) –

  Defines new metadata keys from existing ones that we can refer to in *args and **kwargs. E.g. to define a new key “param_level” as the concatenated value of the “param” and “level” keys use:

  remapping={"param_level": "{param}{level}"}
  

  See below for a more elaborate example.

• **kwargs (dict, optional) – Other keyword arguments specifying the metadata keys to perform the ordering on. (See below for details)

Returns:

Returns a new object with reordered elements. It contains a view to the data in the original object, so no data is copied.

Return type:

object

Examples

Ordering by a single metadata key (“param”). The default ordering direction is ascending:

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test6.grib")
>>> for f in ds.order_by("param"):
...     print(f)
...
GribField(t,850,20180801,1200,0,0)
GribField(t,1000,20180801,1200,0,0)
GribField(u,850,20180801,1200,0,0)s
GribField(u,1000,20180801,1200,0,0)
GribField(v,850,20180801,1200,0,0)
GribField(v,1000,20180801,1200,0,0)

Ordering by multiple keys (first by “level” then by “param”):

>>> for f in ds.order_by(["level", "param"]):
...     print(f)
...
GribField(t,850,20180801,1200,0,0)
GribField(u,850,20180801,1200,0,0)
GribField(v,850,20180801,1200,0,0)
GribField(t,1000,20180801,1200,0,0)
GribField(u,1000,20180801,1200,0,0)
GribField(v,1000,20180801,1200,0,0)

Specifying the ordering direction:

>>> for f in ds.order_by(param="ascending", level="descending"):
...     print(f)
...
GribField(t,1000,20180801,1200,0,0)
GribField(t,850,20180801,1200,0,0)
GribField(u,1000,20180801,1200,0,0)
GribField(u,850,20180801,1200,0,0)
GribField(v,1000,20180801,1200,0,0)
GribField(v,850,20180801,1200,0,0)

Using the list of all the values of a key (“param”) to define the order:

>>> for f in ds.order_by(param=["u", "t", "v"]):
...     print(f)
...
GribField(u,1000,20180801,1200,0,0)
GribField(u,850,20180801,1200,0,0)
GribField(t,1000,20180801,1200,0,0)
GribField(t,850,20180801,1200,0,0)
GribField(v,1000,20180801,1200,0,0)
GribField(v,850,20180801,1200,0,0)

Using remapping to specify the order by a key created from two other keys (we created key “param_level” from “param” and “levelist”):

>>> ordering = ["t850", "t1000", "u1000", "v850", "v1000", "u850"]
>>> remapping = {"param_level": "{param}{levelist}"}
>>> for f in ds.order_by(param_level=ordering, remapping=remapping):
...     print(f)
...
GribField(t,850,20180801,1200,0,0)
GribField(t,1000,20180801,1200,0,0)
GribField(u,1000,20180801,1200,0,0)
GribField(v,850,20180801,1200,0,0)
GribField(v,1000,20180801,1200,0,0)
GribField(u,850,20180801,1200,0,0)

property parent

projection()

Return the projection information shared by all the fields.

Return type:

Projection

Raises:

ValueError – When not all the fields have the same grid geometry

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test.grib")
>>> ds.projection()
<Projected CRS: +proj=eqc +ellps=WGS84 +a=6378137.0 +lon_0=0.0 +to ...>
Name: unknown
Axis Info [cartesian]:
- E[east]: Easting (unknown)
- N[north]: Northing (unknown)
- h[up]: Ellipsoidal height (metre)
Area of Use:
- undefined
Coordinate Operation:
- name: unknown
- method: Equidistant Cylindrical
Datum: Unknown based on WGS 84 ellipsoid
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich
>>> ds.projection().to_proj_string()
'+proj=eqc +ellps=WGS84 +a=6378137.0 +lon_0=0.0 +to_meter=111319.4907932736 +no_defs +type=crs'

save(filename, append=False, **kwargs)

Write all the fields into a file.

Parameters:

• filename (str, optional) – The target file path, if not defined attempts will be made to detect the filename

• append (bool, optional) – When it is true append data to the target file. Otherwise the target file be overwritten if already exists. Default is False

• **kwargs (dict, optional) – Other keyword arguments passed to write.

See also

write GribFieldList.save() SimpleFieldList.save()

sel(*args, remapping=None, **kwargs)

Uses metadata values to select a subset of the elements from a fieldlist-like object.

Parameters:

• *args (tuple) – Positional arguments specifying the filter condition as dict. (See below for details).

• remapping (dict) –

  Creates new metadata keys from existing ones that we can refer to in *args and **kwargs. E.g. to define a new key “param_level” as the concatenated value of the “param” and “level” keys use:

  remapping={"param_level": "{param}{level}"}
  

  See below for a more elaborate example.

• **kwargs (dict, optional) – Other keyword arguments specifying the filter conditions. (See below for details).

Returns:

Returns a new object with the filtered elements. It contains a view to the data in the original object, so no data is copied.

Return type:

object

Notes

Filter conditions are specified by a set of metadata keys either by a dictionary (in *args) or a set of **kwargs. Both single or multiple keys are allowed to use and each can specify the following type of filter values:

• single value:

  ds.sel(param="t")
  

• list of values:

  ds.sel(param=["u", "v"])
  

• slice of values (defines a closed interval, so treated as inclusive of both the start

and stop values, unlike normal Python indexing):

# filter levels between 300 and 500 inclusively
ds.sel(level=slice(300, 500))

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/tuv_pl.grib")
>>> len(ds)
18

Selecting by a single key (“param”):

>>> subset = ds.sel(param="t")
>>> for f in subset:
...     print(f)
...
GribField(t,1000,20180801,1200,0,0)
GribField(t,850,20180801,1200,0,0)
GribField(t,700,20180801,1200,0,0)
GribField(t,500,20180801,1200,0,0)
GribField(t,400,20180801,1200,0,0)
GribField(t,300,20180801,1200,0,0)

Selecting by multiple keys (“param”, “level”) with a list and slice of values:

>>> subset = ds.sel(param=["u", "v"], level=slice(400, 700))
>>> for f in subset:
...     print(f)
...
GribField(u,700,20180801,1200,0,0)
GribField(v,700,20180801,1200,0,0)
GribField(u,500,20180801,1200,0,0)
GribField(v,500,20180801,1200,0,0)
GribField(u,400,20180801,1200,0,0)
GribField(v,400,20180801, 1200,0,0)

Using remapping to specify the selection by a key created from two other keys (we created key “param_level” from “param” and “levelist”):

>>> subset = ds.sel(
...     param_level=["t850", "u1000"],
...     remapping={"param_level": "{param}{levelist}"},
... )
>>> for f in subset:
...     print(f)
...
GribField(u,1000,20180801,1200,0,0)
GribField(t,850,20180801,1200,0,0)

tail(n=5, **kwargs)

Generate a list like summary of the last n Fields. Same as calling ls with -n.

Parameters:

• n (int, None) – The number of messages (n > 0) to be printed from the back.

• **kwargs (dict, optional) – Other keyword arguments passed to ls.

Returns:

See ls.

Return type:

Pandas DataFrame

See also

head, ls

Notes

The following calls are equivalent:

ds.tail()
ds.tail(5)
ds.tail(n=5)
ds.ls(-5)
ds.ls(n=-5)

to_array(**kwargs)

Return all the fields’ values as an array. It is formed as the array of the data.core.fieldlist.Field.to_array values per field.

Parameters:

**kwargs (dict, optional) – Keyword arguments passed to data.core.fieldlist.Field.to_array

Returns:

Array containing the field values.

Return type:

array-like

See also

values, to_numpy

to_fieldlist(array_backend=None, array_namespace=None, device=None, **kwargs)

Convert to a new FieldList.

Parameters:

• array_backend (str, array_namespace or None) – Specify the array namespace for the generated FieldList. Deprecated in version 0.19.0. Use array_namespace instead. In versions before 0.19.0 an ArrayBackend was also accepted here, which is no longer the case.

• array_namespace (str, array_namespace or None) – The array namespace to be used. New in version 0.19.0.

• device (str or None) – The device where the array will be allocated. When it is None the default device is used. New in version 0.19.0.

• **kwargs (dict, optional) – kwargs are passed to to_array to extract the field values the resulting object will store.

Returns:

• a new fieldlist containing :class`ArrayField` fields

Return type:

SimpleFieldList

Examples

The following example will convert a fieldlist read from a file into a SimpleFieldList storing single precision field values.

>>> import numpy as np
>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/tuv_pl.grib")
>>> ds.path
'docs/examples/tuv_pl.grib'
>>> r = ds.to_fieldlist(array_namespace="numpy", dtype=np.float32)
>>> r
SimpleFieldList(fields=18)
>>> hasattr(r, "path")
False
>>> r.to_numpy().dtype
dtype('float32')

to_latlon(index=None, **kwargs)

Return the latitudes/longitudes shared by all the fields.

Parameters:

• index (array indexing object, optional) – The index of the latitudes/longitudes to be extracted. When it is None all the values are extracted.

• **kwargs (dict, optional) – Keyword arguments passed to Field.to_latlon()

Returns:

Dictionary with items “lat” and “lon”, containing the arrays of the latitudes and longitudes, respectively.

Return type:

dict

Raises:

ValueError – When not all the fields have the same grid geometry

Examples

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test.grib")
>>> for f in ds:
...     print(f.shape)
...
(11, 19)
(11, 19)
>>> r = ds.to_latlon()
>>> for k, v in r.items():
...     print(f"{k}: shape={v.shape}")
...
lat: shape=(11, 19)
lon: shape=(11, 19)
>>> r["lon"][:2]
array([[-27., -23., -19., -15., -11.,  -7.,  -3.,   1.,   5.,   9.,  13.,
 17.,  21.,  25.,  29.,  33.,  37.,  41.,  45.],
[-27., -23., -19., -15., -11.,  -7.,  -3.,   1.,   5.,   9.,  13.,
 17.,  21.,  25.,  29.,  33.,  37.,  41.,  45.]])

to_numpy(**kwargs)

Return all the fields’ values as an ndarray. It is formed as the array of the data.core.fieldlist.Field.to_numpy values per field.

Parameters:

**kwargs (dict, optional) – Keyword arguments passed to data.core.fieldlist.Field.to_numpy

Returns:

Array containing the field values.

Return type:

ndarray

See also

to_array, values

abstract to_pandas(**kwargs)

Convert into a pandas dataframe

to_points(**kwargs)

Return the geographical coordinates shared by all the fields in the data’s original Coordinate Reference System (CRS).

Parameters:

**kwargs (dict, optional) – Keyword arguments passed to Field.to_points()

Returns:

Dictionary with items “x” and “y”, containing the arrays of the x and y coordinates, respectively.

Return type:

dict

Raises:

ValueError – When not all the fields have the same grid geometry.

to_target(target, *args, **kwargs)

Write data into the specified target.

to_tensor(*args, **kwargs)

abstract to_xarray(**kwargs)

Convert into an xarray dataset

unique_values(*coords, remapping=None, patches=None, progress_bar=False)

Given a list of metadata attributes, such as date, param, levels, returns the list of unique values for each attributes

property values

Get all the fields’ values as a 2D array. It is formed as the array of GribField.values per field.

See also

to_array

>>> import earthkit.data
>>> ds = earthkit.data.from_source("file", "docs/examples/test.grib")
>>> for f in ds:
...     print(f.values.shape)
...
(209,)
(209,)
>>> v = ds.values
>>> v.shape
(2, 209)
>>> v[0][:3]
array([262.78027344, 267.44726562, 268.61230469])

Type:

array-like

write(f, **kwargs)

Write all the fields to a file object.

Parameters:

• f (file object) – The target file object.

• **kwargs (dict, optional) – Other keyword arguments passed to the underlying field implementation.

See also

read