earthkit.data.readers.geotiff.fieldlist¶

Classes¶

GeoTIFFFieldList

A list of GeoTIFF bands.

Module Contents¶

class earthkit.data.readers.geotiff.fieldlist.GeoTIFFFieldList(path, **kwargs)¶

Bases: earthkit.data.indexing.simple.SimpleFieldListBase, earthkit.data.readers.geotiff.reader.GeoTIFFReaderBase

A list of GeoTIFF bands.

property appendable¶

batched(n)¶

Iterate through the fieldlist in batches of n fields.

Parameters:: n (int) – Batch size.
Returns:: Returns an iterator yielding batches of n fields. Each batch is a new fieldlist containing a view to the data in the original object, so no data is copied. The last batch may contain fewer than n fields.
Return type:: object

property binary¶

data(keys=('lat', 'lon', 'value'), flatten=False, dtype=None, index=None)¶

Return the values and/or the geographical coordinates.

Only works when all the fields have the same grid geometry.

Parameters:

keys (str, list or tuple) – Specifies the type of data to be returned. Any combination of “lat”, “lon” and “value” is allowed here.
flatten (bool) – When it is True the “lat”, “lon” arrays and the “value” arrays per field will all be flattened. Otherwise they will preserve the field’s shape.
dtype (str, array.dtype or None) – Typecode or data-type of the arrays. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.
index (array indexing object, optional) – The index of the values to be extracted from each field. When it is None all the values are extracted.

Returns:

The elements of the array (in the order of the keys) are as follows:

the latitudes array from the first field when “lat” is in keys
the longitudes array from the first field when “lon” is in keys
a values array per field when “values” is in keys

Return type:

array-like

Raises:

ValueError – When not all the fields have the same grid geometry.

See also

geography, to_numpy, values

Examples

How-to examples:

More examples:

>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "test6.grib").to_fieldlist()
>>> len(fl)
6
>>> d = fl.data()
>>> d.shape
(8, 7, 12)
>>> d[0, 0, 0]  # first latitude
90.0
>>> d[1, 0, 0]  # first longitude
0.0
>>> d[2:, 0, 0]  # first value per field
array([272.56417847,  -6.28688049,   7.83348083, 272.53916931,
        -4.89837646,   8.66096497])
>>> d = fl.data(keys="lon")
>>> d.shape
(1, 7, 12)
>>> d[0, 0, 0]  # first longitude
0.0

describe(*args, **kwargs)¶: Generate a summary of the fieldlist.

property filter¶

static from_fields(fields=None)¶

Create a fieldlist from the given fields.

Parameters:: fields (iterable, Field, None) – Iterable of Field objects. When it is None or empty, an empty fieldlist is returned.
Returns:: A fieldlist object containing the given fields. When fields is None or empty, an EmptyFieldList is returned. Otherwise a SimpleFieldList is returned.
Return type:: SimpleFieldList, EmptyFieldList

property geography¶

Return the geography of the FieldList.

Returns:: If the fields in the FieldList have the same grid geometry, the returned geography is the one of the first field. Otherwise an error is raised.
Return type:: GeographyBase
Raises:: ValueError – When not all the fields have the same grid geometry or the FieldList is empty.

get(keys, default=None, astype=None, raise_on_missing=False, output='auto', group_by_key=False, flatten_dict=False, remapping=None, patch=None)¶

Return values for the specified keys from all the fields.

Parameters:

keys (str, list, tuple) – Specify the field metadata keys to extract. Can be a single key (str) or multiple keys as a list/tuple of str. Keys are assumed to be of the form “component.key”. For example, “time.valid_datetime” or “parameter.name”. Keys from the raw field metadata (if any) can be accessed using the “metadata.key” syntax. For example, when a Field was created from a GRIB message, the ecCodes GRIB keys can be accessed as “metadata.shortName” or “metadata.level”.
default (Any, None) – Specify the default value(s) for keys. Returned when the given key is not found and raise_on_missing is False. When default is a single value, it is used for all the keys. Otherwise it must be a list/tuple of the same length as keys.
astype (type as str, int or float) – Return type for keys. When astype is a single type, it is used for all the keys. Otherwise it must be a list/tuple of the same length as keys.
raise_on_missing (bool) – When True, raises KeyError if any of keys is not found.
output (type, str) –
Specify the output structure type in conjunction with group_by_key. When group_by is False (default) the output is a list with one item per field and output has the following effect on the items:
- ”auto” (default):
  - when keys is a str returns a single value per field
  - when keys is a list/tuple returns a list/tuple of values per field
- list or “list”: returns a list of values per field.
- tuple or “tuple”: returns a tuple of values per field.
- dict or “dict”: returns a dictionary with keys and their values per field.
When group_by_key is True the output is grouped by key and return an object with one item per key. The item per key contains the list of values for that key from all the fields. When output is “dict” a dict is returned, otherwise a list.
group_by_key (bool) – When True the output is grouped by key as described in output.
flatten_dict (bool) – When True and output is dict, for each field if any of the values in the returned dict is itself a dict, it is flattened to depth 1 by concatenating the keys with a dot. For example, if the returned dict is {"a": {"x": 1, "y": 2}, "b": 3}, it becomes {"a.x": 1, " a.y": 2, "b": 3}. This option is ignored when output is not dict.
remapping (dict, optional) –
Create new metadata keys from existing ones. E.g. to define a new key “param_level” as the concatenated value of the “parameter.variable” and “vertical.level” keys use:
```
remapping={"param_level": "{parameter.variable}{vertical.level}"}
```
patch (dict, optional) – A dictionary of patch to be applied to the returned values.

Returns:

The returned value depends on the output and group_by_key parameters. See above.

Return type:

list, dict

Raises:

KeyError – If raise_on_missing is True and any of keys is not found.

Examples

>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "test.grib").to_fieldlist()
>>> fl.get("parameter.variable")
['2t', 'msl']
>>> fl.get(["parameter.variable", "parameter.units"])
[('2t', 'K'), ('msl', 'Pa')]
>>> fl.get(("parameter.variable", "parameter.units"))
[['2t', 'K'], ['msl', 'Pa']]

graph(depth=0)¶

group_by(*keys, sort=True)¶

Iterate through the fieldlist in groups defined by metadata keys.

Parameters:

*keys (tuple) – Positional arguments specifying the metadata keys to group by. Keys can be a single or multiple str, or a list or tuple of str.
sort (bool, optional) – If True (default), the fieldlist is sorted by the metadata keys before grouping.

Returns:

Returns an iterator yielding batches of fields grouped by the metadata keys. Each batch is a new fieldlist containing a view to the data in the original object, so no data is copied. It generates a new group every time the value of the keys change.

Return type:

object

head(n=5, **kwargs)¶

Generate a list like summary of the first n fields.

Same as calling ls with n.

Parameters:

n (int, None) – The number of fields (n > 0) to be printed from the front.
**kwargs (dict, optional) – Other keyword arguments passed to ls.

Returns:

See ls.

Return type:

Pandas DataFrame

See also

ls, tail

Notes

The following calls are equivalent:

fl.head()
fl.head(5)
fl.head(n=5)
fl.ls(5)
fl.ls(n=5)

ignore()¶

Indicates to ignore this source in concatenation/merging.

Return type:: bool

ls(n=None, keys='default', extra_keys=None, collections=None)¶

Generate a list like summary using a set of metadata keys.

Parameters:

n (int, None) – The number of fields to be listed. None means all the fields, n > 0 means fields from the front, while n < 0 means fields from the back of the fieldlist.
keys (list of str, dict, None) – The metadata keys to extract. If keys="default", a built-in default set of keys is used. To specify a column title for each key in the output use a dict as a mapping from the keys to the column titles.
extra_keys (list of str, dict, None) – List of additional keys on top of keys. To specify a column title for each key in the output use a dict as a mapping from the keys to the column titles.
collections (str, list of str, None) – The collections to extract. Can be a single collection (str) or multiple collections as a list of str. A collection is a component of the field (e.g. “time”, “parameter”, “geography”, etc.) as a dictionary. It can also be a collection within the raw “metadata” component. For example, when a Field was created from a GRIB message, the ecCodes GRIB “namespaces” can be accessed as collections, e.g. “metadata.mars” means the ecCodes GRIB “mars” namespace.

Returns:

DataFrame with one row per Field.

Return type:

Pandas DataFrame

See also

head, tail

classmethod merge(sources)¶

property merger¶

metadata(keys, **kwargs)¶

Return the raw metadata values for each field.

Parameters:

*args (tuple) – Positional arguments defining the metadata keys. Passed to Field.metadata()
**kwargs (dict, optional) – Keyword arguments passed to Field.metadata()

Returns:

List with one item per Field

Return type:

list

Examples

>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "test.grib").to_fieldlist()
>>> fl.metadata("param")
['2t', 'msl']
>>> fl.metadata("param", "units")
[('2t', 'K'), ('msl', 'Pa')]
>>> fl.metadata(["param", "units"])
[['2t', 'K'], ['msl', 'Pa']]

mutate()¶

mutate_source()¶

name = None¶

classmethod new_mask_index(*args, **kwargs)¶

order_by(*args, remapping=None, patch=None, **kwargs)¶

Change the order of the fields in a fieldlist.

Parameters:

*args (tuple) – Positional arguments specifying the metadata keys to perform the ordering on. Each argument can be a single key (str) or multiple keys as a list/tuple of str or a dictionary. Any metadata keys that earthkit.data.core.field.Field.get() accepts can be used here. The order of the keys defines the priority of the ordering. When a dictionary is used it must specify the ordering direction or the order of the values for each key. The ordering direction can be either “ascending” or “descending” (the default is “ascending”). The order of values for a key is defined by a list of values for that key, which must include all the available values for that key in the fieldlist. See the examples below for more details.
remapping (dict) –
Define new metadata keys from existing ones to use in *args and **kwargs. E.g. to define a new key “param_level” as the concatenated value of the “parameter.variable” and “vertical.level” keys use:
```
remapping={"param_level": "{parameter.variable}{vertical.level}"}
```
See below for a more elaborate example.
**kwargs (dict, optional) – Other keyword arguments specifying the metadata keys to perform the ordering on. Used in the same way as a dictionary in *args.

Returns:

Returns a MaskFieldList with the reordered fields. It provides a view to the data in the original object, so no data is copied. When called without any arguments it returns the original fieldlist.

Return type:

MaskFieldList, FieldList

Examples

How-to examples:

GRIB: ordering fields

Ordering by a single metadata key (“parameter.variable”). The default ordering direction is ascending:

>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "test6.grib").to_fieldlist()
>>> for f in fl.order_by("parameter.variable"):
...     print(f)
...
Field(t,850,20180801,1200,0,0)
Field(t,1000,20180801,1200,0,0)
Field(u,850,20180801,1200,0,0)
Field(u,1000,20180801,1200,0,0)
Field(v,850,20180801,1200,0,0)
Field(v,1000,20180801,1200,0,0)

Ordering by multiple keys (first by “vertical.level” then by “parameter.variable”). The default ordering direction is ascending for both keys:

>>> for f in fl.order_by(["vertical.level", "parameter.variable"]):
...     print(f)
...
Field(t,850,20180801,1200,0,0)
Field(u,850,20180801,1200,0,0)
Field(v,850,20180801,1200,0,0)
Field(t,1000,20180801,1200,0,0)
Field(u,1000,20180801,1200,0,0)
Field(v,1000,20180801,1200,0,0)

Specifying the ordering direction:

>>> for f in fl.order_by(
...     {"parameter.variable": "ascending", "vertical.level": "descending"}
... ):
...     print(f)
Field(t,1000,20180801,1200,0,0)
Field(t,850,20180801,1200,0,0)
Field(u,1000,20180801,1200,0,0)
Field(u,850,20180801,1200,0,0)
Field(v,1000,20180801,1200,0,0)
Field(v,850,20180801,1200,0,0)

Using the list of all the values of a key (“parameter.variable”) to define the order:

>>> for f in fl.order_by({"parameter.variable": ["u", "t", "v"]}):
...     print(f)
Field(u,1000,20180801,1200,0,0)
Field(u,850,20180801,1200,0,0)
Field(t,1000,20180801,1200,0,0)
Field(t,850,20180801,1200,0,0)
Field(v,1000,20180801,1200,0,0)
Field(v,850,20180801,1200,0,0)

Using remapping to specify the order by a key created from two other keys (we created key “param_level” from “parameter.variable” and “vertical.level”):

>>> ordering = ["t850", "t1000", "u1000", "v850", "v1000", "u850"]
>>> remapping = {"param_level": "{parameter.variable}{vertical.level}"}
>>> for f in fl.order_by({"param_level": ordering}, remapping=remapping):
...     print(f)
Field(t,850,20180801,1200,0,0)
Field(t,1000,20180801,1200,0,0)
Field(u,1000,20180801,1200,0,0)
Field(v,850,20180801,1200,0,0)
Field(v,1000,20180801,1200,0,0)
Field(u,850,20180801,1200,0,0)

property parent¶: The parent source, if any.

property parts¶

path¶

sel(*args, remapping=None, **kwargs)¶

Select the fields matching the given metadata conditions.

Parameters:

*args (tuple) –
Positional arguments specifying the filter conditions as a dict. Both single or multiple keys are allowed to use. When multiple filter conditions are specified, they are combined with a logical AND operator. Each metadata key in the filter conditions can specify the following type of filter values:
- single value:
```
fl.sel({parameter.variable: "t"})
```
- list of values:
```
fl.sel({parameter.variable: ["u", "v"]})
```
- slice of values (defines a closed interval, so treated as inclusive of both the start
and stop values, unlike normal Python indexing). The following example filters the fields with “vertical.level” between 300 and 500 inclusively:
```
fl.sel({vertical.level: slice(300, 500)})
```
Date and time related keys from the “time” field component are automatically normalised for comparison. This is also applied to the following keys from the raw metadata: “metadata.base_datetime”, “metadata.valid_datetime” and “metadata.step_timedelta”.

For example, when filtering by “time.valid_datetime” the following calls are equivalent:
```
>>> fl.sel({ "time.valid_datetime": "2018-08-01T12:00:00"})
>>> fl.sel({ "time.valid_datetime": datetime(2018, 8, 1, 12, 0) })
```
Similarly, when filtering by “time.step” the following calls are equivalent (values are assumed to be in hours when the unit is not specified):
```
>>> fl.sel({ "time.step": "6h"})
>>> fl.sel({ "time.step": 6})
>>> fl.sel({ "time.step": "360m"})
>>> fl.sel({ "time.step": timedelta(hours=6)})
```
remapping (dict) –
Define new metadata keys from existing ones to use in *args and **kwargs. E.g. to define a new key “param_level” as the concatenated value of the “parameter.variable” and “vertical.level” keys use:
```
>>> remapping={"param_level": "{parameter.variable}{vertical.level}"}
```
See below for a more elaborate example.
**kwargs (dict, optional) – Other keyword arguments specifying the filter conditions.

Returns:

Returns a MaskFieldList with the reordered fields. It provides a view to the data in the original object, so no data is copied. When called without any arguments it returns the original fieldlist.

Return type:

MaskFieldlist, FieldList

Examples

How-to examples:

GRIB: selection using metadata

More examples:

>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "tuv_pl.grib").to_fieldlist()
>>> len(fl)
18

Selecting by a single key (“parameter.variable”) with a single value:

>>> fl1 = fl.sel({parameter.variable: "t"})
>>> for f in fl1:
...     print(f)
...
Field(t,1000,20180801,1200,0,0)
Field(t,850,20180801,1200,0,0)
Field(t,700,20180801,1200,0,0)
Field(t,500,20180801,1200,0,0)
Field(t,400,20180801,1200,0,0)
Field(t,300,20180801,1200,0,0)

Selecting by multiple keys (“parameter.variable”, “vertical.level”) with a list and slice of values:

>>> fl1 = fl.sel(
...     {parameter.variable: ["u", "v"], vertical.level: slice(400, 700)}
... )
>>> for f in fl1:
...     print(f)
...
Field(u,700,20180801,1200,0,0)
Field(v,700,20180801,1200,0,0)
Field(u,500,20180801,1200,0,0)
Field(v,500,20180801,1200,0,0)
Field(u,400,20180801,1200,0,0)
Field(v,400,20180801, 1200,0,0)

Using remapping to specify the selection by a key created from two other keys (we created key “param_level” from “parameter.variable” and “vertical.level”):

>>> fl1 = fl.sel(
...     param_level=["t850", "u1000"],
...     remapping={"param_level": "{parameter.variable}{vertical.level}"},
... )
>>> for f in fl1:
...     print(f)
...
Field(u,1000,20180801,1200,0,0)
Field(t,850,20180801,1200,0,0)

set(*args, **kwargs)¶

Set the metadata values for each field.

All the arguments are passed to earthkit.data.core.field.Field.set() for each field in the fieldlist.

Parameters:

*args (tuple) – Positional arguments defining the metadata keys and values.
**kwargs (dict, optional) – Keyword arguments defining the metadata keys and values.

Returns:

New FieldList with the updated metadata values.

Return type:

FieldList

See also

earthkit.data.core.field.Field.set()

property source¶

source_filename = None¶

property stream¶

tail(n=5, **kwargs)¶

Generate a list like summary of the last n fields.

Same as calling ls with -n.

Parameters:

n (int, None) – The number of fields (n > 0) to be printed from the back.
**kwargs (dict, optional) – Other keyword arguments passed to ls.

Returns:

See ls.

Return type:

Pandas DataFrame

See also

head, ls

Notes

The following calls are equivalent:

fl.tail()
fl.tail(5)
fl.tail(n=5)
fl.ls(-5)
fl.ls(n=-5)

to_array(**kwargs)¶

Return the values of all the fields as an array.

It is formed as the array of the earthkit.data.core.field.Field.to_array values per field.

Parameters:

flatten (bool) – When it is True the values are flattened per field. Otherwise an array with the field’s shape is returned per field.
dtype (str, array.dtype or None) – Typecode or data-type of the array. When it is None the default type used by the underlying data accessor is used. For GRIB it is float64.
copy (bool) – When it is True a copy of the data values per field is returned. Otherwise a view is returned where possible.
array_namespace (str, array_namespace or None) – The array namespace to be used. When it is None the underlying array format of the field is used. For GRIB it is “numpy”.
device (str or None) – The device where the array will be allocated. When it is None the default device is used.
index (array indexing object, optional) – The index of the values to be extracted per field. When it is None all the values are extracted. is None all the values are extracted.

Returns:

Array containing the field values. It is formed as the array of values extracted by earthkit.data.core.field.Field.to_array per field.

Return type:

array-like

Examples

How-to examples:

GRIB: using array namespaces

to_cube(*args, **kwargs)¶

Convert to a cube-like structure.

This method is intended to support fieldlist usage in the anemoi-datasets package. Planned to be removed in the future and use to_tensor instead.

to_data_object()¶: Convert this source into a data object, if possible.

to_fieldlist(array_namespace=None, device=None, flatten=False, dtype=None)¶

Change how the values stored in each field in a fieldlist.

This method converts the data values in each field to an array with a given array_namespace and/or device. The resulting fieldlist is then composed of the converted fields. The field values are extracted using the earthkit.data.core.field.Field.to_array() method of each field.

When a field already stores its values as an array with a matching array_namespace and device, a copy of that array is made and stored in the resulting field. This means that even if called without any arguments, the resulting fieldlist will have its own copy of the data values.

The primary use of this method is to convert the values in GRIB fields loaded from disk to in-memory arrays. By default, the values in GRIB fields loaded from disk are not stored as arrays in memory, but rather as references to the on-disk data. This allows for efficient access to the data without loading it all into memory at once. However, in some cases it may be desirable to convert these values to in-memory arrays for faster access or for compatibility with other libraries. This method provides a way to do that while preserving the metadata of the fields.

Parameters:

array_namespace (str, array_namespace or None) – The array namespace to be used for the field values in the resulting fieldlist. When it is None, the default array namespace used by the underlying data accessor of the field is used. For GRIB it is “numpy”.
device (str or None) – The device where the array will be allocated. When it is None the default device is used.
flatten (bool) – When it is True the values are flattened per field. Otherwise the array will have the field’s shape per field.
dtype (str, array.dtype or None) – Typecode or data-type of the array. When it is None the default type used by the underlying data accessor per field is used. For GRIB it is float64.

Returns:

New fieldlist formed from the converted fields.

Return type:

SimpleFieldList

Examples

How-to examples:

GRIB: using array namespaces

The following example will convert a fieldlist read from a GRIB file into a SimpleFieldList storing data values as single precision arrays in each field.

>>> import numpy as np
>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "tuv_pl.grib").to_fieldlist()
>>> r = fl.to_fieldlist(array_namespace="numpy", dtype=np.float32)
>>> r.to_numpy().dtype
dtype('float32')

to_numpy(**kwargs)¶

Return the values of all the fields as a Numpy array.

Parameters:

flatten (bool) – When it is True the values are flattened per field. Otherwise an ndarray with the field’s shape is returned per field.
dtype (str, numpy.dtype or None) – Typecode or data-type of the array. When it is None the default type used by the underlying data accessor of the field is used. For GRIB it is float64.
copy (bool) – When it is True a copy of the data values per field is returned. Otherwise a view is returned where possible.
index (ndarray indexing object, optional) – The index of the values to be extracted per field. When it is None all the values are extracted.

Returns:

Array containing the field values. It is formed as the array of values extracted by earthkit.data.core.field.Field.to_numpy per field.

Return type:

ndarray

Examples

How-to examples:

GRIB: using array namespaces

to_pandas(latitude=None, longitude=None, **kwargs)¶

to_target(target, *args, **kwargs)¶

to_tensor(*args, **kwargs)¶

Convert to a tensor-like structure.

This method is intended to use internally to support the Xarray engine, which converts fieldlist to a tensor-like structure before converting it to an Xarray Dataset or DataArray.

to_xarray(engine='earthkit', xarray_open_dataset_kwargs=None, **kwargs)¶

Convert the FieldList into an Xarray Dataset.

Parameters:

engine (str, optional) – The Xarray engine to use for generating the dataset. Default value is "earthkit". If set to cfgrib, the cfgirb engine is used, which can only work with GRIB data. No other values are supported.
split_dims (str, or iterable of str, None) – Dimension or list of dimensions to use for splitting the data into multiple hypercubes. Default is None. Only used when engine="earthkit". Please note that split_dims is not a valid option when the Xarray is directly generated via xarray.open_dataset().
xarray_open_dataset_kwargs (dict, optional) – Keyword arguments passed to xarray.open_dataset(). Either this or **kwargs can be used, but not both.
**kwargs (dict, optional) –
Any keyword arguments that can be passed to xarray.open_dataset(). Engine specific keywords are automatically grouped and passed as backend_kwargs. Either **kwargs or xarray_open_dataset_kwargs can be used, but not both.

When engine="earthkit" the following engine specific kwargs are supported:
- profile: str, dict or None
  Provide custom default values for most of the kwargs. The default profile is “earthkit”. An explicit dict can be used. None is equivalent to an empty dict. When a kwarg is specified it will update the corresponding profile value if it is a dict otherwise it will overwrite it. See: Xarray engine: profiles for more information.
- variable_key: str, None
  The metadata key which will be used to name the Xarray Dataset variables. Default is “parameter.variable” (which in the case of GRIB data is the same as “metadata.shortName” and “metadata.param”). The same key cannot be used to define any dimension. Only enabled when mono_variable is False or None.
- drop_variables: str, or iterable of str, None
  A variable or list of variables to drop from the dataset. Default is None. Only used when variable_key is enabled.
- rename_variables: dict, None
  Mapping to rename variables. Default is None. Only used when variable_key is enabled.
- mono_variable: bool, str, None
  If True or str, the dataset will contain a single variable called “data” (or the value of the mono_variable kwarg when it is a str). If False, the dataset will contain one variable for each distinct value of variable_key metadata key. The default value (None) expands to False unless the profile overwrites it.
- extra_dims: str, or list of str, dict or tuple, or None
  Define additional dimensions on top of the predefined dimensions. Only enabled when no fixed_dims is specified. Default is None. It can be a single metadata key or a list. If a list, each item is either a metadata key, or a dict/tuple defining mapping between the dimension name and the metadata key. The whole option can be a dict. E.g.
  # use GRIB key "expver" as a dimension extra_dims = "metadata.expver" # use keys "metadata.expver" and "metadata.steam" as a dimension extra_dims = ["metadata.expver", "metadata.stream"] # define dimensions "expver", "mars_stream" and "mars_type" from # GRIB keys "expver", "stream" and "type" extra_dims = [ "metadata.expver", {"mars_stream": "metadata.stream"}, ("mars_type", "metadata.type"), ] extra_dims = [ { "expver": "metadata.expver", "mars_stream": "metadata.stream", "mars_type": "metadata.type", } ]
- drop_dims: str, or iterable of str, None
  Single or multiple dimensions to be ignored. Default is None. Default is None.
- ensure_dims: str, or iterable of str, None
  Every item may be one of the following:
  - Dimension name: A dimension that must always be preserved in the output, even when squeeze=True and its size is 1, or when it appears in dims_as_attrs.
  - Metadata key: A key whose value defines an additional, non-squeezable dimension. When a metadata key is listed here, it does not need to be repeated in extra_dims.
  Default is None.
- fixed_dims: str, or iterable of str, None
  Define all the dimensions to be generated. When used no other dimensions will be created. Might be incompatible with other settings. Default is None. It can be a single item or a list. Each item is either a metadata key, or a dict/tuple defining mapping between the dimension name and the metadata key. The whole option can be a dict. E.g.:
  # use key "time.step" as a dimension fixed_dims = "time.step" # use keys "time.step" and "vertical.level" as a dimension extra_dims = ["time.step", "vertical.level"] # define dimensions "step", level" and "level_type" from # metadata keys "metadata.step", "metadata.levelist" and "metadata.levtype" extra_dims = [ "metadata.step", {"level": "metadata.levelist"}, ("level_type", "metadata.levtype"), ] extra_dims = [ {"step": "metadata.step", "level": "metadata.levelist", "level_type": "metadata.levtype"} ]
- dim_roles: dict, None
  Specify the “roles” used to form the predefined dimensions. The predefined dimensions are automatically generated when no fixed_dims specified and comprise the following (in a fixed order):
  - ensemble forecast member dimension
  - temporal dimensions (controlled by time_dims)
  - vertical dimensions (controlled by level_dim_mode)
  dim_roles is a mapping between the “roles” and the metadata keys representing the roles. The possible roles are as follows:
  - ”member”: metadata key interpreted as ensemble forecast members
  - ”forecast_reference_time”: metadata key interpreted as forecast reference time. Can be a single metadata key, or a list/tuple of two metadata keys representing the date and time parts of the forecast reference time. Alternatively, it can be a dict with “date” and “time” keys specifying the corresponding metadata keys. Used when "forecast_reference_time" is in time_dims.
  - ”step”: metadata key interpreted as forecast step
  - ”valid_time”: metadata key interpreted as valid time. Used when "valid_time" is in time_dims or add_valid_time_coord is True.
  - ”date”: metadata key interpreted as base date. Used when "date" is in time_dims.
  - ”time”: metadata key interpreted as base time. Used when "time" is in time_dims.
  - ”level”: metadata key interpreted as level
  - ”level_type”: metadata key interpreted as level type
  The default values are as follows:
  { "member": "ensemble.member", "forecast_reference_time": "time.forecast_reference_time", "step": "time.step", "valid_time": "time.valid_datetime", "date": "time.base_date", "time": "time.base_time", "level": "vertical.level", "level_type": "vertical.level_type", }
  dims_roles behaves differently to the other kwargs in the sense that it does not override but update the default values. So e.g. to change only “member” in the default it is enough to specify: “dim_roles={“member”: “metadata.perturbationNumber”}.
- dim_name_from_role_name: bool, None
  If True, the dimension names are formed from the role names. Otherwise, the dimension names are formed from the metadata keys specified in dim_roles. Its default value (None) expands to True unless the profile overwrites it. Only used when no fixed_dims are specified. New in version 0.15.0.
- rename_dims: dict, None
  Mapping to rename dimensions. Default is None.
- dims_as_attrs: str, or iterable of str, None
  A dimension name or a list of dimension names that should be converted into variable attributes when they have only a single value for the corresponding variable. Note that such size-1 dimensions are still preserved if they are explicitly listed in ensure_dims. The default is None.
- time_dims: str, list of str, or None
  Explicitly specify the time dimension(s) to construct, together with their order. Each element is a role name from dim_roles. The default is ["forecast_reference_time", "step"]. Common configurations:
  - ["forecast_reference_time", "step"]: two dimensions for forecast reference time and step (default)
  - ["valid_time"]: a single valid-time dimension
  - ["date", "time", "step"]: three separate raw dimensions
- level_dim_mode: str, None
  Controls how predefined vertical dimensions are constructed. The default is "level". Valid values are:
  - "level": Creates two separate dimensions, "level" and "level_type", as defined by the corresponding roles in dim_roles.
  - "level_per_type": Uses a template dimension "<level_per_type>" that is expanded into one or more vertical dimensions. The dimension name is taken from the metadata key with the role "level_type" (e.g. "pressure"), and the coordinate values come from the metadata key with the role "level" (e.g. [500, 700, 850, 1000]).
  - "level_and_type": Produces a single combined dimension, "level_and_type", in which the level value and the level type are merged.
- squeeze: bool, None
  Remove dimensions which have only one valid value. Not applies to dimensions in ensure_dims. Its default value (None) expands to True unless the profile overwrites it.
- add_valid_time_coord: bool, None
  If True, add the valid_time coordinate containing np.datetime64 values to the dataset. Only takes effect when "valid_time" is not in time_dims. Its default value (None) expands to False unless the profile overwrites it.
- decode_times: bool, None
  If True, decode date and datetime coordinates into datetime64 values. If False, leave the coordinates in their native type (e.g. int if the coordinates come from the GRIB key like “date” or “validityDate”). The default value (None) expands to True unless the profile overwrites it.
- decode_timedelta: bool, None
  If True, decode time-like or duration-like coordinates into timedelta64 values. If False, leave the coordinates in their native type (e.g. int if the coordinates come from the GRIB key like “time”, “validityTime”, “step”); additionally, the duration-like coordinates (e.g. derived from the GRIB key like “step”, “endStep”, etc.) will have the attribute “units” appropriately set (to “minutes”, “hours”, etc.). If None (default), assume the same value of decode_times unless the profile overwrites it.
- aux_coords: dict, None
  Mapping from an auxiliary coordinate label to a tuple: (metadata_key: str, dataset_dimension(s): str or iterable of str). The default value is None.
- add_geo_coords: bool, None
  Add geographic coordinates to the dataset when field values are represented by a single “values” dimension. Its default value (None) expands to True unless the profile overwrites it.
- flatten_values: bool, None
  If True, flatten the values per field resulting in a single dimension called “values” representing a field. If False, the field shape is used to form the field dimensions. When the fields are defined on an unstructured grid (e.g. reduced Gaussian) or are spectral (e.g. spherical harmonics) this option is ignored and the field values are always represented by a single “values” dimension. Its default value (None) expands to False unless the profile overwrites it.
- attrs_mode: str, None
  Define how attributes are generated. Default is “fixed”. The possible values are:
  - ”fixed”: Use the attributes defined in variable_attrs as variables attributes and global_attrs as global attributes.
  - ”unique”: Use all the attributes defined in attrs, variable_attrs and global_attrs. When an attribute from attrs has unique value for a dataset it will be a global attribute, otherwise it will be a variable attribute. However, this logic is only applied if a unique variable attribute can be a global attribute according to the CF conventions Appendix A. (e.g. “units” cannot be a global attribute). Additionally, keys in variable_attrs are always used as variable attributes, while keys in global_attrs are always used as global attributes.
- attrs: str, number, callable, dict or list of these, None
  Attribute or list of attributes. Only used when attrs_mode is unique. Its default value (None) expands to [] unless the profile overwrites it. The following attributes are supported:
  - str: Name of the attribute used as a metadata key to generate the value of the attribute. Can also be specified by prefixing with “key=” (e.g. “key=vertical.level”). When prefixed with “namespace=” it specifies a metadata namespace (e.g. “namespace=parameter”), which will be added as a dict to the attribute.
  - callable: A callable that takes a Field object and returns a dict of attributes, e.g.:
    
    def rounded_wavelength(field): wl = field.get("metadata.wavelength") if wl is not None: return {"wavelength": round(wl)} else: return {}
  - dict: A dictionary of attributes with the keys as the attribute names. If the value is a callable it takes the attribute name and a Field object and returns the value of the attribute, e.g.:
    
    def ensure_rounded(key, field): val = field.get(key) try: return round(val) except Exception: return val
    
    A str value prefixed with “key=” or “namespace=” is interpreted as explained above. Any other values are used as the pre-defined value for the attribute.
- variable_attrs: str, number, callable, dict or list of these, None
  Variable attribute or attributes. For the allowed values see attrs. Its default value (None) expands to [] unless the profile overwrites it.
- global_attrs: str, number, dict or list of these, None
  Global attribute or attributes. For the allowed values see attrs. Its default value (None) expands to [] unless the profile overwrites it.
- coord_attrs: dict, None
  To be documented. Default is None.
- add_earthkit_attrs: bool, None
  If True, add earthkit specific attributes to the dataset. Its default value (None) expands to True unless the profile overwrites it.
- rename_attrs: dict, None
  A dictionary of attribute to rename. Default is None.
- fill_metadata: dict, None
  Define fill values to metadata keys. Default is None.
- remapping: dict, None
  Define new metadata keys for indexing. Any key provided in remapping may be referenced when specifying options such as variable_key, extra_dims, ensure_dims, aux_coords and others. Default is None.
- lazy_load: bool, None
  If True, the resulting Dataset will load data lazily from the underlying data source. If False, a Dataset holding all the data in memory and decoupled from the backend source will be created. Using lazy_load=False with release_source=True can provide optimised memory usage in certain cases. The default value of lazy_load (None) expands to True unless the profile overwrites it.
- release_source: bool, None
  Only used when lazy_load=False. If True, memory held in the input fields are released as soon as their values are copied into the resulting Dataset. This is done per field to avoid memory spikes. The release operation is currently only supported for GRIB fields stored entirely in memory, e.g. when read from a stream. When a field does not support the release operation, this option is ignored. Having run to_xarray the input data becomes unusable, so use this option carefully. The default value of release_source (None) expands to False unless the profile overwrites it.
- allow_holes: bool, None
  If False, GRIB fields must form a full hypercube (without holes). If True, a dataset will be created from any GRIB fields and its coordinates will be a union of coordinates of the fields (outer join). Values corresponding to missing GRIB fields will be filled with NaN. The default value of allow_holes (None) expands to False unless the profile overwrites it.
- strict: bool, None
  If True, perform stricter checks on hypercube consistency. Its default value (None) expands to False unless the profile overwrites it.
- dtype: str, numpy.dtype or None
  Typecode or data-type of the array data.
- array_backend: str, array namespace, None
  The array namespace to use for array operations. The default value (None) is expanded to “numpy”. Deprecated since version 0.19.0. Please use array_namespace instead. In versions before 0.19.0 an ArrayBackend was also accepted here, which is no longer the case.
- array_namespace: str, array namespace, None
  The array namespace to use for array operations. The default value (None) is expanded to “numpy”. New in version 0.19.0.
- direct_backend: bool, None
  If True, the backend is used directly bypassing xarray.open_dataset() and ignoring all non-backend related kwargs. If False, the data is read via xarray.open_dataset(). Its default value (None) expands to False unless the profile overwrites it.
When engine="cfgrib" the following engine specific kwargs are supported:
- ignore_keys: list, None
  It specifies the metadata keys that should be ignored when reading the GRIB messages in the backend. Please note that is not supported by cfgirb, but implemented in earthkit-data.
- For the rest of the supported keyword arguments, please refer to the cfgirb documentation.

Returns:

When split_dims is unset a Dataset is returned. When engine="earthkit" and split_dims is set a tuple is returned. The first element of the tuple is the list of Datasets and the second element is the list of corresponding dictionaries with the spitting keys/values (one dictionary per Dataset).

Return type:

Xarray Dataset or tuple

Notes

The default values of xarray_open_dataset_kwargs or **kwargs passed to xarray.open_dataset() are as follows:

when engine="earthkit":

{"cache": True, "chunks": None, "engine": "earthkit"}

when engine="cfgrib":

{
    "backend_kwargs": {"errors": "raise", "ignore_keys": [], "squeeze": False},
    "cache": True,
    "chunks": None,
    "engine": "cfgrib"
}

Please note that settings errors="raise" and engine are always enforced and cannot be changed.

Examples

>>> import earthkit.data
>>> fs = earthkit.data.from_source("sample", "pl.grib")
>>> ds = fs.to_xarray(time_dims=["forecast_reference_time", "step"])
>>> # also possible to use the xarray_open_dataset_kwargs
>>> ds = fs.to_xarray(
...     xarray_open_dataset_kwargs={
...         "backend_kwargs": {"time_dims": ["forecast_reference_time", "step"]}
...     }
... )

unique(*args, sort=False, drop_none=True, squeeze=False, unwrap_single=False, remapping=None, patch=None, cache=True, progress_bar=False)¶

Return the unique values for a given set of metadata keys.

Parameters:

*args (tuple) – Positional arguments specifying the metadata keys to collect unique values for.
sort (bool, optional) – Whether to sort the collected unique values. Default is False.
drop_none (bool, optional) – Whether to drop None values from the collected unique values. Default is True.
squeeze (bool, optional) – When True only returns the metadata keys that have more than one values. Default is False.
unwrap_single (bool, optional) – When True and only one metadata key is specified, the unique values are returned as a tuple instead of a dict. Default is False.
remapping (dict, optional) – A dictionary for remapping keys or values during collection. Default is None.
patch (dict, optional) – A dictionary for patching key values during collection. Default is None.
cache (bool, optional) – Whether to use an a cache attached to the fieldlist for previously collected unique values. Default is True.
progress_bar (bool, optional) – Whether to display a progress bar during collection. Default is False.

Returns:

A dictionary containing the unique values for the specified metadata keys.

Return type:

dict

property values¶

Return the values of all the fields as a 2D array.

Returns:: Array containing the values of all the fields. The return array is formed as the array of the flattened values extracted from each field by Field.values.
Return type:: array-like

Examples

>>> import earthkit.data as ekd
>>> fl = ekd.from_source("sample", "test.grib").to_fieldlist()
>>> for f in fl:
...     print(f.values.shape)
...
(209,)
(209,)
>>> v = fl.values
>>> v.shape
(2, 209)
>>> v[0][:3]
array([262.78027344, 267.44726562, 268.61230469])