to_target

to_target(name, *args, data=None, **kwargs)

Write data to a target.

The function can be either invoked on a data object or the data object can be specified as data.

Parameters:

  • name (str) – the target (see below)

  • *args (tuple) –

    specify target parameters

  • data – specify the data object to write. Cannot be set when to_target() is called on a data object.

  • **kwargs (dict) –

    specify additional target parameters. Also specify the encoder parameters.

Built in targets

earthkit-data has the following built-in targets:

Name

Description

Class

file

write data to a file/files

FileTarget

file-pattern

write data to multiple files based on a filename pattern

FilePatternTarget

fdb

add data to a Fields DataBase (FDB)

FDBTarget

zarr

add data to a zarr store

ZarrTarget

file

to_target("file", file, append=False, data=None, encoder=None, template=None, metadata=None, **kwargs)

The file target writes data into a file.

Parameters:

  • file (str, file-like object, None) – The file path or file-like object to write to. When None, tries to guess the file name from the data if it is passed as a kwarg. When the file name cannot be constructed, a ValueError is raised. When file is a path, a file object is automatically created and closed when the target is closed. When file is a file-like object, its ownership is not transferred to the target. As a consequence, the file-like object is not closed when the writing is finished and to_target() returns.

  • append (bool) – If True, the file is opened in append mode. Only used if file is a path.

  • data – specify the data to write. Cannot be set when to_target() is called on a data object.

  • encoder (str, Encoder, None) – The encoder to use to encode the data. When it is a str, the encoder is looked up in the available encoders. When None, the encoder type will be determined from the data to write (if possible), from the target file suffix or from the Target properties. When a suitable encoder cannot be instantiated a ValueError is raise.

  • template (obj, None) – The template to be used by the encoder.

  • **kwargs (dict) –

    other keyword arguments passed to the encoder

import earthkit.data as ekd

# read GRIB data into a fieldlist.
ds = ekd.from_source("sample", "test.grib").to_fieldlist()

# write first field
ds[0].to_target("file", "_my_res_1.grib")

# write whole fieldlist
ds.to_target("file", "_my_res_2.grib")

Notebook examples:

file-pattern

to_target("file-pattern", file, append=False, data=None, encoder=None, template=None, metadata=None, **kwargs)

The file-pattern target writes data into multiple files based on a filename pattern.

Parameters:

  • file (str) – The file path to write to. The output file name defines a pattern containing metadata keys in the format of {key}. Each data item (e.g. a field) will be written into a file with a name created by substituting the relevant metadata values in the filename pattern.

  • append (bool) – If True, the files are opened in append mode.

  • data – specify the data to write. Cannot be set when to_target() is called on a data object.

  • encoder (str, Encoder, None) – The encoder to use to encode the data. When it is a str, the encoder is looked up in the available encoders. When None, the encoder type will be determined from the data to write (if possible), from the target file suffix or from the Target properties. When a suitable encoder cannot be instantiated a ValueError is raised.

  • template (obj, None) – The template to be used by the encoder.

  • **kwargs (dict) –

    other keyword arguments passed to the encoder

import earthkit.data as ekd

# read GRIB data into a fieldlist.
# Contains 2 fields: msl and 2t
ds = ekd.from_source("sample", "test.grib").to_fieldlist()

# this code results in 2 files: _my_res_msl.grib and _my_res_2t.grib
ds.to_target("file-pattern", "_my_res_{shortName}.grib")

Notebook examples:

fdb

to_target("fdb", fdb=None, config=None, userconfig=None, data=None, encoder=None, template=None, metadata=None, **kwargs)

The fdb target writes to an FDB (Fields DataBase), which is a domain-specific object store developed at ECMWF for storing, indexing and retrieving GRIB data. earthkit-data uses the pyfdb package to add data to FDB.

Parameters:

  • fdb (pyfdb.FDB, None) – the FDB to write to

  • config (dict,str) – the FDB configuration directly passed to pyfdb.FDB(). If not provided, the configuration is either read from the environment or the default configuration is used. Only used if no fdb is specified.

  • userconfig (dict,str) – the FDB user configuration directly passed to pyfdb.FDB(). If not provided, the configuration is either read from the environment or the default configuration is used. Only used if no fdb is specified.

  • data – specify the data to write. Cannot be set when to_target() is called on a data object.

  • encoder (str, Encoder, None) – The encoder to use to encode the data. When it is a str, the encoder is looked up in the available encoders. When None, the encoder type will be determined from the data to write (if possible) or from the Target properties. When a suitable encoder cannot be instantiated a ValueError is raised.

  • template (obj, None) – The template to be used by the encoder.

  • **kwargs (dict) –

    other keyword arguments passed to the encoder

import earthkit.data as ekd

ds = ekd.from_source("sample", "tuv_pl.grib").to_fieldlist()

# config contains the FDB configuration

# writing a field
ds[0].to_target("fdb", config=config)

# writing a whole fieldlist
ds.to_target("fdb", config=config)

Notebook examples:

zarr

to_target("zarr", earthkit_to_xarray_kwargs=None, xarray_to_zarr_kwargs=None, data=None)

The zarr target writes to a Zarr store.

Parameters:

  • earthkit_to_xarray_kwargs (dict) – the keyword arguments passed to the to_xarray() function. If not provided, the default values are used.

  • xarray_to_zarr_kwargs (dict) – the keyword arguments passed to the xarray.Dataset.to_zarr() function. As a bare minimum, the store keyword argument must be provided.

  • data – specify the data to write. Cannot be set when to_target() is called on a data object.

This target converts the data to an xarray.Dataset and then writes it to a Zarr store using the xarray.Dataset.to_zarr() function. The conversion to an Xarray dataset is done by the to_xarray() function.

Notebook examples: