earthkit.data.utils.patterns¶

Attributes¶

`FUNCTIONS`
`RE1`
`RE2`
`TYPES`

Classes¶

`Any`	Represents a type that accepts any value.
`Constant`	Represents a constant value in a pattern.
`Datetime`	Represents a type that accepts datetime values.
`DatetimeDelta`	Represents a type that accepts datetime deltas.
`Enum`	Represents a type that accepts a value from a predefined set of strings.
`Float`	Represents a type that accepts float values.
`Function`	Represents a function applied to a variable in a pattern.
`HivePattern`	Hive pattern.
`Int`	Represents a type that accepts integer values.
`Pattern`	Represents a pattern with variables and constants.
`Str`	Represents a type that accepts string values.
`Variable`	Represents a variable in a pattern.

Module Contents¶

class earthkit.data.utils.patterns.Any¶

Represents a type that accepts any value.

substitute(value, name)¶

Substitute the value without any validation.

Parameters:

value (Any) – The value to substitute.
name (str) – The name of the parameter.

Returns:

The substituted value.

Return type:

Any

class earthkit.data.utils.patterns.Constant(value)¶

Represents a constant value in a pattern.

Parameters:: value (Any) – The constant value.

name: str | None = None¶

pattern()¶

substitute(params, **kwargs)¶

Substitute the constant value.

Parameters:: params (dict) – Parameters to substitute (not used for constants).
Returns:: The constant value.
Return type:: Any

substitute_many(params, **kwargs)¶

value: Any¶

class earthkit.data.utils.patterns.Datetime(format)¶

Represents a type that accepts datetime values.

Parameters:: format (str) – Format string for datetime substitution.

format: str¶

substitute(value, name)¶

Substitute the value as a formatted datetime string.

Parameters:

value (Any) – The value to substitute.
name (str) – The name of the parameter.

Returns:

The substituted datetime string.

Return type:

str

class earthkit.data.utils.patterns.DatetimeDelta(params)¶

Represents a type that accepts datetime deltas.

Parameters:: params (str) – Parameters for delta and format, separated by a semicolon.

delta¶

Timedelta string for datetime substitution. Accepted formats use the following suffixes: - “h” for hours (the default) - “m” for minutes - “s” for seconds

Type:: str

format¶

Format string for datetime substitution.

Type:: str

delta¶

format¶

substitute(value, name)¶

Substitute the datetime value, add the delta and format the result.

Parameters:

value (Any) – The value to substitute. Must be convertible to a datetime object.
name (str) – The name of the parameter.

Returns:

The substituted datetime value with the delta added to it and formatted.

Return type:

str

Raises:

ValueError – If the delta format is invalid.

class earthkit.data.utils.patterns.Enum(enum='')¶

Represents a type that accepts a value from a predefined set of strings.

Parameters:: enum (str, optional) – Comma-separated string of allowed values, by default “”.

enum: set[str]¶

substitute(value, name)¶

Substitute the value if it is in the predefined set.

Parameters:

value (str) – The value to substitute.
name (str) – The name of the parameter.

Returns:

The substituted value.

Return type:

str

Raises:

ValueError – If the value is not in the predefined set.

earthkit.data.utils.patterns.FUNCTIONS¶

class earthkit.data.utils.patterns.Float(format='%g')¶

Represents a type that accepts float values.

Parameters:: format (str, optional) – Format string for float substitution, by default “%g”.

format: str = '%g'¶

substitute(value, name)¶

Substitute the value if it is a float or integer.

Parameters:

value (int or float) – The value to substitute.
name (str) – The name of the parameter.

Returns:

The substituted value.

Return type:

str

Raises:

ValueError – If the value is not a float or integer.

class earthkit.data.utils.patterns.Function(value)¶

Represents a function applied to a variable in a pattern.

Parameters:: value (str) – The function definition string.

functions¶

name¶

pattern()¶

substitute(params, **kwargs)¶

Substitute the variable and apply functions.

Parameters:: params (dict) – Parameters to substitute.
Returns:: The substituted and transformed value.
Return type:: Any

substitute_many(params, **kwargs)¶

variable¶

class earthkit.data.utils.patterns.HivePattern(pattern, values=None)¶

Hive pattern.

Parameters:

pattern (str) – The hive pattern string.
values (dict, optional) – Dictionary of values for substitution, by default None.

pattern¶

The original pattern string.

Type:: str

root¶

The root directory of the pattern. This is the beginning of the path not containing any pattern parameters.

Type:: str

rest¶

The remaining part of the pattern after the root.

Type:: str

params¶

List of parameters in the pattern.

Type:: list

dynamic_params¶

List of parameters without user specified values in the pattern.

Type:: list

fixed_single_params¶

Dictionary of user specified single value parameters.

Type:: dict

fixed_multi_keys¶

Dictionary of user specified multi value parameters.

Type:: dict

parts¶

List of pattern parts. Each part is a Pattern object representing a part of the path.

Type:: list

collect(file, part, params)¶

dynamic_params = []¶

fixed_multi_params¶

fixed_single_params¶

params = []¶

parts = []¶

pattern¶

rest = ''¶

root = ''¶

scan(*args, **kwargs)¶

Scan the file system for files matching the pattern.

Parameters:

args (tuple of dicts) – Positional dictionaries of parameters for scanning.
kwargs (dict) – Additional keyword arguments for scanning.

Returns:

List of file paths matching the pattern.

Return type:

list

Raises:

ValueError – If a parameter is not valid for the pattern.

class earthkit.data.utils.patterns.Int(format='%d')¶

Represents a type that accepts integer values.

Parameters:: format (str, optional) – Format string for integer substitution, by default “%d”.

format: str = '%d'¶

substitute(value, name)¶

Substitute the value if it is an integer.

Parameters:

value (int) – The value to substitute.
name (str) – The name of the parameter.

Returns:

The substituted value.

Return type:

str

Raises:

ValueError – If the value is not an integer.

class earthkit.data.utils.patterns.Pattern(pattern)¶

Represents a pattern with variables and constants.

Parameters:: pattern (str) – The pattern string.

is_constant()¶

match(value)¶

Match pattern regex against value.

Parameters:: value (str) – Value to match
Returns:: re.Match object if the value matches the pattern. None otherwise.
Return type:: re.Match

Example

>>> p = Pattern("t_{my_date:date(%Y-%m-%d)}.grib")
>>> p.match("t_2000-01-01.grib")
<re.Match object; span=(0, 17), match='t_2000-01-01.grib'>

>>> p = Pattern("t_{my_date:date(%Y-%m-%d)}.grib")
>>> p.match("2000-01-01.grib")
None

>>> p = Pattern("{shortName}_{my_date:date(%Y-%m-%d)}.grib")
>>> p.match("t_2000-01-01.grib")
<re.Match object; span=(0, 17), match='t_2000-01-01.grib'>

>>> p = Pattern("data/t/level")
>>> p.match("data/t/level")
<re.Match object; span=(0, 12), match='data/t/level'>
>>> p.match("data/t/level/500")
None

property names: List[str]¶

pattern = []¶

regex()¶

substitute(*args, allow_extra=False, **kwargs)¶

Substitute values into the pattern.

Parameters:

args (tuple of dict) – Positional dictionaries of parameters to substitute.
allow_extra (bool, optional) – Whether to allow using input with parameters not part of the Pattern.
kwargs (dict) – Additional keyword arguments for substitution.

Returns:

The substituted pattern as a string or list of strings.

Return type:

str or list

Raises:

ValueError – If there are unused parameters and allow_extra is False.

variables = []¶

earthkit.data.utils.patterns.RE1¶

earthkit.data.utils.patterns.RE2¶

class earthkit.data.utils.patterns.Str(format='%s')¶

Represents a type that accepts string values.

Parameters:: format (str, optional) – Format string for string substitution, by default “%s”.

format: str = '%s'¶

substitute(value, name)¶

Substitute the value if it is a string.

Parameters:

value (str) – The value to substitute.
name (str) – The name of the parameter.

Returns:

The substituted value.

Return type:

str

Raises:

ValueError – If the value is not a string.

earthkit.data.utils.patterns.TYPES¶

class earthkit.data.utils.patterns.Variable(value)¶

Represents a variable in a pattern.

Parameters:: value (str) – The variable definition string.

name¶

pattern()¶

substitute(params)¶

Substitute value for a parameter.

Parameters:: params (dict) – The value belonging to self.name in params are substituted into the Variable. The value to substitute must be a single value.
Return type:: Substituted value.
Raises:: ValueError – If self.name is not in params.

Example

>>> v = Variable("my_date:date(%Y-%m-%d)")
>>> v.substitute({"my_date": "2000-01-01"})
'2000-01-01'

>>> v = Variable("my_date:date(%Y-%m-%d)")
>>> v.substitute({"level": "500"})
ValueError: Missing parameter 'my_date'

substitute_many(params)¶

Substitute all values for a parameter.

Parameters:: params (dict) – The values belonging to self.name in params are substituted into the Variable.
Returns:: List of substituted values.
Return type:: list
Raises:: ValueError – If self.name is not in params.

Example

>>> v = Variable("my_date:date(%Y-%m-%d)")
>>> v.substitute_many({"my_date": ["2000-01-01", "2000-01-02"]})
['2000-01-01', '2000-01-02']

>>> v = Variable("my_date:date(%Y-%m-%d)")
>>> v.substitute_many({"my_date": "2000-01-01"})
['2000-01-01']

>>> v = Variable("my_date:date(%Y-%m-%d)")
>>> v.substitute_many({"level": "500"})
ValueError: Missing parameter 'my_date'

value¶