earthkit.data.sources.url¶

Attributes¶

`SCHEME_TO_SOURCE`
`source`

Classes¶

`SingleUrlStream`	Base class for all sources.
`Url`	Represent a URL source.
`UrlBase`	Base class for all sources.
`UrlSourcePathAndParts`

Functions¶

`download_and_cache`(url, *[, owner, parts, verify, ...])
`download_to_target`(url, target, *[, parts, verify, ...])

Module Contents¶

earthkit.data.sources.url.SCHEME_TO_SOURCE¶

class earthkit.data.sources.url.SingleUrlStream(url, **kwargs)¶

Bases: UrlBase

Base class for all sources.

connect_to_mirror(mirror)¶

content_type = None¶

filter = None¶

graph(depth=0)¶

ignore()¶

Indicates to ignore this source in concatenation/merging.

Return type:: bool

classmethod merge(sources)¶

merger = None¶

mutate()¶

mutate_source()¶

name = None¶

property parent¶: The parent source, if any.

property parts¶

property path¶

prepare_headers(url)¶

source_filename = None¶

stream = False¶

to_data_object()¶: Convert this source into a data object, if possible.

to_stream()¶

to_target(*args, **kwargs)¶

property url¶

property url_parts¶

class earthkit.data.sources.url.Url(url, *, update_if_out_of_date=False, force=None, stream=False, **kwargs)¶

Bases: UrlBase

Represent a URL source.

Parameters:

url (str, list, tuple, UrlSpecItem, dict or an iterable of these) –

A single url can be:

a str
a list/tuple of two items. The first item is the url as a str, while the second item defines the parts for the given url. In this case the parts kwargs cannot be used.
a UrlSpecItem
a dict with same content as UrlSpecItem. Mandatory key is “url” and optionally “parts” and “kwargs” can be used.

Multiple urls can be provided as a list/tuple of of the single url formats above. Mixing single url formats in a list/tuple is not allowed.

connect_to_mirror(mirror)¶

content_type = None¶

filter = None¶

force = None¶

graph(depth=0)¶

ignore()¶

Indicates to ignore this source in concatenation/merging.

Return type:: bool

classmethod merge(sources)¶

merger = None¶

mutate()¶

mutate_source()¶

name = None¶

other_source = None¶

out_of_date(url, path, cache_data)¶

property parent¶: The parent source, if any.

property parts¶

property path¶

prepare_headers(url)¶

source_filename = None¶

stream = False¶

to_data_object()¶: Convert this source into a data object, if possible.

to_target(*args, **kwargs)¶

update_if_out_of_date = False¶

property url¶

property url_parts¶

class earthkit.data.sources.url.UrlBase(url, chunk_size=1024 * 1024, parts=None, filter=None, merger=None, verify=True, range_method='auto', http_headers=None, fake_headers=None, stream=False, auth=None, **kwargs)¶

Bases: earthkit.data.sources.file.FileSource

Base class for all sources.

connect_to_mirror(mirror)¶

content_type = None¶

filter = None¶

graph(depth=0)¶

ignore()¶

Indicates to ignore this source in concatenation/merging.

Return type:: bool

classmethod merge(sources)¶

merger = None¶

mutate()¶

mutate_source()¶

name = None¶

property parent¶: The parent source, if any.

property parts¶

property path¶

prepare_headers(url)¶

source_filename = None¶

stream = False¶

to_data_object()¶: Convert this source into a data object, if possible.

to_target(*args, **kwargs)¶

property url¶

property url_parts¶

class earthkit.data.sources.url.UrlSourcePathAndParts(path, parts)¶

Bases: earthkit.data.utils.parts.PathAndParts

compress = True¶

classmethod from_paths(path, parts)¶

is_empty()¶

parts¶

path¶

sequence = None¶

sorted()¶

update(path)¶

zipped()¶

earthkit.data.sources.url.download_and_cache(url, *, owner='url', parts=None, verify=True, force=None, chunk_size=1024 * 1024, range_method='auto', http_headers=None, update_if_out_of_date=False, fake_headers=None, **kwargs)¶

earthkit.data.sources.url.download_to_target(url, target, *, parts=None, verify=True, chunk_size=1024 * 1024, range_method='auto', http_headers=None, fake_headers=None, **kwargs)¶

earthkit.data.sources.url.source¶