Metadata
[1]:
import earthkit.data as ekd
Metadata of the various data formats are represented by Metadata objects, which are:
immutable
behave like a dict
can be updated with the override() method creating a new object
derived from the abstract Metadata base class
RawMetadata
The simplest metadata type is RawMetadata, which we can create in the same way as a dict:
[2]:
from earthkit.data.core.metadata import RawMetadata
# from dict
md = RawMetadata({"shortName": "2t", "perturbationNumber": 5})
# from list of key/value pairs
md = RawMetadata([("shortName", "2t"), ("perturbationNumber", 5)])
# from keyword arguments
md = RawMetadata(shortName="2t", perturbationNumber=5)
[3]:
md
[3]:
RawMetadata({'shortName': '2t', 'perturbationNumber': 5})
Value access
We can get the value for a key using [] and get() just like for a dict:
[4]:
print(md["shortName"])
print(md.get("shortName"))
2t
2t
When a key is not available [] raises a KeyError.
[5]:
try:
md["nonExistentKey"]
except KeyError as e:
print(f"KeyError: {e}")
KeyError: 'nonExistentKey'
get() can take a default value as a second argument. When the key is not available the default value is returned. If default is not given, it defaults to None, so that this method never raises a KeyError.
[6]:
print(md.get("nonExistentKey"))
print(md.get("nonExistentKey", 12))
None
12
Iteration
[7]:
list(md.keys())
[7]:
['shortName', 'perturbationNumber']
[8]:
for k,v in md.items():
print(f"{k}: {v}")
shortName: 2t
perturbationNumber: 5
Override
The metadata object is immutable but we can use override() to create a new object with a modified content:
[9]:
md1 = md.override({"shortName": "2d", "step": 6})
[10]:
md1
[10]:
RawMetadata({'shortName': '2d', 'perturbationNumber': 5, 'step': 6})
The original metadata object did not change:
[11]:
md
[11]:
RawMetadata({'shortName': '2t', 'perturbationNumber': 5})
GribMetadata
For this exercise we read a GRIB file containing 4 messages:
[12]:
ekd.download_example_file("test4.grib")
ds = ekd.from_source("file", "test4.grib")
ds.ls()
[12]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 500 | 20070101 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | z | isobaricInhPa | 500 | 20070101 | 1200 | 0 | an | 0 | regular_ll |
| 2 | ecmf | t | isobaricInhPa | 850 | 20070101 | 1200 | 0 | an | 0 | regular_ll |
| 3 | ecmf | z | isobaricInhPa | 850 | 20070101 | 1200 | 0 | an | 0 | regular_ll |
We cannot create GRIB metadata object from a dict or RawMetadata, but can get it from a GRIB field. The resulting GribMetadata instance will store a reference to the ecCodes handle of its parent field.
[13]:
md = ds[0].metadata()
GribMetadata works like a dict:
[14]:
print(md["shortName"])
print(md.get("shortName"))
print(md.get("nonExistentKey"))
print(md.get("nonExistentKey", 12))
t
t
None
12
However, it is also aware of the ecCodes namespaces, when we use as_namespace() or dump().
[15]:
md.as_namespace("vertical")
[15]:
{'typeOfLevel': 'isobaricInhPa', 'level': 500}
[16]:
md.dump()
[16]:
GribFieldMetadata
| globalDomain | g |
| GRIBEditionNumber | 1 |
| eps | 0 |
| offsetSection0 | 0 |
| section0Length | 8 |
| totalLength | 130428 |
| editionNumber | 1 |
| WMO | 0 |
| productionStatusOfProcessedData | 0 |
| section1Length | 52 |
| wrongPadding | 0 |
| table2Version | 128 |
| centre | ecmf |
| centreDescription | European Centre for Medium-Range Weather Forecasts |
| generatingProcessIdentifier | 145 |
| gridDefinition | 255 |
| indicatorOfParameter | 130 |
| parameterName | Temperature |
| parameterUnits | K |
| indicatorOfTypeOfLevel | pl |
| pressureUnits | hPa |
| typeOfLevelECMF | isobaricInhPa |
| typeOfLevel | isobaricInhPa |
| level | 500 |
| yearOfCentury | 7 |
| month | 1 |
| day | 1 |
| hour | 12 |
| minute | 0 |
| second | 0 |
| unitOfTimeRange | 1 |
| P1 | 0 |
| P2 | 0 |
| timeRangeIndicator | 0 |
| numberIncludedInAverage | 0 |
| numberMissingFromAveragesOrAccumulations | 0 |
| centuryOfReferenceTimeOfData | 21 |
| subCentre | 0 |
| paramIdECMF | 130 |
| paramId | 130 |
| cfNameECMF | air_temperature |
| cfName | air_temperature |
| unitsECMF | K |
| units | K |
| nameECMF | Temperature |
| name | Temperature |
| decimalScaleFactor | 0 |
| setLocalDefinition | 0 |
| optimizeScaleFactor | 0 |
| dataDate | 20070101 |
| year | 2007 |
| dataTime | 1200 |
| julianDay | 2454102.0 |
| stepUnits | 1 |
| stepType | instant |
| stepRange | 0 |
| startStep | 0 |
| endStep | 0 |
| marsParam | 130.128 |
| validityDate | 20070101 |
| validityTime | 1200 |
| validityDateTime | 2454101.5083333333 |
| deleteLocalDefinition | 0 |
| localUsePresent | 1 |
| reservedNeedNotBePresent | [''] |
| localDefinitionNumber | 1 |
| GRIBEXSection1Problem | 0 |
| marsClass | ea |
| marsType | an |
| marsStream | oper |
| experimentVersionNumber | 0001 |
| perturbationNumber | 0 |
| numberOfForecastsInEnsemble | 0 |
| padding_local1_1 | 00 |
| grib2LocalSectionNumber | 1 |
| localExtensionPadding | |
| _x | None |
| section1Padding | |
| shortNameECMF | t |
| shortName | t |
| cfVarNameECMF | t |
| cfVarName | t |
| ifsParam | 130 |
| stepTypeForConversion | unknown |
| md5Section1 | e4c6d9f0d01b01247d9c57dad662ee30 |
| md5Product | ed3159ec68703fb0415e15e859b4eb02 |
| paramIdForConversion | 0 |
| gridDescriptionSectionPresent | 1 |
| bitmapPresent | 0 |
| angleSubdivisions | 1000 |
| section2Length | 32 |
| radius | 6367470 |
| numberOfVerticalCoordinateValues | 0 |
| neitherPresent | 255 |
| pvlLocation | 255 |
| dataRepresentationType | 0 |
| gridDefinitionDescription | Latitude/Longitude Grid |
| gridDefinitionTemplateNumber | 0 |
| Ni | 360 |
| Nj | 181 |
| latitudeOfFirstGridPoint | 90000 |
| latitudeOfFirstGridPointInDegrees | 90.0 |
| longitudeOfFirstGridPoint | 0 |
| longitudeOfFirstGridPointInDegrees | 0.0 |
| resolutionAndComponentFlags | 128 |
| ijDirectionIncrementGiven | 1 |
| earthIsOblate | 0 |
| resolutionAndComponentFlags3 | 0 |
| resolutionAndComponentFlags4 | 0 |
| uvRelativeToGrid | 0 |
| resolutionAndComponentFlags6 | 0 |
| resolutionAndComponentFlags7 | 0 |
| resolutionAndComponentFlags8 | 0 |
| latitudeOfLastGridPoint | -90000 |
| latitudeOfLastGridPointInDegrees | -90.0 |
| longitudeOfLastGridPoint | 359000 |
| longitudeOfLastGridPointInDegrees | 359.0 |
| iDirectionIncrement | 1000 |
| jDirectionIncrement | 1000 |
| isGridded | 1 |
| scanningMode | 0 |
| iScansNegatively | 0 |
| jScansPositively | 0 |
| jPointsAreConsecutive | 0 |
| alternativeRowScanning | 0 |
| iScansPositively | 1 |
| jScansNegatively | 1 |
| scanningMode4 | 0 |
| scanningMode5 | 0 |
| scanningMode6 | 0 |
| scanningMode7 | 0 |
| scanningMode8 | 0 |
| swapScanningAlternativeRows | 0 |
| jDirectionIncrementInDegrees | 1.0 |
| iDirectionIncrementInDegrees | 1.0 |
| numberOfDataPoints | 65160 |
| numberOfValues | 65160 |
| zeros | |
| PVPresent | 0 |
| padding_sec2_2 | |
| PLPresent | 0 |
| padding_sec2_1 | |
| deletePV | 1 |
| padding_sec2_3 | |
| md5Section2 | d0ccb07e4b36a8911817cc07539cf859 |
| isSpectral | 0 |
| lengthOfHeaders | 85 |
| md5Headers | b2ff552cc12a4ea08bfbd989ed20c0fb |
| missingValue | 9999 |
| tableReference | 0 |
| section4Length | 130332 |
| halfByte | 8 |
| dataFlag | 8 |
| binaryScaleFactor | -10 |
| referenceValue | 224.05772399902344 |
| referenceValueError | 1.52587890625e-05 |
| sphericalHarmonics | 0 |
| complexPacking | 0 |
| integerPointValues | 0 |
| additionalFlagPresent | 0 |
| orderOfSPD | 2 |
| boustrophedonic | 0 |
| hideThis | 0 |
| packingType | grid_simple |
| bitsPerValue | 16 |
| constantFieldHalfByte | 8 |
| bitMapIndicator | 255 |
| numberOfCodedValues | 65160 |
| packingError | 0.00049591064453125 |
| unpackedError | 1.52587890625e-05 |
| maximum | 273.33799743652344 |
| minimum | 224.05772399902344 |
| average | 252.5098487718183 |
| standardDeviation | 13.372886915179645 |
| skewness | -0.2209591997514728 |
| kurtosis | -1.2738579926507174 |
| isConstant | 0.0 |
| numberOfMissing | 0 |
| dataLength | 16290 |
| changeDecimalPrecision | 0 |
| decimalPrecision | 0 |
| bitsPerValueAndRepack | 16 |
| setPackingType | grid_simple |
| scaleValuesBy | 1.0 |
| offsetValuesBy | 0.0 |
| gridType | regular_ll |
| getNumberOfValues | 65160 |
| padding_sec4_1 | |
| md5Section4 | 73097e43c94a4e2947a55799222424c2 |
| section5Length | 4 |
| 7777 | 7777 |
| edition | 1 |
| centre | ecmf |
| typeOfLevel | isobaricInhPa |
| level | 500 |
| dataDate | 20070101 |
| stepRange | 0 |
| dataType | an |
| shortName | t |
| packingType | grid_simple |
| gridType | regular_ll |
| bitmapPresent | 0 |
| Ni | 360 |
| Nj | 181 |
| latitudeOfFirstGridPointInDegrees | 90.0 |
| longitudeOfFirstGridPointInDegrees | 0.0 |
| latitudeOfLastGridPointInDegrees | -90.0 |
| longitudeOfLastGridPointInDegrees | 359.0 |
| iScansNegatively | 0 |
| jScansPositively | 0 |
| jPointsAreConsecutive | 0 |
| jDirectionIncrementInDegrees | 1.0 |
| iDirectionIncrementInDegrees | 1.0 |
| gridType | regular_ll |
| domain | g |
| levtype | pl |
| levelist | 500 |
| date | 20070101 |
| time | 1200 |
| step | 0 |
| param | t |
| class | ea |
| type | an |
| stream | oper |
| expver | 0001 |
| centre | ecmf |
| paramId | 130 |
| units | K |
| name | Temperature |
| shortName | t |
| max | 273.33799743652344 |
| min | 224.05772399902344 |
| avg | 252.5098487718183 |
| sd | 13.372886915179645 |
| skew | -0.2209591997514728 |
| kurt | -1.2738579926507174 |
| const | 0.0 |
| dataDate | 20070101 |
| dataTime | 1200 |
| stepUnits | 1 |
| stepType | instant |
| stepRange | 0 |
| startStep | 0 |
| endStep | 0 |
| validityDate | 20070101 |
| validityTime | 1200 |
| typeOfLevel | isobaricInhPa |
| level | 500 |