GRIB: inspecting contents
We will work with a GRIB file containing 6 messages. First we ensure the example file is available, then read the file with from_source().
[1]:
import earthkit.data as ekd
ekd.download_example_file("test6.grib")
[2]:
fs = ekd.from_source("file", "test6.grib")
Using head(), tail() and ls()
All these methods (head(), tail() and ls()) take the same set of keyword arguments:
n: number of messages to list
keys: the metadata keys to dump
extra_keys: extra keys of top of the default set of keys
namespace: the ecCodes namespace to use
head()
head() displays the first 5 messages by default:
[3]:
fs.head()
[3]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 2 | ecmf | v | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 3 | ecmf | t | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 4 | ecmf | u | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
The number of fields can be passed as an argument:
[4]:
fs.head(2)
[4]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
[5]:
fs.head(n=2)
[5]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
The keys can be taken from an ecCodes GRIB namespace:
[6]:
fs.head(namespace="statistics")
[6]:
| max | min | avg | sd | skew | kurt | const | |
|---|---|---|---|---|---|---|---|
| 0 | 320.564178 | 240.564178 | 279.707036 | 19.674217 | -0.731230 | -0.169046 | 0.0 |
| 1 | 17.713120 | -6.286880 | -0.382119 | 5.605310 | 1.139004 | 1.018780 | 0.0 |
| 2 | 11.833481 | -16.166519 | -0.071281 | 6.140457 | -0.217654 | -0.839983 | 0.0 |
| 3 | 304.539169 | 232.539169 | 272.729646 | 19.241867 | -0.998189 | 0.135718 | 0.0 |
| 4 | 27.101624 | -12.898376 | 0.339719 | 8.141047 | 1.542573 | 1.864089 | 0.0 |
[7]:
fs.head(namespace="parameter")
[7]:
| centre | paramId | units | name | shortName | |
|---|---|---|---|---|---|
| 0 | ecmf | 130 | K | Temperature | t |
| 1 | ecmf | 131 | m s**-1 | U component of wind | u |
| 2 | ecmf | 132 | m s**-1 | V component of wind | v |
| 3 | ecmf | 130 | K | Temperature | t |
| 4 | ecmf | 131 | m s**-1 | U component of wind | u |
The list of keys can be prescribed:
[8]:
fs.head(n=2, keys=["bitsPerValue", "packingType"])
[8]:
| bitsPerValue | packingType | |
|---|---|---|
| 0 | 4 | grid_simple |
| 1 | 4 | grid_simple |
The list of keys can also be extended:
[9]:
fs.head(extra_keys="paramId")
[9]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | paramId | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll | 130 |
| 1 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll | 131 |
| 2 | ecmf | v | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll | 132 |
| 3 | ecmf | t | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll | 130 |
| 4 | ecmf | u | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll | 131 |
tail()
tail() displays the last 5 messages by default:
[10]:
fs.tail(5)
[10]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | v | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 2 | ecmf | t | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 3 | ecmf | u | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 4 | ecmf | v | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
ls()
By default ls() lists all the fields:
[11]:
fs.ls()
[11]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 2 | ecmf | v | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 3 | ecmf | t | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 4 | ecmf | u | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 5 | ecmf | v | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
The number of fields to list can also be specified:
[12]:
fs.ls(2)
[12]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | t | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | u | isobaricInhPa | 1000 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
[13]:
fs.ls(-2)
[13]:
| centre | shortName | typeOfLevel | level | dataDate | dataTime | stepRange | dataType | number | gridType | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | ecmf | u | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
| 1 | ecmf | v | isobaricInhPa | 850 | 20180801 | 1200 | 0 | an | 0 | regular_ll |
Using describe
We can describe() the whole object:
[14]:
fs.describe()
[14]:
| level | date | time | step | paramId | class | stream | type | experimentVersionNumber | ||
|---|---|---|---|---|---|---|---|---|---|---|
| shortName | typeOfLevel | |||||||||
| t | isobaricInhPa | 1000,850 | 20180801 | 1200 | 0 | 130 | od | oper | an | 0001 |
| u | isobaricInhPa | 1000,850 | 20180801 | 1200 | 0 | 131 | od | oper | an | 0001 |
| v | isobaricInhPa | 1000,850 | 20180801 | 1200 | 0 | 132 | od | oper | an | 0001 |
We can also describe() a given parameter (defined by shortName or paramId):
[15]:
fs.describe("t")
[15]:
| shortName | t |
|---|---|
| typeOfLevel | isobaricInhPa |
| level | 1000,850 |
| date | 20180801 |
| time | 1200 |
| step | 0 |
| paramId | 130 |
| class | od |
| stream | oper |
| type | an |
| experimentVersionNumber | 0001 |
[16]:
fs.describe(131)
[16]:
| shortName | u |
|---|---|
| typeOfLevel | isobaricInhPa |
| level | 1000,850 |
| date | 20180801 |
| time | 1200 |
| step | 0 |
| paramId | 131 |
| class | od |
| stream | oper |
| type | an |
| experimentVersionNumber | 0001 |
Accessing metadata
metadata() with positional arguments works for both for a field and a fieldlist. Key type qualifiers are allowed to be used.
[17]:
fs.metadata("typeOfLevel")
[17]:
['isobaricInhPa',
'isobaricInhPa',
'isobaricInhPa',
'isobaricInhPa',
'isobaricInhPa',
'isobaricInhPa']
[18]:
fs[0].metadata(["typeOfLevel", "level", "centre", "centre"], astype=(None, None, str, int))
[18]:
['isobaricInhPa', 1000, 'ecmf', 98]
For single fields the bracket operator can also be used:
[19]:
fs[0]["param"]
[19]:
't'
Namespace keys can be fetched as a dict for single fields:
[20]:
fs[0].metadata(namespace="parameter")
[20]:
{'centre': 'ecmf',
'paramId': 130,
'units': 'K',
'name': 'Temperature',
'shortName': 't'}
Inspecting all the namespace keys for a message
dump() gives a tabbed interface to inspect the various namespaces:
[21]:
fs[0].dump()
[21]:
GribFieldMetadata
| globalDomain | g |
| GRIBEditionNumber | 1 |
| eps | 0 |
| offsetSection0 | 0 |
| section0Length | 8 |
| totalLength | 150 |
| editionNumber | 1 |
| WMO | 0 |
| productionStatusOfProcessedData | 0 |
| section1Length | 52 |
| wrongPadding | 0 |
| table2Version | 128 |
| centre | ecmf |
| centreDescription | European Centre for Medium-Range Weather Forecasts |
| generatingProcessIdentifier | 254 |
| gridDefinition | 255 |
| indicatorOfParameter | 130 |
| parameterName | Temperature |
| parameterUnits | K |
| indicatorOfTypeOfLevel | pl |
| pressureUnits | hPa |
| typeOfLevelECMF | isobaricInhPa |
| typeOfLevel | isobaricInhPa |
| level | 1000 |
| yearOfCentury | 18 |
| month | 8 |
| day | 1 |
| hour | 12 |
| minute | 0 |
| second | 0 |
| unitOfTimeRange | 1 |
| P1 | 0 |
| P2 | 0 |
| timeRangeIndicator | 0 |
| numberIncludedInAverage | 0 |
| numberMissingFromAveragesOrAccumulations | 0 |
| centuryOfReferenceTimeOfData | 21 |
| subCentre | 0 |
| paramIdECMF | 130 |
| paramId | 130 |
| cfNameECMF | air_temperature |
| cfName | air_temperature |
| unitsECMF | K |
| units | K |
| nameECMF | Temperature |
| name | Temperature |
| decimalScaleFactor | 0 |
| setLocalDefinition | 0 |
| optimizeScaleFactor | 0 |
| dataDate | 20180801 |
| year | 2018 |
| dataTime | 1200 |
| julianDay | 2458332.0 |
| stepUnits | 1 |
| stepType | instant |
| stepRange | 0 |
| startStep | 0 |
| endStep | 0 |
| marsParam | 130.128 |
| validityDate | 20180801 |
| validityTime | 1200 |
| validityDateTime | 2458331.5083333333 |
| deleteLocalDefinition | 0 |
| localUsePresent | 1 |
| reservedNeedNotBePresent | [''] |
| localDefinitionNumber | 1 |
| GRIBEXSection1Problem | 0 |
| marsClass | od |
| marsType | an |
| marsStream | oper |
| experimentVersionNumber | 0001 |
| perturbationNumber | 0 |
| numberOfForecastsInEnsemble | 0 |
| padding_local1_1 | 00 |
| grib2LocalSectionNumber | 1 |
| localExtensionPadding | |
| _x | None |
| section1Padding | |
| shortNameECMF | t |
| shortName | t |
| cfVarNameECMF | t |
| cfVarName | t |
| ifsParam | 130 |
| stepTypeForConversion | unknown |
| md5Section1 | 3ae6b1f6f34f431c6134c40ec42f27fa |
| md5Product | 54697e1d910c88b76a8759b67e5c7a9c |
| paramIdForConversion | 0 |
| gridDescriptionSectionPresent | 1 |
| bitmapPresent | 0 |
| angleSubdivisions | 1000 |
| section2Length | 32 |
| radius | 6367470 |
| numberOfVerticalCoordinateValues | 0 |
| neitherPresent | 255 |
| pvlLocation | 255 |
| dataRepresentationType | 0 |
| gridDefinitionDescription | Latitude/Longitude Grid |
| gridDefinitionTemplateNumber | 0 |
| Ni | 12 |
| Nj | 7 |
| latitudeOfFirstGridPoint | 90000 |
| latitudeOfFirstGridPointInDegrees | 90.0 |
| longitudeOfFirstGridPoint | 0 |
| longitudeOfFirstGridPointInDegrees | 0.0 |
| resolutionAndComponentFlags | 128 |
| ijDirectionIncrementGiven | 1 |
| earthIsOblate | 0 |
| resolutionAndComponentFlags3 | 0 |
| resolutionAndComponentFlags4 | 0 |
| uvRelativeToGrid | 0 |
| resolutionAndComponentFlags6 | 0 |
| resolutionAndComponentFlags7 | 0 |
| resolutionAndComponentFlags8 | 0 |
| latitudeOfLastGridPoint | -90000 |
| latitudeOfLastGridPointInDegrees | -90.0 |
| longitudeOfLastGridPoint | 330000 |
| longitudeOfLastGridPointInDegrees | 330.0 |
| iDirectionIncrement | 30000 |
| jDirectionIncrement | 30000 |
| isGridded | 1 |
| scanningMode | 0 |
| iScansNegatively | 0 |
| jScansPositively | 0 |
| jPointsAreConsecutive | 0 |
| alternativeRowScanning | 0 |
| iScansPositively | 1 |
| jScansNegatively | 1 |
| scanningMode4 | 0 |
| scanningMode5 | 0 |
| scanningMode6 | 0 |
| scanningMode7 | 0 |
| scanningMode8 | 0 |
| swapScanningAlternativeRows | 0 |
| jDirectionIncrementInDegrees | 30.0 |
| iDirectionIncrementInDegrees | 30.0 |
| numberOfDataPoints | 84 |
| numberOfValues | 84 |
| zeros | |
| PVPresent | 0 |
| padding_sec2_2 | |
| PLPresent | 0 |
| padding_sec2_1 | |
| deletePV | 1 |
| padding_sec2_3 | |
| md5Section2 | e09e4d6171c0ac85da1d256b2f8acf88 |
| isSpectral | 0 |
| lengthOfHeaders | 85 |
| md5Headers | 9160fb809a9d7b1efc95163bf36e6b51 |
| missingValue | 9999 |
| tableReference | 0 |
| section4Length | 54 |
| halfByte | 8 |
| dataFlag | 8 |
| binaryScaleFactor | 3 |
| referenceValue | 240.56417846679688 |
| referenceValueError | 1.52587890625e-05 |
| sphericalHarmonics | 0 |
| complexPacking | 0 |
| integerPointValues | 0 |
| additionalFlagPresent | 0 |
| orderOfSPD | 2 |
| boustrophedonic | 0 |
| hideThis | 0 |
| packingType | grid_simple |
| bitsPerValue | 4 |
| constantFieldHalfByte | 8 |
| bitMapIndicator | 255 |
| numberOfCodedValues | 84 |
| packingError | 4.000007629394531 |
| unpackedError | 1.52587890625e-05 |
| maximum | 320.5641784667969 |
| minimum | 240.56417846679688 |
| average | 279.70703560965404 |
| standardDeviation | 19.67421739058438 |
| skewness | -0.7312302105044429 |
| kurtosis | -0.16904561574741717 |
| isConstant | 0.0 |
| numberOfMissing | 0 |
| dataLength | 5 |
| changeDecimalPrecision | 0 |
| decimalPrecision | 0 |
| bitsPerValueAndRepack | 4 |
| setPackingType | grid_simple |
| scaleValuesBy | 1.0 |
| offsetValuesBy | 0.0 |
| gridType | regular_ll |
| getNumberOfValues | 84 |
| padding_sec4_1 | |
| md5Section4 | 7ea3331d80a962b5dc99276b76451eac |
| section5Length | 4 |
| 7777 | 7777 |
| edition | 1 |
| centre | ecmf |
| typeOfLevel | isobaricInhPa |
| level | 1000 |
| dataDate | 20180801 |
| stepRange | 0 |
| dataType | an |
| shortName | t |
| packingType | grid_simple |
| gridType | regular_ll |
| bitmapPresent | 0 |
| Ni | 12 |
| Nj | 7 |
| latitudeOfFirstGridPointInDegrees | 90.0 |
| longitudeOfFirstGridPointInDegrees | 0.0 |
| latitudeOfLastGridPointInDegrees | -90.0 |
| longitudeOfLastGridPointInDegrees | 330.0 |
| iScansNegatively | 0 |
| jScansPositively | 0 |
| jPointsAreConsecutive | 0 |
| jDirectionIncrementInDegrees | 30.0 |
| iDirectionIncrementInDegrees | 30.0 |
| gridType | regular_ll |
| domain | g |
| levtype | pl |
| levelist | 1000 |
| date | 20180801 |
| time | 1200 |
| step | 0 |
| param | t |
| class | od |
| type | an |
| stream | oper |
| expver | 0001 |
| centre | ecmf |
| paramId | 130 |
| units | K |
| name | Temperature |
| shortName | t |
| max | 320.5641784667969 |
| min | 240.56417846679688 |
| avg | 279.70703560965404 |
| sd | 19.67421739058438 |
| skew | -0.7312302105044429 |
| kurt | -0.16904561574741717 |
| const | 0.0 |
| dataDate | 20180801 |
| dataTime | 1200 |
| stepUnits | 1 |
| stepType | instant |
| stepRange | 0 |
| startStep | 0 |
| endStep | 0 |
| validityDate | 20180801 |
| validityTime | 1200 |
| typeOfLevel | isobaricInhPa |
| level | 1000 |