{
"cells": [
{
"cell_type": "markdown",
"id": "102b1cf9-33a5-4310-af4d-bcd17d598d29",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": []
},
"source": [
"## BUFR: using SYNOP data"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e11bb141-5f5d-42c2-9083-3cccbf9b7799",
"metadata": {},
"source": [
"We load a BUFR file with SYNOP observations taken from ECMWF MARS archive. First we ensure the example file is available."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "41f29800-f75d-4adf-a7b9-7fb0381dadd5",
"metadata": {},
"outputs": [],
"source": [
"import earthkit.data as ekd\n",
"ekd.download_example_file(\"synop_10.bufr\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f0430464-fd40-4964-b64d-4629ddd98909",
"metadata": {},
"outputs": [],
"source": [
"ds = ekd.from_source(\"file\", \"synop_10.bufr\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "07571864-48af-451b-be4a-6a42436d193c",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"10"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(ds)"
]
},
{
"cell_type": "raw",
"id": "a1f9fd95-d020-4941-b616-f01da9177b4e",
"metadata": {
"editable": true,
"raw_mimetype": "text/restructuredtext",
"slideshow": {
"slide_type": ""
},
"tags": []
},
"source": [
"We use :py:meth:`~data.readers.bufr.bufr.BUFRList.ls` to see metadata from the header section of the BUFR messages: "
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "30e7749a-ea8b-43d2-90d2-31907f228513",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" edition | \n",
" type | \n",
" subtype | \n",
" c | \n",
" mv | \n",
" lv | \n",
" subsets | \n",
" compr | \n",
" typicalDate | \n",
" typicalTime | \n",
" ident | \n",
" lat | \n",
" lon | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 91648 | \n",
" -10.75 | \n",
" 179.50 | \n",
"
\n",
" \n",
" | 1 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 89514 | \n",
" -70.77 | \n",
" 11.75 | \n",
"
\n",
" \n",
" | 2 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 60545 | \n",
" 33.77 | \n",
" 2.93 | \n",
"
\n",
" \n",
" | 3 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 30823 | \n",
" 51.83 | \n",
" 107.60 | \n",
"
\n",
" \n",
" | 4 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 30846 | \n",
" 51.35 | \n",
" 112.47 | \n",
"
\n",
" \n",
" | 5 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 48352 | \n",
" 17.86 | \n",
" 102.75 | \n",
"
\n",
" \n",
" | 6 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 98747 | \n",
" 8.41 | \n",
" 124.61 | \n",
"
\n",
" \n",
" | 7 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 68267 | \n",
" -26.50 | \n",
" 29.98 | \n",
"
\n",
" \n",
" | 8 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 68592 | \n",
" -29.60 | \n",
" 31.12 | \n",
"
\n",
" \n",
" | 9 | \n",
" 3 | \n",
" 0 | \n",
" 1 | \n",
" 98 | \n",
" 13 | \n",
" 1 | \n",
" 1 | \n",
" 0 | \n",
" 20230602 | \n",
" 120000 | \n",
" 91701 | \n",
" -2.77 | \n",
" -171.72 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" edition type subtype c mv lv subsets compr typicalDate typicalTime \\\n",
"0 3 0 1 98 13 1 1 0 20230602 120000 \n",
"1 3 0 1 98 13 1 1 0 20230602 120000 \n",
"2 3 0 1 98 13 1 1 0 20230602 120000 \n",
"3 3 0 1 98 13 1 1 0 20230602 120000 \n",
"4 3 0 1 98 13 1 1 0 20230602 120000 \n",
"5 3 0 1 98 13 1 1 0 20230602 120000 \n",
"6 3 0 1 98 13 1 1 0 20230602 120000 \n",
"7 3 0 1 98 13 1 1 0 20230602 120000 \n",
"8 3 0 1 98 13 1 1 0 20230602 120000 \n",
"9 3 0 1 98 13 1 1 0 20230602 120000 \n",
"\n",
" ident lat lon \n",
"0 91648 -10.75 179.50 \n",
"1 89514 -70.77 11.75 \n",
"2 60545 33.77 2.93 \n",
"3 30823 51.83 107.60 \n",
"4 30846 51.35 112.47 \n",
"5 48352 17.86 102.75 \n",
"6 98747 8.41 124.61 \n",
"7 68267 -26.50 29.98 \n",
"8 68592 -29.60 31.12 \n",
"9 91701 -2.77 -171.72 "
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds.ls()"
]
},
{
"cell_type": "markdown",
"id": "442edb8e-8b80-4397-9eea-2cc9a607c28e",
"metadata": {},
"source": [
"#### Extracting 2m temperature"
]
},
{
"cell_type": "raw",
"id": "ab0308f1-a43d-4713-844d-49cbfe03453a",
"metadata": {
"editable": true,
"raw_mimetype": "text/restructuredtext",
"slideshow": {
"slide_type": ""
},
"tags": []
},
"source": [
"BUFR data can be extracted into a Pandas dataframe using :py:meth:`~data.readers.bufr.bufr.BUFRList.to_pandas`, which passes all the arguments to the `read_bufr() `_ method from :xref:`pdbufr`.\n",
"\n",
"SYNOP data can be encoded into BUFR in many different ways. For the data we have the location and the 2m temperature can be extracted in the following way into a Pandas dataframe:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "53ac87d6-09b9-46c3-bc54-07fd2a175b05",
"metadata": {
"editable": true,
"slideshow": {
"slide_type": ""
},
"tags": []
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" latitude | \n",
" longitude | \n",
" heightOfStation | \n",
" airTemperatureAt2M | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" -10.75 | \n",
" 179.50 | \n",
" 3.0 | \n",
" 300.4 | \n",
"
\n",
" \n",
" | 1 | \n",
" -70.77 | \n",
" 11.75 | \n",
" NaN | \n",
" 255.2 | \n",
"
\n",
" \n",
" | 2 | \n",
" 33.77 | \n",
" 2.93 | \n",
" 763.0 | \n",
" 296.3 | \n",
"
\n",
" \n",
" | 3 | \n",
" 51.83 | \n",
" 107.60 | \n",
" 515.0 | \n",
" 291.6 | \n",
"
\n",
" \n",
" | 4 | \n",
" 51.35 | \n",
" 112.47 | \n",
" 743.0 | \n",
" 287.4 | \n",
"
\n",
" \n",
" | 5 | \n",
" 17.86 | \n",
" 102.75 | \n",
" 176.0 | \n",
" 307.9 | \n",
"
\n",
" \n",
" | 6 | \n",
" 8.41 | \n",
" 124.61 | \n",
" 188.0 | \n",
" 299.4 | \n",
"
\n",
" \n",
" | 7 | \n",
" -26.50 | \n",
" 29.98 | \n",
" 1774.0 | \n",
" 281.9 | \n",
"
\n",
" \n",
" | 8 | \n",
" -29.60 | \n",
" 31.12 | \n",
" 105.0 | \n",
" 299.8 | \n",
"
\n",
" \n",
" | 9 | \n",
" -2.77 | \n",
" -171.72 | \n",
" 2.0 | \n",
" 302.1 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" latitude longitude heightOfStation airTemperatureAt2M\n",
"0 -10.75 179.50 3.0 300.4\n",
"1 -70.77 11.75 NaN 255.2\n",
"2 33.77 2.93 763.0 296.3\n",
"3 51.83 107.60 515.0 291.6\n",
"4 51.35 112.47 743.0 287.4\n",
"5 17.86 102.75 176.0 307.9\n",
"6 8.41 124.61 188.0 299.4\n",
"7 -26.50 29.98 1774.0 281.9\n",
"8 -29.60 31.12 105.0 299.8\n",
"9 -2.77 -171.72 2.0 302.1"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n",
" \"heightOfStation\",\"airTemperatureAt2M\"])\n",
"df"
]
},
{
"cell_type": "markdown",
"id": "4cbe46fc-bacc-4857-a482-32a6a38719cb",
"metadata": {},
"source": [
"#### Using filters"
]
},
{
"cell_type": "markdown",
"id": "20806a27-ed82-4cc6-ad55-ed4c55baa748",
"metadata": {},
"source": [
"Specify station WMO IDs:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "26cebaca-9b17-4e15-9564-d92804886ea9",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" latitude | \n",
" longitude | \n",
" heightOfStation | \n",
" airTemperatureAt2M | \n",
" WMO_station_id | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" -70.77 | \n",
" 11.75 | \n",
" NaN | \n",
" 255.2 | \n",
" 89514 | \n",
"
\n",
" \n",
" | 1 | \n",
" 51.35 | \n",
" 112.47 | \n",
" 743.0 | \n",
" 287.4 | \n",
" 30846 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" latitude longitude heightOfStation airTemperatureAt2M WMO_station_id\n",
"0 -70.77 11.75 NaN 255.2 89514\n",
"1 51.35 112.47 743.0 287.4 30846"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n",
" \"heightOfStation\",\"airTemperatureAt2M\", \"WMO_station_id\"],\n",
" filters={\"WMO_station_id\": [30846, 89514]})\n",
"df"
]
},
{
"cell_type": "markdown",
"id": "c2290edb-ed8e-47b7-a059-51e31e84051e",
"metadata": {},
"source": [
"Temperature values <= 290 K:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "04ec7393-8969-4f46-82a6-5feb8f8a54c8",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" latitude | \n",
" longitude | \n",
" heightOfStation | \n",
" airTemperatureAt2M | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" -70.77 | \n",
" 11.75 | \n",
" NaN | \n",
" 255.2 | \n",
"
\n",
" \n",
" | 1 | \n",
" 51.35 | \n",
" 112.47 | \n",
" 743.0 | \n",
" 287.4 | \n",
"
\n",
" \n",
" | 2 | \n",
" -26.50 | \n",
" 29.98 | \n",
" 1774.0 | \n",
" 281.9 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" latitude longitude heightOfStation airTemperatureAt2M\n",
"0 -70.77 11.75 NaN 255.2\n",
"1 51.35 112.47 743.0 287.4\n",
"2 -26.50 29.98 1774.0 281.9"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n",
" \"heightOfStation\",\"airTemperatureAt2M\"],\n",
" filters={\"airTemperatureAt2M\": slice(None,290)})\n",
"df"
]
},
{
"cell_type": "markdown",
"id": "a5c35127-834f-4f1a-981d-ae9532bd6d3a",
"metadata": {},
"source": [
"Temperature values >= 290 K and <= 300 K:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "d75fe63b-baaa-43fa-8689-c3838e6b0fe5",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" latitude | \n",
" longitude | \n",
" heightOfStation | \n",
" airTemperatureAt2M | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 33.77 | \n",
" 2.93 | \n",
" 763 | \n",
" 296.3 | \n",
"
\n",
" \n",
" | 1 | \n",
" 51.83 | \n",
" 107.60 | \n",
" 515 | \n",
" 291.6 | \n",
"
\n",
" \n",
" | 2 | \n",
" 8.41 | \n",
" 124.61 | \n",
" 188 | \n",
" 299.4 | \n",
"
\n",
" \n",
" | 3 | \n",
" -29.60 | \n",
" 31.12 | \n",
" 105 | \n",
" 299.8 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" latitude longitude heightOfStation airTemperatureAt2M\n",
"0 33.77 2.93 763 296.3\n",
"1 51.83 107.60 515 291.6\n",
"2 8.41 124.61 188 299.4\n",
"3 -29.60 31.12 105 299.8"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n",
" \"heightOfStation\",\"airTemperatureAt2M\"],\n",
" filters={\"airTemperatureAt2M\": slice(290,300)})\n",
"df"
]
},
{
"cell_type": "markdown",
"id": "ee9ed59e-8b2f-4aaa-86bc-7e77805f089c",
"metadata": {},
"source": [
"Temperature values >= 300 K:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "c7611278-9426-46fd-8c49-402545122414",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" latitude | \n",
" longitude | \n",
" heightOfStation | \n",
" airTemperatureAt2M | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" -10.75 | \n",
" 179.50 | \n",
" 3 | \n",
" 300.4 | \n",
"
\n",
" \n",
" | 1 | \n",
" 17.86 | \n",
" 102.75 | \n",
" 176 | \n",
" 307.9 | \n",
"
\n",
" \n",
" | 2 | \n",
" -2.77 | \n",
" -171.72 | \n",
" 2 | \n",
" 302.1 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" latitude longitude heightOfStation airTemperatureAt2M\n",
"0 -10.75 179.50 3 300.4\n",
"1 17.86 102.75 176 307.9\n",
"2 -2.77 -171.72 2 302.1"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n",
" \"heightOfStation\",\"airTemperatureAt2M\"],\n",
" filters={\"airTemperatureAt2M\": slice(300, None)})\n",
"df"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eb71e23d-d813-471e-8749-c283ee7e9ada",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "dev_ecc",
"language": "python",
"name": "dev_ecc"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
}
},
"nbformat": 4,
"nbformat_minor": 5
}