{ "cells": [ { "cell_type": "markdown", "id": "102b1cf9-33a5-4310-af4d-bcd17d598d29", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## BUFR: using SYNOP data" ] }, { "attachments": {}, "cell_type": "markdown", "id": "e11bb141-5f5d-42c2-9083-3cccbf9b7799", "metadata": {}, "source": [ "We load a BUFR file with SYNOP observations taken from ECMWF MARS archive. First we ensure the example file is available." ] }, { "cell_type": "code", "execution_count": 1, "id": "41f29800-f75d-4adf-a7b9-7fb0381dadd5", "metadata": {}, "outputs": [], "source": [ "import earthkit.data as ekd\n", "ekd.download_example_file(\"synop_10.bufr\")" ] }, { "cell_type": "code", "execution_count": 2, "id": "f0430464-fd40-4964-b64d-4629ddd98909", "metadata": {}, "outputs": [], "source": [ "ds = ekd.from_source(\"file\", \"synop_10.bufr\")" ] }, { "cell_type": "code", "execution_count": 3, "id": "07571864-48af-451b-be4a-6a42436d193c", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "10" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(ds)" ] }, { "cell_type": "raw", "id": "a1f9fd95-d020-4941-b616-f01da9177b4e", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "We use :py:meth:`~data.readers.bufr.bufr.BUFRList.ls` to see metadata from the header section of the BUFR messages: " ] }, { "cell_type": "code", "execution_count": 4, "id": "30e7749a-ea8b-43d2-90d2-31907f228513", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
editiontypesubtypecmvlvsubsetscomprtypicalDatetypicalTimeidentlatlon
030198131102023060212000091648-10.75179.50
130198131102023060212000089514-70.7711.75
23019813110202306021200006054533.772.93
33019813110202306021200003082351.83107.60
43019813110202306021200003084651.35112.47
53019813110202306021200004835217.86102.75
6301981311020230602120000987478.41124.61
730198131102023060212000068267-26.5029.98
830198131102023060212000068592-29.6031.12
930198131102023060212000091701-2.77-171.72
\n", "
" ], "text/plain": [ " edition type subtype c mv lv subsets compr typicalDate typicalTime \\\n", "0 3 0 1 98 13 1 1 0 20230602 120000 \n", "1 3 0 1 98 13 1 1 0 20230602 120000 \n", "2 3 0 1 98 13 1 1 0 20230602 120000 \n", "3 3 0 1 98 13 1 1 0 20230602 120000 \n", "4 3 0 1 98 13 1 1 0 20230602 120000 \n", "5 3 0 1 98 13 1 1 0 20230602 120000 \n", "6 3 0 1 98 13 1 1 0 20230602 120000 \n", "7 3 0 1 98 13 1 1 0 20230602 120000 \n", "8 3 0 1 98 13 1 1 0 20230602 120000 \n", "9 3 0 1 98 13 1 1 0 20230602 120000 \n", "\n", " ident lat lon \n", "0 91648 -10.75 179.50 \n", "1 89514 -70.77 11.75 \n", "2 60545 33.77 2.93 \n", "3 30823 51.83 107.60 \n", "4 30846 51.35 112.47 \n", "5 48352 17.86 102.75 \n", "6 98747 8.41 124.61 \n", "7 68267 -26.50 29.98 \n", "8 68592 -29.60 31.12 \n", "9 91701 -2.77 -171.72 " ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds.ls()" ] }, { "cell_type": "markdown", "id": "442edb8e-8b80-4397-9eea-2cc9a607c28e", "metadata": {}, "source": [ "#### Extracting 2m temperature" ] }, { "cell_type": "raw", "id": "ab0308f1-a43d-4713-844d-49cbfe03453a", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "BUFR data can be extracted into a Pandas dataframe using :py:meth:`~data.readers.bufr.bufr.BUFRList.to_pandas`, which passes all the arguments to the `read_bufr() `_ method from :xref:`pdbufr`.\n", "\n", "SYNOP data can be encoded into BUFR in many different ways. For the data we have the location and the 2m temperature can be extracted in the following way into a Pandas dataframe:" ] }, { "cell_type": "code", "execution_count": 5, "id": "53ac87d6-09b9-46c3-bc54-07fd2a175b05", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
latitudelongitudeheightOfStationairTemperatureAt2M
0-10.75179.503.0300.4
1-70.7711.75NaN255.2
233.772.93763.0296.3
351.83107.60515.0291.6
451.35112.47743.0287.4
517.86102.75176.0307.9
68.41124.61188.0299.4
7-26.5029.981774.0281.9
8-29.6031.12105.0299.8
9-2.77-171.722.0302.1
\n", "
" ], "text/plain": [ " latitude longitude heightOfStation airTemperatureAt2M\n", "0 -10.75 179.50 3.0 300.4\n", "1 -70.77 11.75 NaN 255.2\n", "2 33.77 2.93 763.0 296.3\n", "3 51.83 107.60 515.0 291.6\n", "4 51.35 112.47 743.0 287.4\n", "5 17.86 102.75 176.0 307.9\n", "6 8.41 124.61 188.0 299.4\n", "7 -26.50 29.98 1774.0 281.9\n", "8 -29.60 31.12 105.0 299.8\n", "9 -2.77 -171.72 2.0 302.1" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n", " \"heightOfStation\",\"airTemperatureAt2M\"])\n", "df" ] }, { "cell_type": "markdown", "id": "4cbe46fc-bacc-4857-a482-32a6a38719cb", "metadata": {}, "source": [ "#### Using filters" ] }, { "cell_type": "markdown", "id": "20806a27-ed82-4cc6-ad55-ed4c55baa748", "metadata": {}, "source": [ "Specify station WMO IDs:" ] }, { "cell_type": "code", "execution_count": 6, "id": "26cebaca-9b17-4e15-9564-d92804886ea9", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
latitudelongitudeheightOfStationairTemperatureAt2MWMO_station_id
0-70.7711.75NaN255.289514
151.35112.47743.0287.430846
\n", "
" ], "text/plain": [ " latitude longitude heightOfStation airTemperatureAt2M WMO_station_id\n", "0 -70.77 11.75 NaN 255.2 89514\n", "1 51.35 112.47 743.0 287.4 30846" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n", " \"heightOfStation\",\"airTemperatureAt2M\", \"WMO_station_id\"],\n", " filters={\"WMO_station_id\": [30846, 89514]})\n", "df" ] }, { "cell_type": "markdown", "id": "c2290edb-ed8e-47b7-a059-51e31e84051e", "metadata": {}, "source": [ "Temperature values <= 290 K:" ] }, { "cell_type": "code", "execution_count": 7, "id": "04ec7393-8969-4f46-82a6-5feb8f8a54c8", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
latitudelongitudeheightOfStationairTemperatureAt2M
0-70.7711.75NaN255.2
151.35112.47743.0287.4
2-26.5029.981774.0281.9
\n", "
" ], "text/plain": [ " latitude longitude heightOfStation airTemperatureAt2M\n", "0 -70.77 11.75 NaN 255.2\n", "1 51.35 112.47 743.0 287.4\n", "2 -26.50 29.98 1774.0 281.9" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n", " \"heightOfStation\",\"airTemperatureAt2M\"],\n", " filters={\"airTemperatureAt2M\": slice(None,290)})\n", "df" ] }, { "cell_type": "markdown", "id": "a5c35127-834f-4f1a-981d-ae9532bd6d3a", "metadata": {}, "source": [ "Temperature values >= 290 K and <= 300 K:" ] }, { "cell_type": "code", "execution_count": 8, "id": "d75fe63b-baaa-43fa-8689-c3838e6b0fe5", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
latitudelongitudeheightOfStationairTemperatureAt2M
033.772.93763296.3
151.83107.60515291.6
28.41124.61188299.4
3-29.6031.12105299.8
\n", "
" ], "text/plain": [ " latitude longitude heightOfStation airTemperatureAt2M\n", "0 33.77 2.93 763 296.3\n", "1 51.83 107.60 515 291.6\n", "2 8.41 124.61 188 299.4\n", "3 -29.60 31.12 105 299.8" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n", " \"heightOfStation\",\"airTemperatureAt2M\"],\n", " filters={\"airTemperatureAt2M\": slice(290,300)})\n", "df" ] }, { "cell_type": "markdown", "id": "ee9ed59e-8b2f-4aaa-86bc-7e77805f089c", "metadata": {}, "source": [ "Temperature values >= 300 K:" ] }, { "cell_type": "code", "execution_count": 9, "id": "c7611278-9426-46fd-8c49-402545122414", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
latitudelongitudeheightOfStationairTemperatureAt2M
0-10.75179.503300.4
117.86102.75176307.9
2-2.77-171.722302.1
\n", "
" ], "text/plain": [ " latitude longitude heightOfStation airTemperatureAt2M\n", "0 -10.75 179.50 3 300.4\n", "1 17.86 102.75 176 307.9\n", "2 -2.77 -171.72 2 302.1" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = ds.to_pandas(columns=[\"latitude\", \"longitude\", \n", " \"heightOfStation\",\"airTemperatureAt2M\"],\n", " filters={\"airTemperatureAt2M\": slice(300, None)})\n", "df" ] }, { "cell_type": "code", "execution_count": null, "id": "eb71e23d-d813-471e-8749-c283ee7e9ada", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "dev_ecc", "language": "python", "name": "dev_ecc" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.13" } }, "nbformat": 4, "nbformat_minor": 5 }