{ "cells": [ { "cell_type": "markdown", "id": "d383b3fe-52c1-4fca-8c02-16dc017d0a6d", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## GRIB: getting latitudes, longitudes and values (regular LL grid)" ] }, { "cell_type": "markdown", "id": "c74a86a6-79ae-44bc-a9c9-8cc8a3aaa484", "metadata": {}, "source": [ "In this example we will work with GRIB data on a regular latitude-longitude grid. " ] }, { "cell_type": "code", "execution_count": 1, "id": "922c87f2-af84-44c0-9820-771f925f18c0", "metadata": {}, "outputs": [], "source": [ "import earthkit.data as ekd" ] }, { "cell_type": "raw", "id": "a419d6c2-79e4-48e2-b8e2-bcaf957b30cd", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "We will only use the temperature fields, so we :ref:`read ` the file and extract them with :py:meth:`~data.readers.grib.index.GribFieldList.sel`." ] }, { "cell_type": "code", "execution_count": 2, "id": "8a3afa0b-8701-497e-920c-0d44933679c3", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "", "version_major": 2, "version_minor": 0 }, "text/plain": [ "tuv_pl.grib: 0%| | 0.00/4.22k [00:00\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
centreshortNametypeOfLevelleveldataDatedataTimestepRangedataTypenumbergridType
0ecmftisobaricInhPa10002018080112000an0regular_ll
1ecmftisobaricInhPa8502018080112000an0regular_ll
2ecmftisobaricInhPa7002018080112000an0regular_ll
3ecmftisobaricInhPa5002018080112000an0regular_ll
4ecmftisobaricInhPa4002018080112000an0regular_ll
5ecmftisobaricInhPa3002018080112000an0regular_ll
\n", "" ], "text/plain": [ " centre shortName typeOfLevel level dataDate dataTime stepRange \\\n", "0 ecmf t isobaricInhPa 1000 20180801 1200 0 \n", "1 ecmf t isobaricInhPa 850 20180801 1200 0 \n", "2 ecmf t isobaricInhPa 700 20180801 1200 0 \n", "3 ecmf t isobaricInhPa 500 20180801 1200 0 \n", "4 ecmf t isobaricInhPa 400 20180801 1200 0 \n", "5 ecmf t isobaricInhPa 300 20180801 1200 0 \n", "\n", " dataType number gridType \n", "0 an 0 regular_ll \n", "1 an 0 regular_ll \n", "2 an 0 regular_ll \n", "3 an 0 regular_ll \n", "4 an 0 regular_ll \n", "5 an 0 regular_ll " ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds = ekd.from_source(\"sample\", \"tuv_pl.grib\").sel(param=\"t\")\n", "ds.ls()" ] }, { "cell_type": "markdown", "id": "be38f80c-4abf-4fbe-aeac-df941cbf537a", "metadata": {}, "source": [ "In the data each field is defined on a regular latitude-longitude grid. These grids can be represented as a 2D array so the field shape is always 2 dimensional:" ] }, { "cell_type": "code", "execution_count": 3, "id": "5a745568-83fe-4b98-a30b-585af7e26b71", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(7, 12)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds[0].shape" ] }, { "cell_type": "markdown", "id": "aab6ed54-3f74-430e-bd2b-4ab119433439", "metadata": {}, "source": [ "### Using the data() method" ] }, { "cell_type": "raw", "id": "f28bb8e9-3ec9-40a7-b42f-f6b4e6930da8", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "The simplest way to access the latitudes, longitudes and values is using the :py:meth:`~data.core.fieldlist.FieldList.data` method. It works both on :py:meth:`fields ` and :py:meth:`fieldlists `." ] }, { "cell_type": "markdown", "id": "c8c67310-78c7-4dea-be92-2bf51860b2a4", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "#### Fields" ] }, { "cell_type": "raw", "id": "ca4c158d-bda5-449a-a812-3f598444b222", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ ":py:meth:`~data.core.fieldlist.Field.data` returns the latitude, longitude and values arrays from a field as an ndarray. By default the field's shape is kept." ] }, { "cell_type": "code", "execution_count": 4, "id": "fbe608e7-1a94-44a1-b0f5-d84adabc370a", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "(numpy.ndarray, (3, 7, 12))" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "llv = ds[0].data()\n", "type(llv), llv.shape" ] }, { "cell_type": "markdown", "id": "a95f88f6-4136-472c-9e45-f4c744f97f5f", "metadata": {}, "source": [ "Here *llv[0]* contains the latitudes, *llv[1]* the longitudes, while *llv[2]* the values. Each of these arrays has the same shape that of the field *(7, 12)*). " ] }, { "cell_type": "code", "execution_count": 5, "id": "9ab0ec71-98ae-4c82-a8bc-0a1825a8b843", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 12)\n" ] }, { "data": { "text/plain": [ "90.0" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(llv[0].shape)\n", "# first latitude\n", "llv[0, 0, 0]" ] }, { "cell_type": "code", "execution_count": 6, "id": "0c0f0e5e-c99b-40c7-aaef-04931f391d1a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 12)\n" ] }, { "data": { "text/plain": [ "0.0" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(llv[1].shape)\n", "# first longitude\n", "llv[1, 0, 0]" ] }, { "cell_type": "code", "execution_count": 7, "id": "4161101c-96bd-4f2a-a17f-411b3150ecbd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 12)\n" ] }, { "data": { "text/plain": [ "272.5641784667969" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(llv[2].shape)\n", "# first value\n", "llv[2, 0, 0]" ] }, { "cell_type": "markdown", "id": "66c0a9b7-abc4-459f-acef-91e953ead484", "metadata": {}, "source": [ "When using the *flatten* keyword 1D arrays are returned:" ] }, { "cell_type": "code", "execution_count": 8, "id": "27660a88-f064-4f2a-a8b3-9145a3c665d2", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(3, 84)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "llv = ds[0].data(flatten=True)\n", "llv.shape" ] }, { "cell_type": "markdown", "id": "c754b09b-50fe-4b06-8ba4-2cef8aed1cd3", "metadata": {}, "source": [ "#### FieldLists" ] }, { "cell_type": "raw", "id": "c6c5f3fe-c602-405c-8eb4-d09c3da44c2b", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ ":py:meth:`~data.core.fieldlist.FieldList.data` only works on a fieldlist if all the fields have the same grid. The first two elements of the resulting ndarray are the latitude and longitude arrays (shared between fields), while the rest of the elements are the value arrays per field. Since we have 6 fields in our data the size of the first axis of the resulting ndarray is 2+6=8. Each of these arrays has the same shape that of the field *(7, 12)*)." ] }, { "cell_type": "code", "execution_count": 9, "id": "03c4a09d-ec99-442c-91e3-292d0541c1dd", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "(numpy.ndarray, (8, 7, 12))" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "llv = ds.data()\n", "type(llv), llv.shape" ] }, { "cell_type": "markdown", "id": "28912fa6-4471-412c-bcf1-c186212bf092", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Latitudes can be accessed as *llv[0]*." ] }, { "cell_type": "code", "execution_count": 10, "id": "c7f453d4-ba7e-4895-8e5b-14aa531129a4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 12)\n" ] }, { "data": { "text/plain": [ "90.0" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(llv[0].shape)\n", "# first latitude\n", "llv[0, 0, 0]" ] }, { "cell_type": "markdown", "id": "f93bae8f-5906-40a2-a3d9-166358c3a27c", "metadata": {}, "source": [ "Longitudes can be accessed as *llv[1]*." ] }, { "cell_type": "code", "execution_count": 11, "id": "754853f8-e394-445e-9eab-78bbf9eadf30", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 12)\n" ] }, { "data": { "text/plain": [ "0.0" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(llv[1].shape)\n", "# first longitude\n", "llv[1, 0, 0]" ] }, { "cell_type": "markdown", "id": "e4e9c3d3-6905-4aa9-9857-eb2a58406ac2", "metadata": {}, "source": [ "The field values can be accessed as *llv[2:]*." ] }, { "cell_type": "code", "execution_count": 12, "id": "9c461ac5-5eea-460d-af41-c93af51e6f45", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(6, 7, 12)\n", "272.5641784667969\n", "[272.56417847 272.53916931 271.26531982 255.84306335 244.00323486\n", " 226.65315247]\n" ] } ], "source": [ "print(llv[2:].shape)\n", "\n", "# first value in first field\n", "print(llv[2, 0, 0])\n", "\n", "# first value from all the fields\n", "print(llv[2:, 0, 0])" ] }, { "cell_type": "markdown", "id": "5b0cd71f-86da-4aab-b456-b50b382a6650", "metadata": {}, "source": [ "When using the *flatten* keyword 1D arrays are returned for latitude and longitude, and the values array will be flattened per field:" ] }, { "cell_type": "code", "execution_count": 13, "id": "dc0b8aa4-088b-4573-9688-46a0f0a3fdb1", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(8, 84)" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "llv = ds.data(flatten=True)\n", "llv.shape" ] }, { "cell_type": "markdown", "id": "04812891-8465-458b-b3f2-9f818de4939d", "metadata": {}, "source": [ "### Using the to_latlon() and to_numpy() methods" ] }, { "cell_type": "raw", "id": "bc08ca27-52e7-46e3-9780-7d2389956b81", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "We can also get the latitudes, longitudes and values by using the :py:meth:`~data.core.fieldlist.FieldList.to_latlon` and :py:meth:`~data.core.fieldlist.FieldList.to_to_numpy()` methods." ] }, { "cell_type": "markdown", "id": "20e7aa47-5882-42de-974a-f20a129c34e5", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "#### Fields" ] }, { "cell_type": "raw", "id": "5f169944-0ec0-4206-b572-1b76d304bbe2", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ ":py:meth:`~data.core.fieldlist.Field.to_latlon` returns a dict:" ] }, { "cell_type": "code", "execution_count": 14, "id": "b9af7f2c-f120-4437-b59f-b804ce15e84b", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "lat shape=(7, 12)\n", "lon shape=(7, 12)\n" ] } ], "source": [ "ll = ds[0].to_latlon()\n", "for k, v in ll.items():\n", " print(f\"{k} shape={v.shape}\")" ] }, { "cell_type": "raw", "id": "9c5d0956-cb63-4756-be81-1219ff097ac9", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "The field values can be accessed with :py:meth:`~data.core.fieldlist.Field.to_numpy`." ] }, { "cell_type": "code", "execution_count": 15, "id": "c0926aa8-7fa4-4bb6-adba-11711bc7e463", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "(7, 12)" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v = ds[0].to_numpy()\n", "v.shape" ] }, { "cell_type": "markdown", "id": "e6300133-6ae5-4871-b840-cf2fe42c20f3", "metadata": {}, "source": [ "By default both methods keep the field's shape, but we can use the *flatten* keyword to get 1D arrays:" ] }, { "cell_type": "code", "execution_count": 16, "id": "abfb402a-743e-44e5-a01b-bcc98f0e6ed4", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(84,)\n", "(84,)\n", "(84,)\n" ] } ], "source": [ "ll = ds[0].to_latlon(flatten=True)\n", "v = ds[0].to_numpy(flatten=True)\n", "print(ll[\"lat\"].shape)\n", "print(ll[\"lon\"].shape)\n", "print(v.shape)" ] }, { "cell_type": "markdown", "id": "e2b99149-db3b-4f6a-807d-228ef67148e0", "metadata": {}, "source": [ "#### FieldLists" ] }, { "cell_type": "raw", "id": "84f0e9f3-ae1c-4f32-8375-459de36a241f", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ ":py:meth:`~data.core.fieldlist.FieldList.to_latlon` only works on a fieldlist if all the fields have the same grid. In this case it returns the same dict that we would get for any of the fields:" ] }, { "cell_type": "code", "execution_count": 17, "id": "dda20d91-1f1b-4ec5-991e-e50b841312ea", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "lat shape=(7, 12)\n", "lon shape=(7, 12)\n" ] } ], "source": [ "ll = ds.to_latlon()\n", "for k, v in ll.items():\n", " print(f\"{k} shape={v.shape}\")" ] }, { "cell_type": "raw", "id": "6e2f64b5-97f4-45a3-9655-786fb73106bb", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ ":py:meth:`~data.core.fieldlist.FieldList.to_numpy` only works on a fieldlist if all the fields has the same grid. It returns the array of the field value arrays." ] }, { "cell_type": "code", "execution_count": 18, "id": "e21d501a-22ef-4e82-a996-261baa2594de", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "data": { "text/plain": [ "(6, 7, 12)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v = ds.to_numpy()\n", "v.shape" ] }, { "cell_type": "markdown", "id": "3ce8fd6b-4321-4f40-988a-5b03aa550062", "metadata": {}, "source": [ "E.g. the first value from each field can be extracted as:" ] }, { "cell_type": "code", "execution_count": 19, "id": "1edbf41c-437f-41b4-aadb-c840249519b8", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([272.56417847, 272.53916931, 271.26531982, 255.84306335,\n", " 244.00323486, 226.65315247])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v[:,0,0]" ] }, { "cell_type": "markdown", "id": "40254a7c-f70f-4219-8722-a000ae5680db", "metadata": {}, "source": [ "### Specifying the array type" ] }, { "cell_type": "markdown", "id": "3bbbaad8-f9b7-4faa-b9ad-7a696f391b40", "metadata": {}, "source": [ "For all the methods above we can set the array type with the *dtype* keyword both for fields and fieldlists:" ] }, { "cell_type": "code", "execution_count": 20, "id": "bf898af0-7382-40e8-89fd-8a07bc250378", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([272.56418, 272.53918, 271.26532, 255.84306, 244.00323, 226.65315],\n", " dtype=float32)" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "v = ds.to_numpy(dtype=np.float32)\n", "v[:,0,0]" ] }, { "cell_type": "code", "execution_count": 21, "id": "c89a473d-c6fd-4400-bfbc-da35bdcf8dc3", "metadata": {}, "outputs": [], "source": [ "llv = ds.data(dtype=np.float32)" ] }, { "cell_type": "code", "execution_count": 22, "id": "1e888bd9-48ec-4d18-8ef3-ee6d4fd107c0", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 90. , 0. , 272.56418, 272.53918, 271.26532, 255.84306,\n", " 244.00323, 226.65315], dtype=float32)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "llv[:,0,0]" ] } ], "metadata": { "kernelspec": { "display_name": "dev_ecc", "language": "python", "name": "dev_ecc" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.13" } }, "nbformat": 4, "nbformat_minor": 5 }