{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d08d9138-0a0a-495c-b2f0-27b47e45847c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## GRIB: using the metadata object"
   ]
  },
  {
   "cell_type": "raw",
   "id": "25dd1879-9f88-4555-a847-a0ed45e3d6e4",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "In this notebook we will work with a GRIB file containing 6 messages. First we ensure the example file is available, then read the file with :ref:`from_source() <data-sources-file>`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d1294cff-422a-4ed5-a1cf-87c6cf9195ad",
   "metadata": {
    "editable": true,
    "raw_mimetype": "",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import earthkit.data as ekd\n",
    "ekd.download_example_file(\"test6.grib\")\n",
    "ds = ekd.from_source(\"file\", \"test6.grib\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3392799-8f51-47f3-95b1-4a53d67a2c29",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### The metadata object"
   ]
  },
  {
   "cell_type": "raw",
   "id": "b2c90171-2ce4-445c-9993-3354286ba570",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": [],
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "The recommended way to get field metadata for a given key or keys is to use :py:meth:`~data.readers.grib.codes.GribField.metadata`. See :ref:`/examples/grib_metadata.ipynb` for more examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9daeae60-e906-4bd0-9055-e940218bb12c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'t'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds[0].metadata(\"shortName\")"
   ]
  },
  {
   "cell_type": "raw",
   "id": "9282df0e-2430-44fb-9025-29691bb3a24b",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Internally the field metadata is represented by a GribFieldMetadata object. Calling :py:meth:`~data.readers.grib.codes.GribField.metadata` without any arguments will return this object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1b795c50-f942-4417-b7e3-bd4aee4d11f5",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<earthkit.data.readers.grib.metadata.GribFieldMetadata at 0x2a919e170>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md = ds[0].metadata()\n",
    "md"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ad46cbe-9413-435a-bf5c-db2cb8cf8203",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "It can be used as a dict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5d5a9ad8-981e-440e-8327-9b21468ae42c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('t', 1000)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md[\"shortName\"], md.get(\"level\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d422ff7-c5ef-4f8d-b633-ecd93e418f3f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "<b>Warning:</b> this object <b>does not own a GRIB handle</b> but contains a reference to the field, which provides access to the GRIB handle. Therefore if you need to store it for later use it is recommended to create a copy with override() (see below) to decouple it from the field object.</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e50bd099-336f-4952-941b-aa58e93f4c70",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### Creating a copy of the metadata object"
   ]
  },
  {
   "cell_type": "raw",
   "id": "6cd89c6c-bcd5-4f83-b83b-03c28447ae41",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "We can create a copy with :py:meth:`~data.readers.grib.metadata.GribMetadata.override`, which always returns a new metadata object storing a cloned GRIB handle, thus it is decoupled from the field. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1369ca99-40d2-4352-8c9e-94666e5ebeec",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "md_copy = md.override()"
   ]
  },
  {
   "cell_type": "raw",
   "id": "1c511db3-f841-4936-a7cb-25dd3e3a1010",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "By default :py:meth:`~data.readers.grib.metadata.GribMetadata.override` is called with the ``headers_only_clone=True`` option to clone a new GRIB handle with all the data values (and some related information) removed. With this the resulting object can be significantly smaller, especially if the data section is large. The downside is that now the value related keys either cannot be accessed or give back wrong values. E.g when using the \"average\" key we get:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "c9ab97f0-40b7-403e-b7c9-6ce16c894d7d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(279.70703560965404, 47485.4296875)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md[\"average\"], md_copy[\"average\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe340c05-ac04-4f7d-8c43-3ed1f415a979",
   "metadata": {},
   "source": [
    "To get a copy without shrinking the GRIB handle use ``headers_only_clone=False``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a789bb5e-b493-45cd-b7ce-e5cb8b34e125",
   "metadata": {},
   "outputs": [],
   "source": [
    "md_copy_full = md.override(headers_only_clone=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3b030549-2a7e-494e-bd46-0a4cd19a09ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "279.70703560965404"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md_copy_full[\"average\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3240cef2-baa9-4a87-a83a-dafa97b78e43",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### Changing the metadata"
   ]
  },
  {
   "cell_type": "raw",
   "id": "449f7fe0-222b-456f-bb15-dbe263774f4b",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "To change metadata values pass key value pairs or a dict to :py:meth:`~data.readers.grib.metadata.GribMetadata.override`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "40c6d232-03de-402b-82bf-8647e8a7bece",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('z', 850)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md_mod = md.override(shortName=\"z\", level=\"850\")\n",
    "md_mod[\"shortName\"], md_mod[\"level\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31b017d2-843a-4dc5-8d49-b8c05f30ed37",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "Since md_mod contains a clone of the GRIB handle the original metadata did not change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ef78a3ec-4ea2-4ff5-8c90-e60b5e07e77f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('t', 1000)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "md[\"shortName\"], md[\"level\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "759e4311-2419-4790-883a-84884190a8c7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### Creating array fields from metadata and values"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0316233b-8da2-49a0-b8b8-4cedd6aba6b0",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "GRIB metadata objects play a part in building new fieldlists from (altered) values and metadata."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a691f6ef-4b5b-47d0-accf-5e956702b47c",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "The following example computes the wind speed on 1000 hPa and creates a new fieldlist with a single field containing the new values. The metadata is taken from the u field and the shortName is modified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cb59ad5f-c48b-4943-984d-3abdf48fda8d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from earthkit.data import FieldList\n",
    "import numpy as np\n",
    "u = ds.sel(param=\"u\", level=1000)[0]\n",
    "v = ds.sel(param=\"v\", level=1000)[0]\n",
    "speed = np.sqrt(u.values**2 + v.values**2)\n",
    "md_speed = u.metadata().override(shortName=\"ws\")\n",
    "ds_speed = FieldList.from_array(speed, md_speed) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "85c32bfb-c929-404f-add9-9adae40418d2",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>centre</th>\n",
       "      <th>shortName</th>\n",
       "      <th>typeOfLevel</th>\n",
       "      <th>level</th>\n",
       "      <th>dataDate</th>\n",
       "      <th>dataTime</th>\n",
       "      <th>stepRange</th>\n",
       "      <th>dataType</th>\n",
       "      <th>number</th>\n",
       "      <th>gridType</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ecmf</td>\n",
       "      <td>ws</td>\n",
       "      <td>isobaricInhPa</td>\n",
       "      <td>1000</td>\n",
       "      <td>20180801</td>\n",
       "      <td>1200</td>\n",
       "      <td>0</td>\n",
       "      <td>an</td>\n",
       "      <td>0</td>\n",
       "      <td>regular_ll</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  centre shortName    typeOfLevel  level  dataDate  dataTime stepRange  \\\n",
       "0   ecmf        ws  isobaricInhPa   1000  20180801      1200         0   \n",
       "\n",
       "  dataType  number    gridType  \n",
       "0       an       0  regular_ll  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds_speed.ls()"
   ]
  },
  {
   "cell_type": "raw",
   "id": "a5b3c420-f71d-4a58-8cbe-761a0ce73a9b",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": [],
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "The resulting fieldlist contains an :py:class:`~data.sources.array_list.ArrayField`, which is  composed of a numpy array storing the values and a metadata object owning its own GRIB handle with a trimmed down data section. Since the values array is decoupled from the GRIB handle stored in the metadata object, accessing metadata keys related to the data values is forbidden. Getting metadata on these keys will throw an exception. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "c6fe87ed-ee88-4f4d-a2b6-9401b364e2df",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<earthkit.data.readers.grib.metadata.RestrictedGribMetadata at 0x2a9229180>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds_speed[0].metadata()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "27686ac4-9382-4916-ad0e-be96a649d034",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Wind speed'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds_speed[0].metadata(\"name\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "dc28fa77-4020-431f-ad37-e480a69f9d7f",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KeyError 'average'\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    ds_speed[0].metadata(\"average\")\n",
    "except KeyError as e:\n",
    "    print(f\"KeyError {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "331fa5aa-a134-423e-a4e3-bc8adf779297",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "However, strictly speaking these keys do not represent metadata and they can be easily computed from the field values when needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8eab3462-3661-4fc1-9d23-8be05dc99cd8",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7.450183054360252"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds_speed[0].values.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "351236ef-3b2a-4d1e-bef9-3a7154af2270",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dev_ecc",
   "language": "python",
   "name": "dev_ecc"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}