{ "cells": [ { "cell_type": "markdown", "id": "9c53d7c8-7f38-4a5f-93d0-e05919408b6e", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## GRIB: converting to NetCDF" ] }, { "cell_type": "code", "execution_count": 1, "id": "950b9aa8-6b76-47f8-b470-17476f99a733", "metadata": {}, "outputs": [], "source": [ "import earthkit.data as ekd\n", "ekd.download_example_file(\"tuv_pl.grib\")\n", "# we only select the temperature fields\n", "ds = ekd.from_source(\"file\", \"tuv_pl.grib\").sel(param=\"t\")" ] }, { "cell_type": "markdown", "id": "fac6cee2-845e-4d34-94f0-cdf53a757e5a", "metadata": {}, "source": [ "#### Using the earthkit accessor" ] }, { "cell_type": "raw", "id": "d125684b-20c9-49d9-a87c-7880814b60fc", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "To convert GRIB data to NetCDF first we need to convert GRIB to Xarray with :py:meth:`~data.readers.grib.index.GribFieldList.to_xarray` then generate NetCDF from it with :py:meth:`xarray.Dataset.to_netcdf`. Earthkit-data attaches some special attributes to the generated Xarray dataset that cannot be written to NetCDF. In order to make ``to_netcdf`` work we need to call it on the \"earthkit\" accessor and not directly on the Xarray dataset." ] }, { "cell_type": "code", "execution_count": 2, "id": "0efd9676-6c4f-4e07-a14a-a5337de50461", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "ds.to_xarray().earthkit.to_netcdf(\"_tuv_pl.nc\")" ] }, { "cell_type": "code", "execution_count": 3, "id": "a45aa289-8c85-42bd-a637-8999d8720fcf", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "netcdf _tuv_pl {\n", "dimensions:\n", "\tlevel = 6 ;\n", "\tlatitude = 7 ;\n", "\tlongitude = 12 ;\n", "variables:\n", "\tdouble t(level, latitude, longitude) ;\n", "\t\tt:_FillValue = NaN ;\n", "\t\tt:standard_name = \"air_temperature\" ;\n", "\t\tt:long_name = \"Temperature\" ;\n", "\t\tt:units = \"K\" ;\n", "\tint64 level(level) ;\n", "\t\tlevel:units = \"hPa\" ;\n", "\t\tlevel:positive = \"down\" ;\n", "\t\tlevel:stored_direction = \"decreasing\" ;\n", "\t\tlevel:standard_name = \"air_pressure\" ;\n", "\t\tlevel:long_name = \"pressure\" ;\n", "\tdouble latitude(latitude) ;\n", "\t\tlatitude:_FillValue = NaN ;\n", "\t\tlatitude:units = \"degrees_north\" ;\n", "\t\tlatitude:standard_name = \"latitude\" ;\n", "\t\tlatitude:long_name = \"latitude\" ;\n", "\tdouble longitude(longitude) ;\n", "\t\tlongitude:_FillValue = NaN ;\n", "\t\tlongitude:units = \"degrees_east\" ;\n", "\t\tlongitude:standard_name = \"longitude\" ;\n", "\t\tlongitude:long_name = \"longitude\" ;\n", "\n", "// global attributes:\n", "\t\t:param = \"t\" ;\n", "\t\t:paramId = 130LL ;\n", "\t\t:class = \"od\" ;\n", "\t\t:stream = \"oper\" ;\n", "\t\t:levtype = \"pl\" ;\n", "\t\t:type = \"an\" ;\n", "\t\t:expver = \"0001\" ;\n", "\t\t:date = 20180801LL ;\n", "\t\t:time = 1200LL ;\n", "\t\t:domain = \"g\" ;\n", "\t\t:number = 0LL ;\n", "\t\t:Conventions = \"CF-1.8\" ;\n", "\t\t:institution = \"ECMWF\" ;\n", "}\n", "\n" ] } ], "source": [ "from earthkit.data.utils.summary import ncdump \n", "ncdump(\"_tuv_pl.nc\")" ] }, { "cell_type": "markdown", "id": "c56a9200-8695-419c-bfc6-30008faa479c", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "#### Using add_earthkit_attrs=False" ] }, { "cell_type": "raw", "id": "8b71b1c4-7c43-46cf-b6a0-1b38af1a3f74", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Alternatively, we can use the ``add_earthkit_attrs=False`` option in :py:meth:`~data.readers.grib.index.GribFieldList.to_xarray`. With this the earthkit attributes are not added to the generated dataset and it is safe to call :py:meth:`to_netcdf ` directly on it." ] }, { "cell_type": "code", "execution_count": 4, "id": "156e839c-2b03-41d7-a1d0-3c049bc81888", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "netcdf _tuv_pl_1 {\n", "dimensions:\n", "\tlevel = 6 ;\n", "\tlatitude = 7 ;\n", "\tlongitude = 12 ;\n", "variables:\n", "\tdouble t(level, latitude, longitude) ;\n", "\t\tt:_FillValue = NaN ;\n", "\t\tt:standard_name = \"air_temperature\" ;\n", "\t\tt:long_name = \"Temperature\" ;\n", "\t\tt:units = \"K\" ;\n", "\tint64 level(level) ;\n", "\t\tlevel:units = \"hPa\" ;\n", "\t\tlevel:positive = \"down\" ;\n", "\t\tlevel:stored_direction = \"decreasing\" ;\n", "\t\tlevel:standard_name = \"air_pressure\" ;\n", "\t\tlevel:long_name = \"pressure\" ;\n", "\tdouble latitude(latitude) ;\n", "\t\tlatitude:_FillValue = NaN ;\n", "\t\tlatitude:units = \"degrees_north\" ;\n", "\t\tlatitude:standard_name = \"latitude\" ;\n", "\t\tlatitude:long_name = \"latitude\" ;\n", "\tdouble longitude(longitude) ;\n", "\t\tlongitude:_FillValue = NaN ;\n", "\t\tlongitude:units = \"degrees_east\" ;\n", "\t\tlongitude:standard_name = \"longitude\" ;\n", "\t\tlongitude:long_name = \"longitude\" ;\n", "\n", "// global attributes:\n", "\t\t:param = \"t\" ;\n", "\t\t:paramId = 130LL ;\n", "\t\t:class = \"od\" ;\n", "\t\t:stream = \"oper\" ;\n", "\t\t:levtype = \"pl\" ;\n", "\t\t:type = \"an\" ;\n", "\t\t:expver = \"0001\" ;\n", "\t\t:date = 20180801LL ;\n", "\t\t:time = 1200LL ;\n", "\t\t:domain = \"g\" ;\n", "\t\t:number = 0LL ;\n", "\t\t:Conventions = \"CF-1.8\" ;\n", "\t\t:institution = \"ECMWF\" ;\n", "}\n", "\n" ] } ], "source": [ "ds.to_xarray(add_earthkit_attrs=False).to_netcdf(\"_tuv_pl_1.nc\")\n", "ncdump(\"_tuv_pl_1.nc\")" ] }, { "cell_type": "markdown", "id": "23a28542-078b-488c-8b0b-58c7c1add2ba", "metadata": {}, "source": [ "#### Using to_target" ] }, { "cell_type": "raw", "id": "7a1e9600-5b49-4fc8-aef8-ac0b51a42f72", "metadata": { "editable": true, "raw_mimetype": "text/restructuredtext", "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "We can call :func:`to_target` on the fieldlist and the Xarray conversion and writing to NetCDF will happen automatically under the hood using the default options. In this case ``add_earthkit_attrs=False`` is always enforced." ] }, { "cell_type": "code", "execution_count": 5, "id": "1bdebe02-5f4f-4e3f-b434-ebab25959577", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "netcdf _tuv_pl_2 {\n", "dimensions:\n", "\tlevel = 6 ;\n", "\tlatitude = 7 ;\n", "\tlongitude = 12 ;\n", "variables:\n", "\tdouble longitude(longitude) ;\n", "\t\tlongitude:units = \"degrees_east\" ;\n", "\t\tlongitude:standard_name = \"longitude\" ;\n", "\t\tlongitude:long_name = \"longitude\" ;\n", "\t\tlongitude:_FillValue = NaN ;\n", "\tdouble latitude(latitude) ;\n", "\t\tlatitude:units = \"degrees_north\" ;\n", "\t\tlatitude:standard_name = \"latitude\" ;\n", "\t\tlatitude:long_name = \"latitude\" ;\n", "\t\tlatitude:_FillValue = NaN ;\n", "\tdouble t(level, latitude, longitude) ;\n", "\t\tt:standard_name = \"air_temperature\" ;\n", "\t\tt:long_name = \"Temperature\" ;\n", "\t\tt:units = \"K\" ;\n", "\t\tt:_FillValue = NaN ;\n", "\tint level(level) ;\n", "\t\tlevel:units = \"hPa\" ;\n", "\t\tlevel:positive = \"down\" ;\n", "\t\tlevel:stored_direction = \"decreasing\" ;\n", "\t\tlevel:standard_name = \"air_pressure\" ;\n", "\t\tlevel:long_name = \"pressure\" ;\n", "\n", "// global attributes:\n", "\t\t:param = \"t\" ;\n", "\t\t:paramId = 130 ;\n", "\t\t:class = \"od\" ;\n", "\t\t:stream = \"oper\" ;\n", "\t\t:levtype = \"pl\" ;\n", "\t\t:type = \"an\" ;\n", "\t\t:expver = \"0001\" ;\n", "\t\t:date = 20180801 ;\n", "\t\t:time = 1200 ;\n", "\t\t:domain = \"g\" ;\n", "\t\t:number = 0 ;\n", "\t\t:Conventions = \"CF-1.8\" ;\n", "\t\t:institution = \"ECMWF\" ;\n", "}\n", "\n" ] } ], "source": [ "ds.to_target(\"file\", \"_tuv_pl_2.nc\")\n", "ncdump(\"_tuv_pl_2.nc\")" ] }, { "cell_type": "markdown", "id": "0ebd29d0-cce4-4d3e-bbd7-2fcc1729c563", "metadata": {}, "source": [ "To control the Xarray conversion we can pass options to the earthkit Xarray engine via ``earthkit_to_xarray_kwargs``." ] }, { "cell_type": "code", "execution_count": 6, "id": "669cfc69-f64b-490b-83cc-2415ec42ab7a", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "netcdf _tuv_pl_3 {\n", "dimensions:\n", "\tlevel = 6 ;\n", "\tvalues = 84 ;\n", "variables:\n", "\tdouble latitude(values) ;\n", "\t\tlatitude:units = \"degrees_north\" ;\n", "\t\tlatitude:standard_name = \"latitude\" ;\n", "\t\tlatitude:long_name = \"latitude\" ;\n", "\t\tlatitude:_FillValue = NaN ;\n", "\tdouble longitude(values) ;\n", "\t\tlongitude:units = \"degrees_east\" ;\n", "\t\tlongitude:standard_name = \"longitude\" ;\n", "\t\tlongitude:long_name = \"longitude\" ;\n", "\t\tlongitude:_FillValue = NaN ;\n", "\tdouble t(level, values) ;\n", "\t\tt:standard_name = \"air_temperature\" ;\n", "\t\tt:long_name = \"Temperature\" ;\n", "\t\tt:units = \"K\" ;\n", "\t\tt:coordinates = \"latitude longitude\" ;\n", "\t\tt:_FillValue = NaN ;\n", "\tint level(level) ;\n", "\t\tlevel:units = \"hPa\" ;\n", "\t\tlevel:positive = \"down\" ;\n", "\t\tlevel:stored_direction = \"decreasing\" ;\n", "\t\tlevel:standard_name = \"air_pressure\" ;\n", "\t\tlevel:long_name = \"pressure\" ;\n", "\n", "// global attributes:\n", "\t\t:param = \"t\" ;\n", "\t\t:paramId = 130 ;\n", "\t\t:class = \"od\" ;\n", "\t\t:stream = \"oper\" ;\n", "\t\t:levtype = \"pl\" ;\n", "\t\t:type = \"an\" ;\n", "\t\t:expver = \"0001\" ;\n", "\t\t:date = 20180801 ;\n", "\t\t:time = 1200 ;\n", "\t\t:domain = \"g\" ;\n", "\t\t:number = 0 ;\n", "\t\t:Conventions = \"CF-1.8\" ;\n", "\t\t:institution = \"ECMWF\" ;\n", "}\n", "\n" ] } ], "source": [ "ds.to_target(\"file\", \"_tuv_pl_3.nc\", earthkit_to_xarray_kwargs={\"flatten_values\": True})\n", "ncdump(\"_tuv_pl_3.nc\")" ] }, { "cell_type": "code", "execution_count": null, "id": "80f49214-43c6-43be-af56-9d55a16f0f8c", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "dev", "language": "python", "name": "dev" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.12" } }, "nbformat": 4, "nbformat_minor": 5 }