{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "hourly-multimedia",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "## GRIB: writing data into FDB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "behind-carry",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import earthkit.data as ekd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "numerous-france",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "This example demonstrates how to **write earthkit-data GRIB fields into an FDB**. \n",
    "\n",
    "FDB (Fields DataBase) is a domain-specific object store developed at ECMWF for storing, indexing and retrieving GRIB data. For more information on FBD please consult the following pages:\n",
    "\n",
    "- [FDB](https://fields-database.readthedocs.io/en/latest/)\n",
    "- [pyfdb](https://pyfdb.readthedocs.io/en/latest/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5f59f05-2596-407f-9c1d-eb18d0e2e283",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### Setting up the target FDB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d08c2850-8bd2-4c01-bc4c-2a941acb8e1a",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "In this example we will create an FDB in the current folder using the schema taken from the pyfdb test suite. To do so first we need to ensure the directory exists. Next, we have to specify the configuration.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "drawn-renewal",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "fdb_schema = \"./default_fdb_schema\"\n",
    "fdb_dir = \"./_fdb\"\n",
    "os.makedirs(fdb_dir, exist_ok=True)\n",
    "\n",
    "config = {\"type\":\"local\",\n",
    "     \"engine\":\"toc\",\n",
    "     \"schema\":fdb_schema,\n",
    "     \"spaces\":[{\"handler\":\"Default\",\n",
    "                \"roots\":[{\"path\":fdb_dir}]}]}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "compound-pastor",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "#### Archiving GRIB into the target FDB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "considered-excellence",
   "metadata": {},
   "source": [
    "First, we read temperature and wind GRIB data on pressure levels from a file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "signal-rocket",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>centre</th>\n",
       "      <th>shortName</th>\n",
       "      <th>typeOfLevel</th>\n",
       "      <th>level</th>\n",
       "      <th>dataDate</th>\n",
       "      <th>dataTime</th>\n",
       "      <th>stepRange</th>\n",
       "      <th>dataType</th>\n",
       "      <th>number</th>\n",
       "      <th>gridType</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ecmf</td>\n",
       "      <td>t</td>\n",
       "      <td>isobaricInhPa</td>\n",
       "      <td>1000</td>\n",
       "      <td>20180801</td>\n",
       "      <td>1200</td>\n",
       "      <td>0</td>\n",
       "      <td>an</td>\n",
       "      <td>0</td>\n",
       "      <td>regular_ll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ecmf</td>\n",
       "      <td>u</td>\n",
       "      <td>isobaricInhPa</td>\n",
       "      <td>1000</td>\n",
       "      <td>20180801</td>\n",
       "      <td>1200</td>\n",
       "      <td>0</td>\n",
       "      <td>an</td>\n",
       "      <td>0</td>\n",
       "      <td>regular_ll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ecmf</td>\n",
       "      <td>v</td>\n",
       "      <td>isobaricInhPa</td>\n",
       "      <td>1000</td>\n",
       "      <td>20180801</td>\n",
       "      <td>1200</td>\n",
       "      <td>0</td>\n",
       "      <td>an</td>\n",
       "      <td>0</td>\n",
       "      <td>regular_ll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ecmf</td>\n",
       "      <td>t</td>\n",
       "      <td>isobaricInhPa</td>\n",
       "      <td>850</td>\n",
       "      <td>20180801</td>\n",
       "      <td>1200</td>\n",
       "      <td>0</td>\n",
       "      <td>an</td>\n",
       "      <td>0</td>\n",
       "      <td>regular_ll</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ecmf</td>\n",
       "      <td>u</td>\n",
       "      <td>isobaricInhPa</td>\n",
       "      <td>850</td>\n",
       "      <td>20180801</td>\n",
       "      <td>1200</td>\n",
       "      <td>0</td>\n",
       "      <td>an</td>\n",
       "      <td>0</td>\n",
       "      <td>regular_ll</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  centre shortName    typeOfLevel  level  dataDate  dataTime stepRange  \\\n",
       "0   ecmf         t  isobaricInhPa   1000  20180801      1200         0   \n",
       "1   ecmf         u  isobaricInhPa   1000  20180801      1200         0   \n",
       "2   ecmf         v  isobaricInhPa   1000  20180801      1200         0   \n",
       "3   ecmf         t  isobaricInhPa    850  20180801      1200         0   \n",
       "4   ecmf         u  isobaricInhPa    850  20180801      1200         0   \n",
       "\n",
       "  dataType  number    gridType  \n",
       "0       an       0  regular_ll  \n",
       "1       an       0  regular_ll  \n",
       "2       an       0  regular_ll  \n",
       "3       an       0  regular_ll  \n",
       "4       an       0  regular_ll  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ekd.download_example_file(\"tuv_pl.grib\")\n",
    "ds = ekd.from_source(\"file\", \"tuv_pl.grib\")\n",
    "ds.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25b624e7-bef3-4048-9168-a92e5b717568",
   "metadata": {},
   "source": [
    "Next, we write the whole fieldlist into the FDB."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "293022a5-9cda-47ae-b89b-5ae749f41b27",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ds.to_target(\"fdb\", config=config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6036d568-76b4-475a-99dd-9b6bd4f0ddc1",
   "metadata": {},
   "source": [
    "#### Testing our FDB"
   ]
  },
  {
   "cell_type": "raw",
   "id": "7a0c52e4-7f30-4ce5-9b6c-2106e31e61f7",
   "metadata": {
    "editable": true,
    "raw_mimetype": "text/restructuredtext",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "We test our FDB by retrieving some fields from it using :ref:`from_source() <data-sources-fdb>`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d9e52ef7-1673-4a04-8ad7-f2fd5d65dbd7",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GribField(u,500,20180801,1200,0,0)\n",
      "GribField(v,500,20180801,1200,0,0)\n"
     ]
    }
   ],
   "source": [
    "request = {\n",
    "    'class': 'od',\n",
    "    'expver': '0001',\n",
    "    'stream': 'oper',\n",
    "    'date': '20180801',\n",
    "    'time': 1200,\n",
    "    'domain': 'g',\n",
    "    'type': 'an',\n",
    "    'levtype': 'pl',\n",
    "    'levelist': 500,\n",
    "    'step': 0,\n",
    "    'param': [131, 132]\n",
    "}\n",
    "\n",
    "# read fields from fdb as a stream\n",
    "for f in ekd.from_source(\"fdb\", request, config=config):\n",
    "    print(f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d0dc600-0996-4ed4-a2c7-2fd976ae5780",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dev_ecc",
   "language": "python",
   "name": "dev_ecc"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}