{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n# MHDweb Integration Part I\n\nThis is the first of a two-part series demonstrating how to use the\n[MHDweb REST API](https://predsci.com/mhdweb2/api) to retrieve\nPredictive Science MAS model output and spacecraft connectivity data.\n\n[MHDweb](https://predsci.com/mhdweb2) provides programmatic retrieval\nof MAS run HDF5 data files, querying of MHDweb's *MAS Run Database* /\n*Spacecraft Database*, and generation of various data products.\n\n.. attention::\n Access requires a PSI-issued API key (passed as ``Authorization: Api-Key `` in the\n header of every request).\n\n **To request a key, visit the link:** [MHDweb API Key](https://predsci.com/mhdweb2/api).\n\n\nThree endpoints are used here:\n\n- ``mas-run-db`` \u2014 searches the MAS run catalog by time, model, and variable.\n- ``mas-run-db/{id}/{domain}/{state}/{variable}`` \u2014 streams a ZIP archive of\n HDF5 field files for the requested domain and state.\n- ``spacecraft-mapping/{id}/{sc_id}`` \u2014 returns the magnetic connectivity\n mapping for a named spacecraft, serialized as an\n [Astropy ECSV](https://docs.astropy.org/en/stable/io/ascii/ecsv.html)\n byte stream.\n\nThe downloaded files are consumed in\n`sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py`.\n\n

Note

This example requires a valid ``API_KEY`` environment variable (or a\n ``.env`` file in the working directory). Run the script locally with\n your own key to reproduce the downloads.\n\n **To safeguard your API key, do not commit it to version control or share\n it publicly. Use environment variables or secure vaults to manage your credentials.**

\n\n.. seealso::\n\n `sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py`\n Part II \u2014 loads the files downloaded here, traces magnetic\n connectivity, and produces four visualizations.\n [MHDweb References](https://predsci.com/mhdweb2/references)\n A collections of resources for learning more about MHDweb, the MAS model, and related topics.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import os\nfrom pprint import pprint\nfrom pathlib import Path\nimport requests\nfrom io import BytesIO\nfrom astropy.table import Table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Configure Output Paths and Authentication\n\n``STATIC_ASSETS`` is an optional environment variable that redirects output\nto a shared asset directory used by the Sphinx pre-build pipeline. When\nunset, files are written to the current working directory.\n\nAuthentication follows the [MHDweb API key scheme](https://predsci.com/mhdweb2/api): the key is passed as\n``Authorization: Api-Key `` on every request.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "OUTPUT_DIR = Path(os.environ.get(\"STATIC_ASSETS\", \"\")).resolve()\nCOR_OUTPUT_DIR = OUTPUT_DIR / 'cor_mag_field'\nHEL_OUTPUT_DIR = OUTPUT_DIR / 'hel_mag_field'\nBASE_URL = 'https://www.predsci.com/mhdweb2_bu/v2/api'\nAPI_KEY = os.environ.get('API_KEY')\nAUTH = dict(Authorization=f\"Api-Key {API_KEY}\")\n\nif not COR_OUTPUT_DIR.exists():\n COR_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)\nif not HEL_OUTPUT_DIR.exists():\n HEL_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Query the MAS Run Database\n\nThe ``mas-run-db`` endpoint searches the MAS run catalog and returns a list\nof runs ranked by proximity to the requested time of interest (``toi``).\nKey query parameters:\n\n- ``toi`` \u2014 ISO-8601 timestamp; the API selects all runs with a run time\n range encompassing the requested time and ranks them by proximity to it.\n- ``model`` \u2014 MAS model variant *e.g.* ``'thermo_2'``, ``'thermo_1'``, ``'poly'``.\n- ``type`` \u2014 run type; ``'ss'`` is a steady-state solution.\n- ``domain`` \u2014 list of domains to require: ``'cor'`` (corona,\n $1\\text{\u2013}30\\,R_\\odot$) and ``'hel'`` (heliosphere, extending\n to $\\sim 1\\,\\mathrm{AU}$).\n- ``variables`` \u2014 field components needed for tracing:\n $(B_r, B_\\theta, B_\\phi)$.\n\nThe first result is taken; its ``id`` field (``cor_id``) is the run\nidentifier used in all subsequent requests.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "db_query_params = {\n 'toi': '2024-05-09T12:00:00',\n 'model': 'thermo_2',\n 'type': 'ss',\n 'domain': ['cor', 'hel'],\n 'variables': ['br', 'bt', 'bp'],\n}\n\ndb_response = requests.get(\n f'{BASE_URL}/mas-run-db',\n headers=dict(Accept='application/json') | AUTH,\n params=db_query_params)\ndb_response.raise_for_status()\n\nruns = db_response.json()\npprint(runs)\n\ntry:\n run = runs[0]\nexcept IndexError:\n raise IndexError(\"No results found in MAS Run DB for the specified parameters.\")\n\ncor_id = run['id']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Inspect Run Metadata\n\nFetching ``mas-run-db/{id}`` returns a metadata record for the run,\nincluding per-domain ``states`` lists. For steady-state runs there is\na single state (index ``0``); time-dependent runs have multiple states,\neach corresponding to one simulation snapshot.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "dbmeta_response = requests.get(\n f'{BASE_URL}/mas-run-db/{cor_id}',\n headers=dict(Accept='application/json') | AUTH)\ndbmeta_response.raise_for_status()\n\nrun_meta = dbmeta_response.json()\npprint(run_meta['cor'])\nlen(run_meta['cor']['states'])\n\npprint(run_meta['hel'])\nlen(run_meta['hel']['states'])\n\nprint(len(run_meta['omas']))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Download Magnetic Field Files\n\nThe endpoint ``mas-run-db/{cor_id}/{domain}/{state}/{variable}`` streams a\nZIP archive containing HDF5 files for the requested field components.\n``state='0'`` selects the first (and for steady-state runs, only) snapshot.\nRequesting ``variable='br,bt,bp'`` as a comma-separated string fetches all\nthree components in a single archive.\n\nThe coronal and heliospheric archives are downloaded and written separately\nsince they will be extracted to different directories in\n`sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py`.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "cor_files_params = {\n 'cor_id': str(cor_id),\n 'domain': 'cor',\n 'state': '0',\n 'variable': 'br,bt,bp'\n}\n\nprint(\"Fetching coronal magnetic field files...\")\ncor_files_response = requests.get(\n f'{BASE_URL}/mas-run-db/' + '/'.join(cor_files_params.values()),\n headers=AUTH,\n stream=True)\ncor_files_response.raise_for_status()\n\nprint(\"Saving coronal magnetic field files...\")\nwith open(COR_OUTPUT_DIR / 'cor_mag_field.zip', 'wb') as f:\n for chunk in cor_files_response.iter_content(chunk_size=8192):\n f.write(chunk)\n\nhel_files_params = {\n 'cor_id': str(cor_id),\n 'domain': 'hel',\n 'state': '0',\n 'variable': 'br,bt,bp'\n}\n\nprint(\"Fetching heliospheric magnetic field files...\")\nhel_files_response = requests.get(\n f'{BASE_URL}/mas-run-db/' + '/'.join(hel_files_params.values()),\n headers=AUTH,\n stream=True)\nhel_files_response.raise_for_status()\n\nprint(\"Saving heliospheric magnetic field files...\")\nwith open(HEL_OUTPUT_DIR / 'hel_mag_field.zip', 'wb') as f:\n for chunk in hel_files_response.iter_content(chunk_size=8192):\n f.write(chunk)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Download the Spacecraft Mapping\n\nThe ``spacecraft-mapping/{cor_id}/{sc_id}`` endpoint returns the magnetic\nconnectivity mapping for a named spacecraft over the run's time range.\nHere ``sc_id='solo'`` selects Solar Orbiter.\n\nThe response is an [Astropy ECSV](https://docs.astropy.org/en/stable/io/ascii/ecsv.html) byte stream.\nReading it into an :class:`astropy.table.Table` via :class:`~io.BytesIO` preserves\ncolumn units and metadata without writing a temporary file. In this case \u2013 for\ncontinuity with `sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py` \u2013\nthe table is written to disk as an ECSV file for use in Part II.\n\nEach row of the table corresponds to one time step and contains three sets\nof $(r, \\theta, \\phi)$ coordinates (in $R_\\odot$ and radians):\n\n- ``sc_pos_{r,t,p}`` \u2014 actual spacecraft position in the Carrington frame.\n- ``r1_pos_{r,t,p}`` \u2014 position ballistically mapped inward to the outer\n coronal boundary ($r_1 \\approx 30\\,R_\\odot$).\n- ``r0_pos_{r,t,p}`` \u2014 magnetic footpoint traced to the inner boundary\n ($r_0 = 1\\,R_\\odot$).\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "sc_id = 'solo'\n\nresponse = requests.get(\n f'{BASE_URL}/spacecraft-mapping/{cor_id}/{sc_id}',\n headers=AUTH,\n stream=True)\nresponse.raise_for_status()\n\nspacecraft_mapping = Table.read(BytesIO(response.content), format='ascii.ecsv')\nprint(spacecraft_mapping.info(out=None))\n\nspacecraft_mapping.write(OUTPUT_DIR / \"spacecraft_mapping.ecsv\", format='ascii.ecsv', overwrite=True)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.12" } }, "nbformat": 4, "nbformat_minor": 0 }