.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "gallery/99_advanced_plots/p09_integrating_mhdweb_p1.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_gallery_99_advanced_plots_p09_integrating_mhdweb_p1.py: MHDweb Integration Part I ========================= This is the first of a two-part series demonstrating how to use the `MHDweb REST API `_ to retrieve Predictive Science MAS model output and spacecraft connectivity data. `MHDweb `_ provides programmatic retrieval of MAS run HDF5 data files, querying of MHDweb's *MAS Run Database* / *Spacecraft Database*, and generation of various data products. .. attention:: Access requires a PSI-issued API key (passed as ``Authorization: Api-Key `` in the header of every request). **To request a key, visit the link:** `MHDweb API Key `_. Three endpoints are used here: - ``mas-run-db`` — searches the MAS run catalog by time, model, and variable. - ``mas-run-db/{id}/{domain}/{state}/{variable}`` — streams a ZIP archive of HDF5 field files for the requested domain and state. - ``spacecraft-mapping/{id}/{sc_id}`` — returns the magnetic connectivity mapping for a named spacecraft, serialized as an `Astropy ECSV `_ byte stream. The downloaded files are consumed in :ref:`sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py`. .. note:: This example requires a valid ``API_KEY`` environment variable (or a ``.env`` file in the working directory). Run the script locally with your own key to reproduce the downloads. **To safeguard your API key, do not commit it to version control or share it publicly. Use environment variables or secure vaults to manage your credentials.** .. seealso:: :ref:`sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py` Part II — loads the files downloaded here, traces magnetic connectivity, and produces four visualizations. `MHDweb References `_ A collections of resources for learning more about MHDweb, the MAS model, and related topics. .. GENERATED FROM PYTHON SOURCE LINES 50-59 .. code-block:: Python import os from pprint import pprint from pathlib import Path import requests from io import BytesIO from astropy.table import Table .. GENERATED FROM PYTHON SOURCE LINES 61-71 Configure Output Paths and Authentication ----------------------------------------- ``STATIC_ASSETS`` is an optional environment variable that redirects output to a shared asset directory used by the Sphinx pre-build pipeline. When unset, files are written to the current working directory. Authentication follows the `MHDweb API key scheme `_: the key is passed as ``Authorization: Api-Key `` on every request. .. GENERATED FROM PYTHON SOURCE LINES 71-84 .. code-block:: Python OUTPUT_DIR = Path(os.environ.get("STATIC_ASSETS", "")).resolve() COR_OUTPUT_DIR = OUTPUT_DIR / 'cor_mag_field' HEL_OUTPUT_DIR = OUTPUT_DIR / 'hel_mag_field' BASE_URL = 'https://www.predsci.com/mhdweb2_bu/v2/api' API_KEY = os.environ.get('API_KEY') AUTH = dict(Authorization=f"Api-Key {API_KEY}") if not COR_OUTPUT_DIR.exists(): COR_OUTPUT_DIR.mkdir(parents=True, exist_ok=True) if not HEL_OUTPUT_DIR.exists(): HEL_OUTPUT_DIR.mkdir(parents=True, exist_ok=True) .. GENERATED FROM PYTHON SOURCE LINES 85-104 Query the MAS Run Database -------------------------- The ``mas-run-db`` endpoint searches the MAS run catalog and returns a list of runs ranked by proximity to the requested time of interest (``toi``). Key query parameters: - ``toi`` — ISO-8601 timestamp; the API selects all runs with a run time range encompassing the requested time and ranks them by proximity to it. - ``model`` — MAS model variant *e.g.* ``'thermo_2'``, ``'thermo_1'``, ``'poly'``. - ``type`` — run type; ``'ss'`` is a steady-state solution. - ``domain`` — list of domains to require: ``'cor'`` (corona, :math:`1\text{–}30\,R_\odot`) and ``'hel'`` (heliosphere, extending to :math:`\sim 1\,\mathrm{AU}`). - ``variables`` — field components needed for tracing: :math:`(B_r, B_\theta, B_\phi)`. The first result is taken; its ``id`` field (``cor_id``) is the run identifier used in all subsequent requests. .. GENERATED FROM PYTHON SOURCE LINES 104-129 .. code-block:: Python db_query_params = { 'toi': '2024-05-09T12:00:00', 'model': 'thermo_2', 'type': 'ss', 'domain': ['cor', 'hel'], 'variables': ['br', 'bt', 'bp'], } db_response = requests.get( f'{BASE_URL}/mas-run-db', headers=dict(Accept='application/json') | AUTH, params=db_query_params) db_response.raise_for_status() runs = db_response.json() pprint(runs) try: run = runs[0] except IndexError: raise IndexError("No results found in MAS Run DB for the specified parameters.") cor_id = run['id'] .. rst-class:: sphx-glr-script-out .. code-block:: none [{'cor_model': 'mast2', 'end_time': '2024-06-02T10:22:03.179067', 'has_cor': True, 'has_hel': True, 'id': 522, 'name': 'HMI CR2284 Thermo 2', 'np': 299, 'nr': 254, 'nt': 142, 'one_sentence_summary': 'A CORHEL Simulation using the Thermo 2 Model using ' 'data from hmi', 'run_descriptor': 'MHDWeb', 'run_type': 'Steady-State', 'start_time': '2024-05-06T11:44:16.143627', 'surface_data_source': 'hmi'}] .. GENERATED FROM PYTHON SOURCE LINES 130-137 Inspect Run Metadata -------------------- Fetching ``mas-run-db/{id}`` returns a metadata record for the run, including per-domain ``states`` lists. For steady-state runs there is a single state (index ``0``); time-dependent runs have multiple states, each corresponding to one simulation snapshot. .. GENERATED FROM PYTHON SOURCE LINES 137-152 .. code-block:: Python dbmeta_response = requests.get( f'{BASE_URL}/mas-run-db/{cor_id}', headers=dict(Accept='application/json') | AUTH) dbmeta_response.raise_for_status() run_meta = dbmeta_response.json() pprint(run_meta['cor']) len(run_meta['cor']['states']) pprint(run_meta['hel']) len(run_meta['hel']['states']) print(len(run_meta['omas'])) .. rst-class:: sphx-glr-script-out .. code-block:: none {'states': [{'id': 1044, 'sequence': '002', 'state_t0': '2024-05-06T11:44:16.143627', 'state_t1': '2024-06-02T10:22:03.179067', 'time': '2024-05-19T23:03:09.661347'}], 'variables': ['br', 'bt', 'bp', 'jr', 'jt', 'jp', 'vr', 'vt', 'vp', 'em', 'ep', 'heat', 'rho', 't']} {'states': [{'id': 1045, 'sequence': '002', 'state_t0': '2024-05-06T11:44:16.143627', 'state_t1': '2024-06-02T10:22:03.179067', 'time': '2024-05-19T23:03:09.661347'}], 'variables': ['br', 'bt', 'bp', 'jr', 'jt', 'jp', 'vr', 'vt', 'vp', 'rho', 't']} 66340 .. GENERATED FROM PYTHON SOURCE LINES 153-165 Download Magnetic Field Files ------------------------------ The endpoint ``mas-run-db/{cor_id}/{domain}/{state}/{variable}`` streams a ZIP archive containing HDF5 files for the requested field components. ``state='0'`` selects the first (and for steady-state runs, only) snapshot. Requesting ``variable='br,bt,bp'`` as a comma-separated string fetches all three components in a single archive. The coronal and heliospheric archives are downloaded and written separately since they will be extracted to different directories in :ref:`sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py`. .. GENERATED FROM PYTHON SOURCE LINES 165-204 .. code-block:: Python cor_files_params = { 'cor_id': str(cor_id), 'domain': 'cor', 'state': '0', 'variable': 'br,bt,bp' } print("Fetching coronal magnetic field files...") cor_files_response = requests.get( f'{BASE_URL}/mas-run-db/' + '/'.join(cor_files_params.values()), headers=AUTH, stream=True) cor_files_response.raise_for_status() print("Saving coronal magnetic field files...") with open(COR_OUTPUT_DIR / 'cor_mag_field.zip', 'wb') as f: for chunk in cor_files_response.iter_content(chunk_size=8192): f.write(chunk) hel_files_params = { 'cor_id': str(cor_id), 'domain': 'hel', 'state': '0', 'variable': 'br,bt,bp' } print("Fetching heliospheric magnetic field files...") hel_files_response = requests.get( f'{BASE_URL}/mas-run-db/' + '/'.join(hel_files_params.values()), headers=AUTH, stream=True) hel_files_response.raise_for_status() print("Saving heliospheric magnetic field files...") with open(HEL_OUTPUT_DIR / 'hel_mag_field.zip', 'wb') as f: for chunk in hel_files_response.iter_content(chunk_size=8192): f.write(chunk) .. rst-class:: sphx-glr-script-out .. code-block:: none Fetching coronal magnetic field files... Saving coronal magnetic field files... Fetching heliospheric magnetic field files... Saving heliospheric magnetic field files... .. GENERATED FROM PYTHON SOURCE LINES 205-227 Download the Spacecraft Mapping -------------------------------- The ``spacecraft-mapping/{cor_id}/{sc_id}`` endpoint returns the magnetic connectivity mapping for a named spacecraft over the run's time range. Here ``sc_id='solo'`` selects Solar Orbiter. The response is an `Astropy ECSV `_ byte stream. Reading it into an :class:`astropy.table.Table` via :class:`~io.BytesIO` preserves column units and metadata without writing a temporary file. In this case – for continuity with :ref:`sphx_glr_gallery_99_advanced_plots_p11_integrating_mhdweb_p2.py` – the table is written to disk as an ECSV file for use in Part II. Each row of the table corresponds to one time step and contains three sets of :math:`(r, \theta, \phi)` coordinates (in :math:`R_\odot` and radians): - ``sc_pos_{r,t,p}`` — actual spacecraft position in the Carrington frame. - ``r1_pos_{r,t,p}`` — position ballistically mapped inward to the outer coronal boundary (:math:`r_1 \approx 30\,R_\odot`). - ``r0_pos_{r,t,p}`` — magnetic footpoint traced to the inner boundary (:math:`r_0 = 1\,R_\odot`). .. GENERATED FROM PYTHON SOURCE LINES 227-239 .. code-block:: Python sc_id = 'solo' response = requests.get( f'{BASE_URL}/spacecraft-mapping/{cor_id}/{sc_id}', headers=AUTH, stream=True) response.raise_for_status() spacecraft_mapping = Table.read(BytesIO(response.content), format='ascii.ecsv') print(spacecraft_mapping.info(out=None)) spacecraft_mapping.write(OUTPUT_DIR / "spacecraft_mapping.ecsv", format='ascii.ecsv', overwrite=True) .. rst-class:: sphx-glr-script-out .. code-block:: none name dtype shape unit format description class n_bad length ------------------------- ------- ----- ------ ------ ----------- ------ ----- ------ sc_time object Time 0 983 sc_pos_r float64 solRad Column 0 983 sc_pos_t float64 rad Column 0 983 sc_pos_p float64 rad Column 0 983 r1_time object Time 0 983 ... ... ... ... ... ... ... ... ... r0_pos_p float64 rad Column 358 983 flow_speed float64 km / s Column 0 983 estimated_flow_speed_400 bool Column 0 983 r1_timestamp_within_state bool Column 0 983 r1_timestamp_within_run bool Column 0 983 r1_position_maps_to_r0 bool Column 0 983 Length = 17 rows .. rst-class:: sphx-glr-timing **Total running time of the script:** (1 minutes 2.004 seconds) .. _sphx_glr_download_gallery_99_advanced_plots_p09_integrating_mhdweb_p1.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: p09_integrating_mhdweb_p1.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: p09_integrating_mhdweb_p1.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: p09_integrating_mhdweb_p1.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_