MSWMS/WMS - A OGC Web Map Server
The module implements a WSGI Flask based Web Map Service 1.1.1/1.3.0 interface to provide forecast data
from numerical weather predictions to the Mission Support User Interface. Supported operations are GetCapabilities and GetMap for (WMS 1.1.1/1.3.0 compliant) maps and (non-compliant) vertical sections.
Configure the WMS server by modifying the settings in mswms_settings.py (address, products that shall be offered, …).
If you want to define new visualisation styles, the files to put them are mpl_hsec_styles.py and mpl_vsec_styles for maps and vertical sections, respectively.
For more information on WMS, see http://www.opengeospatial.org/standards/wms
Simulated Data and its configuration
We provide demodata by executing the mswms_demodata --seed
program. This creates in your home directory
data files and also the needed server configuration files. The program creates 70MB of examples.
This script does not overwrite an existing mswms_settings.py.
mss
├── mswms_auth.py
├── mswms_settings.py
└── testdata
├── 20121017_12_ecmwf_forecast.ALTITUDE_LEVELS.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.CC.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.CIWC.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.CLWC.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.EMAC.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.P_derived.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.PRESSURE_LEVELS.EUR_LL015.036.pl.nc
├── 20121017_12_ecmwf_forecast.ProbWCB_LAGRANTO_derived.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.ProbWCB_LAGRANTO_derived.EUR_LL015.036.sfc.nc
├── 20121017_12_ecmwf_forecast.PV_derived.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.PVU.EUR_LL015.036.pv.nc
├── 20121017_12_ecmwf_forecast.Q.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.SEA.EUR_LL015.036.sfc.nc
├── 20121017_12_ecmwf_forecast.SFC.EUR_LL015.036.sfc.nc
├── 20121017_12_ecmwf_forecast.T.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.THETA_LEVELS.EUR_LL015.036.tl.nc
├── 20121017_12_ecmwf_forecast.U.EUR_LL015.036.ml.nc
├── 20121017_12_ecmwf_forecast.V.EUR_LL015.036.ml.nc
└── 20121017_12_ecmwf_forecast.W.EUR_LL015.036.ml.nc
Before starting the standalone server you should add the path where the server config is to your python path. e.g.
$ export PYTHONPATH=~/mss
Detailed server configuration mswms_settings.py for this demodata
# -*- coding: utf-8 -*- """ mswms_settings ~~~~~~~~~~~~~~~~ Configuration module for programs accessing data on the MSS server. This file is part of mss. :copyright: 2008-2014 Deutsches Zentrum fuer Luft- und Raumfahrt e.V. :copyright: 2011-2014 Marc Rautenhaus :copyright: Copyright 2017 Jens-Uwe Grooss, Joern Ungermann, Reimar Bauer :copyright: Copyright 2017-2024 by the MSS team, see AUTHORS. :license: APACHE-2.0, see LICENSE for details. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. """ import os import sys # on a productions system you may want to limit the amount of tracebacks to 0 # sys.tracebacklimit = 0 # Configuration of Python's code search path # If you already have set up the PYTHONPATH environment variable for the # stuff you see below, you don't need to do a1) and a2). # a1) Path of the directory where the mss code package is located. # sys.path.insert(0, '/home/mss/INSTANCE/miniconda3/lib/python3.X/site-packages') # a2) Path of the directory where mswms_settings.py is located #MSSCONFIGPATH = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) #sys.path.insert(0, MSSCONFIGPATH) #os.chdir(MSSCONFIGPATH) import mslib.mswms.dataaccess from mslib.mswms import mpl_hsec_styles from mslib.mswms import mpl_vsec_styles from mslib.mswms import mpl_lsec_styles import mslib.mswms # Configuration for mswms_settings accessing data on the MSS server. # This is the data organisation structure of demodata. #service_name = "OGC:WMS" #service_title = "Mission Support System Web Map Service" #service_abstract = "Your Abstract" #service_contact_person = "Your Name" #service_contact_organisation = "Your Organization" #service_address_type = "postal" #service_address = "street" #service_city = "Your City" #service_state_or_province = "" #service_post_code = "12345" #service_country = "Germany" #service_fees = "none" #service_access_constraints = "This service is intended for research purposes only." # # HTTP Authentication # # If you require basic HTTP authentication, set the following variable # to True. Add usernames in the list "allowed:users". Note that the # passwords are not specified in plain text but by their md5 digest. #enable_basic_http_authentication = False #xml_template directory is a sub directory of mswms #base_dir = os.path.abspath(os.path.dirname(mslib.mswms.__file__)) #xml_template_location = os.path.join(base_dir, "xml_templates") # Path to store an optional gallery _gallerypath = r"/path/to/data/mss/gallery" _datapath = r"/path/to/data/mss/testdata" data = { "ecmwf_EUR_LL015": mslib.mswms.dataaccess.DefaultDataAccess(_datapath, "EUR_LL015"), } epsg_to_mpl_basemap_table = { # EPSG:4326, the standard cylindrical lat/lon projection. 4326: {"projection": "cyl"} } basemap_use_cache = True # # Registration of horizontal layers. # # The following list contains tuples of the format (instance of # visualisation classes, data set). The visualisation classes are # defined in mpl_hsec.py and mpl_hsec_styles.py. Add only instances of # visualisation products for which data files are available. The data # sets must be defined in mss_config.py. The WMS will only offer # products registered here. # # For defining new visualisation styles, we provide documentation and examples on # https://mss.rtfd.io/en/stable/mswms.html#mswms-wms-a-ogc-web-map-server # register_horizontal_layers = None if mpl_hsec_styles is not None: register_horizontal_layers = [ # ECMWF standard pressure level products. (mpl_hsec_styles.HS_TemperatureStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_GeopotentialWindStyle_PL, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_RelativeHumidityStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_EQPTStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_WStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_DivStyle_PL_01, ["ecmwf_EUR_LL015"]), ] # # Registration of vertical layers. # # The same as above, but for vertical cross-sections. register_vertical_layers = None if mpl_vsec_styles is not None: register_vertical_layers = [ # ECMWF standard vertical section styles. (mpl_vsec_styles.VS_CloudsStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_HorizontalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_VerticalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_RelativeHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_SpecificHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_TemperatureStyle_01, ["ecmwf_EUR_LL015"]) ] # # Registration of linear layers. # # The same as above, but for 1D sections. register_linear_layers = None if mpl_lsec_styles is not None: register_linear_layers = [ # ECMWF standard 1D section styles. (mpl_lsec_styles.LS_DefaultStyle, "air_temperature", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "divergence_of_wind", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "mole_fraction_of_ozone_in_air", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "air_potential_temperature", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "geopotential_height", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_humidity", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "cloud_area_fraction_in_atmosphere_layer", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_cloud_ice_water_content", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_cloud_liquid_water_content", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "ertel_potential_vorticity", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_RelativeHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_HorizontalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_VerticalVelocityStyle_01, ["ecmwf_EUR_LL015"]) ]
For setting authentication see mswms_auth.py
# -*- coding: utf-8 -*- """ mswms_auth ~~~~~~~~~~~~ Configuration module for authentication to the MSS server. This file is part of mss. :copyright: 2008-2014 Deutsches Zentrum fuer Luft- und Raumfahrt e.V. :copyright: 2011-2014 Marc Rautenhaus :copyright: Copyright 2016-2024 by the MSS team, see AUTHORS. :license: APACHE-2.0, see LICENSE for details. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. """ # # HTTP Authentication ### # # # Use the following code to create a new md5 digest of a password (e.g. in # ipython): # import hashlib; hashlib.md5("my_new_password").hexdigest() allowed_users = [("mswms", "add_md5_digest_of_PASSWORD_here"), ("add_new_user_here", "add_md5_digest_of_PASSWORD_here")]
Configuration file of the wms server
Configuration for the Mission Support System Web Map Service (wms).
In this module the data organisation structure of the available forecast data is described. The class NWPDataAccess is subclassed for each data type in the system and provides methods to determine which file needs to be accessed for a given variable and time. The classes also provide methods to query the available initialisation times for a given variable, and the available valid times for a variable and a given initialisation time. As the latter methods need to open the NetCDF data files to determine the contained time values, a caching system is used to avoid re-opening already searched files.
Replace the name INSTANCE in the following examples by your service name.
The configuration file have to become added to the /home/mss/INSTANCE/config directory
/home/mss/config/mswms_settings.py
# -*- coding: utf-8 -*- """ mswms_settings ~~~~~~~~~~~~~~~~ Configuration module for programs accessing data on the MSS server. This file is part of mss. :copyright: 2008-2014 Deutsches Zentrum fuer Luft- und Raumfahrt e.V. :copyright: 2011-2014 Marc Rautenhaus :copyright: Copyright 2016-2024 by the MSS team, see AUTHORS. :license: APACHE-2.0, see LICENSE for details. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. """ import os import sys # on a productions system you may want to limit the amount of tracebacks to 0 # sys.tracebacklimit = 0 # Configuration of Python's code search path # If you already have set up the PYTHONPATH environment variable for the # stuff you see below, you don't need to do a1) and a2). # a1) Path of the directory where the mss code package is located. # sys.path.insert(0, '/home/mss/miniconda3/envs/instance/lib/python3.X/site-packages') # a2) Path of the directory where mswms_settings.py is located MSSCONFIGPATH = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) sys.path.insert(0, MSSCONFIGPATH) os.chdir(MSSCONFIGPATH) import mslib.mswms.dataaccess from mslib.mswms import mpl_hsec_styles from mslib.mswms import mpl_vsec_styles from mslib.mswms import mpl_lsec_styles import mslib.mswms # # SETTINGS ### # # Path to store an optional gallery _gallerypath = r"/path/to/data/mss/gallery" # Paths to data directories. Process callables (the functions that are # executed by the dispatcher system) can access these paths, hence # simply define the key/value pairs that are required. datapath = { "ecmwf": "/path/to/data/mss/grid/ecmwf/netcdf", "emac": "/path/to/data/mss/grid/emac/netcdf", "meteosat": "/path/to/data/mss/grid/meteosat/netcdf", "camsglobal": "/path/to/data/mss/grid/camsglobal/netcdf", } # Objects that let the user query the filename in which a particular # variable can be found. Objects are instances of subclasses of NWPDataAccess, # which provides the methods fc_filename() and full_fc_path(). data = { "ecmwf_NH_LL05": mslib.mswms.dataaccess.DefaultDataAccess(datapath["ecmwf"], "NH_LL05"), # "ecmwf_EUR_LL015": mslib.mswms.dataaccess.DefaultDataAccess(datapath["ecmwf"], "EUR_LL015"), # "meteosat_EUR_LL05": mslib.mswms.dataaccess.DefaultDataAccess(datapath["meteosat"], "EUR_LL05"), # "emac_GLOBAL_LL1125": mslib.mswms.dataaccess.DefaultDataAccess(datapath["emac"]), # "CAMSglb": mslib.mswms.dataaccess.DefaultDataAccess(datapath["camsglobal"]), } # # HTTP Authentication ### # # If you require basic HTTP authentication, set the following variable # to True. Add usernames in the list "allowed:users". Note that the # passwords are not specified in plain text but by their md5 digest. enable_basic_http_authentication = False # # Basemap cache ### # # Plotting coastlines on horizontal cross-sections requires usually the parsing # of the corresponding databases for each plot. # A simple caching feature allows to reuse this data from previous plots using # the same bounding box and projection parameters, dramatically speeding up # the plotting. 'basemap_cache_size' determines hows many sets of coastlines shall # be stored in memory and 'basemap_request_size' determines the length of history # used to determine, which data shall be purged first if the cache exceeds its # maximum size. basemap_use_cache = False basemap_request_size = 200 basemap_cache_size = 20 # # Registration of horizontal layers. ### # # The following list contains tuples of the format (instance of # visualisation classes, data set). The visualisation classes are # defined in mpl_hsec.py and mpl_hsec_styles.py. Add only instances of # visualisation products for which data files are available. The data # sets must be defined in mss_config.py. The WMS will only offer # products registered here. register_horizontal_layers = [ # ECMWF standard surface level products. (mpl_hsec_styles.MPLBasemapHorizontalSectionStyle, ["ecmwf_NH_LL05"]), (mpl_hsec_styles.HS_CloudsStyle_01, ["ecmwf_EUR_LL015", "ecmwf_NH_LL05"]), (mpl_hsec_styles.HS_MSLPStyle_01, ["ecmwf_EUR_LL015", "ecmwf_NH_LL05"]), (mpl_hsec_styles.HS_SEAStyle_01, ["ecmwf_NH_LL05"]), (mpl_hsec_styles.HS_SeaIceStyle_01, ["ecmwf_NH_LL05"]), (mpl_hsec_styles.HS_VIProbWCB_Style_01, ["ecmwf_EUR_LL015"]), # ECMWF standard pressure level products. (mpl_hsec_styles.HS_TemperatureStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_GeopotentialWindStyle_PL, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_RelativeHumidityStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_EQPTStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_WStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_DivStyle_PL_01, ["ecmwf_EUR_LL015"]), # ECMWF standard model level products. (mpl_hsec_styles.HS_TemperatureStyle_ML_01, ["ecmwf_EUR_LL015"]), # ECMWF standard potential vorticity products. (mpl_hsec_styles.HS_PVTropoStyle_PV_01, ["ecmwf_EUR_LL015"]), # EMAC layers. # (mpl_hsec_styles.HS_EMAC_TracerStyle_ML_01, ["emac_GLOBAL_LL1125"]), # (mpl_hsec_styles.HS_EMAC_TracerStyle_SFC_01, ["emac_GLOBAL_LL1125"]), # Meteosat products. (mpl_hsec_styles.HS_Meteosat_BT108_01, ["meteosat_EUR_LL05"]), # MSS-Chem chemistry forecasts # # The MSS-Chem project (http://mss-chem.rtfd.io) provides an easy way to # download and prepare chemical weather forecasts from a range of different # CTMs. The plot style classes are called `HS_MSSChemStyle_XL_YYY_zzzzz` # and `VS_MSSChemStyle_XL_YYY_zzzzz` for horizontal and vertical sections, # i.e., "maps" and "cross-sections", respectively. # # In the class name, X determines the type of vertical layering of the model # data, and can be one of # - "P" for model data defined on pressure levels # - "M" for model data defined on model levels # - "A" for model data defined on altitude levels. # CAUTION: In the latter case, the vertical axis in vertical section plots # (VS_MSSChemStyle_AL_YYY_zzzz) is only approximate, as the air pressure # can only be estimated using the barometric formula. # # YYY stands for the chemical species to be displayed (in UPPER CASE), e.g., # "NO2". # # zzzzz determines the type of quantity, and can be one of # - mfrac -> mass fraction, kg/kg # - mconc -> mass concentration, kg/m3 # - nfrac -> mole fraction, mol/mol # - nconc -> mole_concentration, mol/m3 # # Putting this together, the following class provides a horizontal ("HS") # section of model-level ("ML") mass fractions ("mfrac") of nitrogen dioxide # ("NO2"): # (mpl_hsec_styles.HS_MSSChemStyle_ML_NO2_mfrac, ["CAMSglb"]), ] # # Registration of vertical layers. ### # # The same as above, but for vertical cross-sections. register_vertical_layers = [ # ECMWF standard vertical section styles. (mpl_vsec_styles.VS_CloudsStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_HorizontalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_PotentialVorticityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_ProbabilityOfWCBStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_VerticalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_RelativeHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_SpecificHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_TemperatureStyle_01, ["ecmwf_EUR_LL015"]), # EMAC layers. # (mpl_vsec_styles.VS_EMACEyja_Style_01, ["emac_GLOBAL_LL1125"]), # MSS-Chem chemistry forecasts # (mpl_vsec_styles.VS_MSSChemStyle_ML_NO2_mfrac, ["CAMSglb"]), ] # # Registration of linear layers. # # The same as above, but for 1D sections. register_linear_layers = None if mpl_lsec_styles is not None: register_linear_layers = [ # ECMWF standard 1D section styles. (mpl_lsec_styles.LS_DefaultStyle, "air_temperature", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "divergence_of_wind", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "mole_fraction_of_ozone_in_air", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "air_potential_temperature", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "geopotential_height", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_humidity", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "cloud_area_fraction_in_atmosphere_layer", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_cloud_ice_water_content", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_cloud_liquid_water_content", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "ertel_potential_vorticity", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_RelativeHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_HorizontalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_VerticalVelocityStyle_01, ["ecmwf_EUR_LL015"]) ] # # Server settings. ### # use_threadpool = False # xml_template directory is a sub directory of mswms base_dir = os.path.abspath(os.path.dirname(mslib.mswms.__file__)) xml_template_location = os.path.join(base_dir, "xml_templates") # get_capabilities.pt service_name = "OGC:WMS" service_title = "Mission Support System Web Map Service" service_abstract = "Your Abstract" service_contact_person = "Your Name" service_contact_organisation = "Your Organization" service_contact_position = "Your Position" service_address_type = "postal" service_address = "street" service_city = "Your City" service_state_or_province = "" service_post_code = "12345" service_country = "Germany" service_email = "mail@example.com" service_fees = "none" service_access_constraints = "This service is intended for research purposes only." # # EPSG Code Definitions for Matplotlib basemap ### # # In this section you can define how EPSG codes are interpreted in # terms of Matplotlib basemap parameters. If you require a new EPSG # code, define it here. # Table to translate EPSG codes to Matplotlib basemap projection parameters. # Extend this table to add further EPSG codes. # Also see: http://external.opengeospatial.org/twiki_public/bin/view/ # MetOceanDWG/MetCoordinateReferenceSystemDefinition epsg_to_mpl_basemap_table = { # EPSG:4326, the standard cylindrical lat/lon projection. 4326: {"projection": "cyl"}, # Non-standard EPSG codes, specifically defined for MSS pruposes. # EPSG:777llLLL, north polar stereographic projections with lat_0=ll and # lon_0=LLL. 77790000: {"projection": "stere", "lat_0": 90., "lon_0": 0.}, 77790010: {"projection": "stere", "lat_0": 90., "lon_0": 10.}, 77790015: {"projection": "stere", "lat_0": 90., "lon_0": 15.}, 77790340: {"projection": "stere", "lat_0": 90., "lon_0": -20.}, 77790105: {"projection": "stere", "lat_0": 90., "lon_0": -105.}, 77890000: {"projection": "spstere", "lat_0": 90., "lon_0": 0.}, 77890010: {"projection": "spstere", "lat_0": 90., "lon_0": 10.}, 77890015: {"projection": "spstere", "lat_0": 90., "lon_0": 15.}, 77890340: {"projection": "spstere", "lat_0": 90., "lon_0": -20.}, 77890105: {"projection": "spstere", "lat_0": 90., "lon_0": -105.} # Feel free to add other projections, e.g. a south polar projection # EPSG:778llLLL. }
You have to adopt this file to your data.
Adopt the mswms_settings.py for your needs
When you want to plot only one variable without any additional data available:
For horizontal plots:
mpl_hsec_styles.make_generic_class("HS_MyStyle_pl_air_temperature",'air_temperature','pl',[],[])
register_horizontal_layers = [
(mpl_hsec_styles.HS_MyStyle_pl_air_temperature, ["model"]),
]
For vertical plots:
mpl_vsec_styles.make_generic_class("VS_MyStyle_pl_air_temperature",'air_temperature','pl',[],[])
register_vertical_layers = [
(mpl_vsec_styles.VS_MyStyle_pl_air_temperature, ["model"]),
]
For linear plots:
register_linear_layers = [
(mpl_lsec_styles.LS_DefaultStyle, "air_temperature","pl", ["model"]),
]
Standalone server setup
MSWMS
This module can be used to run the wms server for development using Werkzeug’s development WSGI server. The development server is not intended for use in production. For production use a production-ready WSGI server such as Waitress, Gunicorn, Nginx, Apache2. See also https://flask.palletsprojects.com/en/latest/tutorial/deploy/?highlight=deploy#run-with-a-production-server
For the standalone server mswms you need the path of your mswms_settings.py and other configuration files added to the PYTHONPATH. E.g.:
export PYTHONPATH=/home/mss/INSTANCE/config
For testing your server you can use the demodata
The plots contained in MSS are mainly defined for meteorological forecast data. The intent is for the user to define their own plotting classes based on the the MSS infrastructure for data access. Some less tested plots are given as examples in the samples part of the documentation as templates. The next configuration exemplarily shows how to include user defined plots:
# -*- coding: utf-8 -*- import numpy as np import matplotlib.pyplot as plt import mpl_toolkits.axes_grid1.inset_locator import mpl_toolkits.basemap from matplotlib import patheffects from mslib.mswms.mpl_hsec import MPLBasemapHorizontalSectionStyle from mslib.mswms.mpl_vsec import AbstractVerticalSectionStyle from mslib.mswms.utils import get_style_parameters, get_cbar_label_format from mslib.utils import convert_to from mslib import thermolib MSSChemSpecies = { 'AERMR01': 'fine_sea_salt_aerosol', 'AERMR02': 'medium_sea_salt_aerosol', 'AERMR03': 'coarse_sea_salt_aerosol', 'AERMR04': 'fine_dust_aerosol', 'AERMR05': 'medium_dust_aerosol', 'AERMR06': 'coarse_dust_aerosol', 'AERMR07': 'hydrophobic_organic_matter_aerosol', 'AERMR08': 'hydrophilic_organic_matter_aerosol', 'AERMR09': 'hydrophobic_black_carbon_aerosol', 'AERMR10': 'hydrophilic_black_carbon_aerosol', 'AERMR11': 'sulfate_aerosol', 'C2H6': 'ethane', 'C3H8': 'propane', 'C5H8': 'isoprene', 'CH4': 'methane', 'CO': 'carbon_monoxide', 'HCHO': 'formaldehyde', 'HNO3': 'nitric_acid', 'NH3': 'ammonia', 'NMVOC': 'nmvoc_expressed_as_carbon', 'NO': 'nitrogen_monoxide', 'NO2': 'nitrogen_dioxide', 'O3': 'ozone', 'OH': 'hydroxyl_radical', 'PAN': 'peroxyacetyl_nitrate', 'PM2P5': 'pm2p5_ambient_aerosol', 'PM10': 'pm10_ambient_aerosol', 'SO2': 'sulfur_dioxide', } MSSChemQuantities = { 'mfrac': ('mass_fraction', 'kg kg-1', 1), 'mconc': ('mass_concentration', 'ug m-3', 1), 'nfrac': ('mole_fraction', 'mol mol-1', 1), 'nconc': ('mole_concentration', 'mol m-3', 1), } MSSChemTargets = { qtylong + '_of_' + species + '_in_air': (key, qty, units, scale) for key, species in MSSChemSpecies.items() for qty, (qtylong, units, scale) in MSSChemQuantities.items()} class HS_MSSChemStyle(MPLBasemapHorizontalSectionStyle): """ Pressure level version for Chemical Mixing ratios. """ styles = [ ("auto", "auto colour scale"), ("autolog", "auto logcolour scale"), ] # In order to use information from the DataAccess class to construct the titles, we override the set_driver to set # self.title. This cannot happen in __init__, as the WMSServer doesn't initialize the layers with the driver but # rather sets the driver only after initialization. def set_driver(self, driver): super(HS_MSSChemStyle, self).set_driver(driver=driver) self.title = self._title_tpl.format(modelname=self.driver.data_access._modelname) def _plot_style(self): bm = self.bm ax = self.bm.ax lonmesh_, latmesh_ = np.meshgrid(self.lons, self.lats) lonmesh, latmesh = bm(lonmesh_, latmesh_) show_data = np.ma.masked_invalid(self.data[self.dataname]) * self.unit_scale # get cmin, cmax, cbar_log and cbar_format for level_key cmin, cmax, clevs, cmap, norm, ticks = get_style_parameters( self.dataname, self.style, None, None, show_data) tc = bm.contourf(lonmesh, latmesh, show_data, levels=clevs, cmap=cmap, extend="both", norm=norm) for cont_data, cont_levels, cont_colour, cont_label_colour, cont_style, cont_lw, pe in self.contours: cs_pv = ax.contour(lonmesh, latmesh, self.data[cont_data], cont_levels, colors=cont_colour, linestyles=cont_style, linewidths=cont_lw) cs_pv_lab = ax.clabel(cs_pv, colors=cont_label_colour, fmt='%i') if pe: plt.setp(cs_pv.collections, path_effects=[patheffects.withStroke(linewidth=cont_lw + 2, foreground="w")]) plt.setp(cs_pv_lab, path_effects=[patheffects.withStroke(linewidth=1, foreground="w")]) # define position of the colorbar and the orientation of the ticks if self.crs.lower() == "epsg:77774020": cbar_location = 3 tick_pos = 'right' else: cbar_location = 4 tick_pos = 'left' # Format for colorbar labels cbar_label = self.title cbar_format = get_cbar_label_format(self.style, np.abs(clevs).max()) if not self.noframe: cbar = self.fig.colorbar(tc, fraction=0.05, pad=0.08, shrink=0.7, norm=norm, label=cbar_label, format=cbar_format, ticks=ticks) cbar.set_ticks(clevs) cbar.set_ticklabels(clevs) else: axins1 = mpl_toolkits.axes_grid1.inset_locator.inset_axes( ax, width="3%", height="40%", loc=cbar_location) self.fig.colorbar(tc, cax=axins1, orientation="vertical", format=cbar_format, ticks=ticks) # adjust colorbar fontsize to figure height fontsize = self.fig.bbox.height * 0.024 axins1.yaxis.set_ticks_position(tick_pos) for x in axins1.yaxis.majorTicks: x.label1.set_path_effects([patheffects.withStroke(linewidth=4, foreground='w')]) x.label1.set_fontsize(fontsize) def make_msschem_hs_class( entity, name, vert, units, scale, add_data=None, add_contours=None, fix_styles=None, add_styles=None, add_prepare=None): if add_data is None: add_data = [] _contourname = "" if add_contours is None: add_contours = [] elif add_contours[0][0] == "air_pressure": _contourname = "_pcontours" class fnord(HS_MSSChemStyle): name = f"HS_{entity}_{vert}{_contourname}" dataname = entity units = units unit_scale = scale _title_tpl = name + " (" + vert + ")" long_name = entity if units: _title_tpl += f"({units})" required_datafields = [(vert, entity, None)] + add_data contours = add_contours fnord.__name__ = name fnord.styles = list(fnord.styles) return fnord for vert in ["ml", "al", "pl"]: for stdname, props in MSSChemTargets.items(): name, qty, units, scale = props key = "HS_MSSChemStyle_" + vert.upper() + "_" + name + "_" + qty globals()[key] = make_msschem_hs_class(stdname, name, vert, units, scale) _pressurelevels = np.linspace(5000, 95000, 19) _npressurelevels = len(_pressurelevels) for vert in ["ml"]: for stdname, props in MSSChemTargets.items(): name, qty, units, scale = props key = "HS_MSSChemStyle_" + vert.upper() + "_" + name + "_" + qty + "_pcontours" globals()[key] = make_msschem_hs_class( stdname, name, vert, units, scale, add_data=[(vert, "air_pressure", None)], add_contours=[("air_pressure", _pressurelevels, ["dimgrey"] * _npressurelevels, ["dimgrey"] * _npressurelevels, ["dotted"] * _npressurelevels, 1, True)],) class VS_MSSChemStyle(AbstractVerticalSectionStyle): """ CTM tracer vertical tracer cross sections via MSS-Chem """ styles = [ ("auto", "auto colour scale"), ("autolog", "auto log colour scale"), ] # Variables with the highest number of dimensions first (otherwise # MFDatasetCommonDims will throw an exception)! required_datafields = [("ml", "air_pressure", "Pa")] # In order to use information from the DataAccess class to construct the titles, we override the set_driver to set # self.title. This cannot happen in __init__, as the WMSServer doesn't initialize the layers with the driver but # rather sets the driver only after initialization. def set_driver(self, driver): super(VS_MSSChemStyle, self).set_driver(driver=driver) # for altitude level model data, when we don't have air_pressure information, we want to warn users that the # vertical section is only an approximation vert = self.name[-2:] if vert != "pl": # look for valid times including air_pressure init_times = self.driver.data_access.get_init_times() valid_times = self.driver.data_access.get_valid_times( "air_pressure", vert, init_times[0]) if len(valid_times) == 0: self.title = self.title.replace( "(" + vert + ")", "(" + vert + "; WARNING: vert. distribution only approximate!)") def _prepare_datafields(self): """Computes potential temperature from pressure and temperature if it has not been passed as a data field. """ if self.name[-2:] == "al": # CAMS Regional Ensemble doesn't provide any pressure information, # but we want to plot vertical sections anyways, so we do a # poor-man's on-the-fly conversion here. if 'air_pressure' not in self.data: self.data["air_pressure"] = np.empty_like(self.data[self.dataname]) self.data['air_pressure'][:] = thermolib.flightlevel2pressure_a(convert_to( self.driver.vert_data[::-self.driver.vert_order, np.newaxis], self.driver.vert_units, "hft")) def _plot_style(self): ax = self.ax curtain_cc = self.data[self.dataname] * self.unit_scale curtain_cc = np.ma.masked_invalid(curtain_cc) curtain_p = self.data["air_pressure"] numlevel = curtain_p.shape[0] numpoints = len(self.lats) curtain_lat = self.lat_inds.repeat(numlevel).reshape((numpoints, numlevel)).transpose() # Filled contour plot of cloud cover. # INFO on COLORMAPS: # http://matplotlib.sourceforge.net/examples/pylab_examples/show_colormaps.html if self.p_bot > self.p_top: visible = (curtain_p <= self.p_bot) & (curtain_p >= self.p_top) else: visible = (curtain_p >= self.p_bot) & (curtain_p <= self.p_top) if visible.sum() == 0: visible = np.ones_like(curtain_cc, dtype=bool) cmin, cmax, clevs, cmap, norm, ticks = get_style_parameters( self.dataname, self.style, None, None, curtain_cc[visible]) cs = ax.contourf(curtain_lat, curtain_p, curtain_cc, clevs, cmap=cmap, extend="both", norm=norm) # Contour lines for cont_data, cont_levels, cont_colour, cont_label_colour, cont_style, cont_lw, pe in self.contours: if cont_levels is None: pl_cont = ax.plot(self.lat_inds, self.data[cont_data].reshape(-1), "o", color="k", zorder=100) plt.setp(pl_cont, path_effects=[patheffects.withStroke(linewidth=4, foreground="w")]) else: cs_pv = ax.contour(curtain_lat, curtain_p, self.data[cont_data], cont_levels, colors=cont_colour, linestyles=cont_style, linewidths=cont_lw) plt.setp(cs_pv.collections, path_effects=[patheffects.withStroke(linewidth=cont_lw + 2, foreground="w")]) cs_pv_lab = ax.clabel(cs_pv, colors=cont_label_colour, fontsize=8, fmt='%i') plt.setp(cs_pv_lab, path_effects=[patheffects.withStroke(linewidth=1, foreground="w")]) # Pressure decreases with index, i.e. orography is stored at the # zero-p-index (data field is flipped in mss_plot_driver.py if # pressure increases with index). self._latlon_logp_setup(titlestring=self.title) # Format for colorbar labels cbar_format = get_cbar_label_format(self.style, np.median(np.abs(clevs))) cbar_label = self.title # Add colorbar. if not self.noframe: self.fig.subplots_adjust(left=0.08, right=0.95, top=0.9, bottom=0.14) self.fig.colorbar(cs, fraction=0.05, pad=0.01, format=cbar_format, label=cbar_label, ticks=ticks) else: axins1 = mpl_toolkits.axes_grid1.inset_locator.inset_axes( ax, width="1%", height="40%", loc=1) self.fig.colorbar(cs, cax=axins1, orientation="vertical", format=cbar_format, ticks=ticks) # adjust colorbar fontsize to figure height fontsize = self.fig.bbox.height * 0.024 axins1.yaxis.set_ticks_position("left") for x in axins1.yaxis.majorTicks: x.label1.set_path_effects([patheffects.withStroke(linewidth=4, foreground='w')]) x.label1.set_fontsize(fontsize) def make_msschem_vs_class( entity, name, vert, units, scale, add_data=None, add_contours=None, fix_styles=None, add_styles=None, add_prepare=None): # This is CTM output, so we cannot expect any additional meteorological # parameters except for air_pressure if add_data is None: if vert == "al": # "al" is altitude layer, i.e., for CTM output with no pressure information # at all (e.g., CAMS reg. Ensemble) # In those cases we derive air_pressure from the altitude alone, in the _prepare_datafields() method add_data = [] elif vert == 'pl': # "pl" are pressure levels. Here, the air_pressure information is implicitly contained in the vertical # dimension coordinate, so we don't need to explicitly load it here. add_data = [] else: # all other layer types need to read air_pressure from the data add_data = [(vert, "air_pressure", "Pa")] if add_contours is None: add_contours = [] class fnord(VS_MSSChemStyle): name = f"VS_{entity}_{vert} " dataname = entity units = units unit_scale = scale title = name + " (" + vert + ")" long_name = entity if units: title += f"({units})" required_datafields = [(vert, entity, None)] + add_data contours = add_contours if add_contours else [] fnord.__name__ = name fnord.styles = list(fnord.styles) if add_styles is not None: fnord.styles += add_styles if fix_styles is not None: fnord.styles = fix_styles if add_prepare is not None: fnord._prepare_datafields = add_prepare return fnord for vert in ["ml", "tl", "pl", "al"]: for stdname, props in MSSChemTargets.items(): name, qty, units, scale = props key = "VS_MSSChemStyle_" + vert.upper() + "_" + name + "_" + qty globals()[key] = make_msschem_vs_class(stdname, name, vert, units, scale)# -*- coding: utf-8 -*- """ mswms_settings ~~~~~~~~~~~~~~~~ Configuration module for programs accessing data on the MSS server. This file is part of mss. :copyright: 2008-2014 Deutsches Zentrum fuer Luft- und Raumfahrt e.V. :copyright: 2011-2014 Marc Rautenhaus :copyright: Copyright 2017 Jens-Uwe Grooss, Joern Ungermann, Reimar Bauer :copyright: Copyright 2017-2024 by the MSS team, see AUTHORS. :license: APACHE-2.0, see LICENSE for details. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. """ import os import sys # on a productions system you may want to limit the amount of tracebacks to 0 # sys.tracebacklimit = 0 # Configuration of Python's code search path # If you already have set up the PYTHONPATH environment variable for the # stuff you see below, you don't need to do a1) and a2). # a1) Path of the directory where the mss code package is located. # sys.path.insert(0, '/home/mss/INSTANCE/miniconda3/lib/python3.X/site-packages') # a2) Path of the directory where mswms_settings.py is located #MSSCONFIGPATH = os.path.abspath(os.path.normpath(os.path.dirname(sys.argv[0]))) #sys.path.insert(0, MSSCONFIGPATH) #os.chdir(MSSCONFIGPATH) import mslib.mswms.dataaccess from mslib.mswms import mpl_hsec_styles from mslib.mswms import mpl_vsec_styles from mslib.mswms import mpl_lsec_styles import mslib.mswms import mss_chem_plots # Configuration for mswms_settings accessing data on the MSS server. # This is the data organisation structure of demodata. #service_name = "OGC:WMS" #service_title = "Mission Support System Web Map Service" #service_abstract = "Your Abstract" #service_contact_person = "Your Name" #service_contact_organisation = "Your Organization" #service_address_type = "postal" #service_address = "street" #service_city = "Your City" #service_state_or_province = "" #service_post_code = "12345" #service_country = "Germany" #service_fees = "none" #service_access_constraints = "This service is intended for research purposes only." # # HTTP Authentication # # If you require basic HTTP authentication, set the following variable # to True. Add usernames in the list "allowed:users". Note that the # passwords are not specified in plain text but by their md5 digest. #enable_basic_http_authentication = False #xml_template directory is a sub directory of mswms #base_dir = os.path.abspath(os.path.dirname(mslib.mswms.__file__)) #xml_template_location = os.path.join(base_dir, "xml_templates") # Path to store an optional gallery _gallerypath = r"/path/to/data/mss/gallery" _datapath = r"/path/to/data/mss/testdata" data = { "ecmwf_EUR_LL015": mslib.mswms.dataaccess.DefaultDataAccess(_datapath, "EUR_LL015"), } epsg_to_mpl_basemap_table = { # EPSG:4326, the standard cylindrical lat/lon projection. 4326: {"projection": "cyl"} } basemap_use_cache = True # # Registration of horizontal layers. # # The following list contains tuples of the format (instance of # visualisation classes, data set). The visualisation classes are # defined in mpl_hsec.py and mpl_hsec_styles.py. Add only instances of # visualisation products for which data files are available. The data # sets must be defined in mss_config.py. The WMS will only offer # products registered here. register_horizontal_layers = None if mpl_hsec_styles is not None: register_horizontal_layers = [ # ECMWF standard pressure level products. (mpl_hsec_styles.HS_TemperatureStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_GeopotentialWindStyle_PL, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_RelativeHumidityStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_EQPTStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_WStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_DivStyle_PL_01, ["ecmwf_EUR_LL015"]), (mpl_hsec_styles.HS_DivStyle_PL_01, ["ecmwf_EUR_LL015"]), (mss_chem_plots.HS_MSSChemStyle_PL_NH3_nfrac, ["ecmwf_EUR_LL015"]), ] # # Registration of vertical layers. # # The same as above, but for vertical cross-sections. register_vertical_layers = None if mpl_vsec_styles is not None: register_vertical_layers = [ # ECMWF standard vertical section styles. (mpl_vsec_styles.VS_CloudsStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_HorizontalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_VerticalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_RelativeHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_SpecificHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_vsec_styles.VS_TemperatureStyle_01, ["ecmwf_EUR_LL015"]) (mss_chem_plots.VS_MSSChemStyle_PL_NH3_nfrac, ["ecmwf_EUR_LL015"]), ] # # Registration of linear layers. # # The same as above, but for 1D sections. register_linear_layers = None if mpl_lsec_styles is not None: register_linear_layers = [ # ECMWF standard 1D section styles. (mpl_lsec_styles.LS_DefaultStyle, "air_temperature", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "divergence_of_wind", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "mole_fraction_of_ozone_in_air", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "air_potential_temperature", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "geopotential_height", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_humidity", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "cloud_area_fraction_in_atmosphere_layer", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_cloud_ice_water_content", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "specific_cloud_liquid_water_content", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_DefaultStyle, "ertel_potential_vorticity", ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_RelativeHumdityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_HorizontalVelocityStyle_01, ["ecmwf_EUR_LL015"]), (mpl_lsec_styles.LS_VerticalVelocityStyle_01, ["ecmwf_EUR_LL015"]) ]
Gallery extension
The gallery builder enables to generate static plots given from data and visualisation styles on server site. An example can be seen on the documentation based on our demodata https://mss.readthedocs.io/en/stable/gallery/index.html
When you use this feature you get a menu entry below the “Mission Support System” Main menu on your server site.
To create all layers of all plots use
mswms gallery --create
With an option –levels you can specify by a comma-separated list of all levels visible on the gallery. Further options are –itimes, –vtimes. If you want to publish on which code the images are based on you can do this by the option –show-code e.g.
mswms gallery --create --show-code --itimes 2012-10-17T12:00:00 --vtimes 2012-10-19T12:00:00 --levels 200,300
For the case you use an url-prefix on your site you have to add this by the –url-prefix parameter too.
WMS Server Deployment
Once installation and configuration are complete, you can start the Web Map Service application (provided you have forecast data to visualise). The file “mswms” is an executable Python script starting up a Flask HTTP server with the WMS WSGI module. A short description of how to start the program is given by the –help option. The file “wms.wsgi” is intended to be used with an Apache web server installation.
We have a single method to use data for ECMWF, CLaMS, GWFC, EMAC, METEOSAT implemented. The data have to use for their parameters the CF attribute standard_name. A new method should be able to deal with any CF conforming file following a couple of simple additional requirements.
Per configuration you could register horizontal (register_horizontal_layers) or vertical layers (register_vertical_layers), give a basemap table for EPSG mapping (epsg_to_mpl_basemap_table) and at all how to access the data.
A few notes:
The Flask WMS currently cannot run multithreaded (Apache does support multiple processes). This is due to that a single instance of the WSGI application handler class MSS_WMSResponse can create only one plot at a time (otherwise you get messed up plots when simultaneous requests occur). In the current implementation, only a single instance is passed to Flask (to do all the initialisation work only once). To extend the software to handle simultaneous requests would probably involve creating a “factory” of MSS_WMSResponse instances.. If you want to do this, check if/how Flask handles “worker” factories.
Creating the capabilities document can take very long (> 1 min) if the forecast data files have to be read for the first time (the WMS program opens all files and tries to determine the available data and elevation ranges). A GetCapabilities request should return a document within a few seconds as long as all files are in the disk cache. The “CachedDataAccess” class offers an in-memory cache to prevent costly file-accesses beyond the first.
A typical bottleneck for plot generation is when the forecast data files are located on a different computer than the WMS server. In this case, large amounts of data have to be transferred over the network. Hence, when possible, try to make sure the WMS runs on the same computer on which the input data files are hosted.
Meteorological data
Data for the MSS server shall be provided in CF-compliant NetCDF format. Several specific data access methods are provided for ECMWF, Meteoc, and several other formats.
The preferred method “DefaultDataAccess” shall supplant most of these, but requires the data to be organised in the fashion described in the following (the others pose mostly the same requirements).
All data files belonging to one “set” shall have a common string in its name that can be used to uniquely identify all files of this set. Each set must share the same time, longitude, and latitude grid. Each set must use the same elevation layers for each type of vertical axis. Different data sets may be used to offer different geographical regions or results of different simulation models.
Each file of a set must contain only one or no vertical axis. If the data is required to be given on multiple vertical axis (such as providing data for horizontal plots on both pressure and theta levels), one (or more separate) file for each vertical axis type must be provided. All files for one axis type shall provide the same levels. If no vertical axis can be identified, it is assumed that the file contains 3-D data (time, lat, lon) such as, e.g., surface pressure or tropopause altitude.
The vertical coordinate variable is identified by the standard_name being one of the following names:
atmosphere_hybrid_sigma_pressure_coordinate - “ml”
atmosphere_pressure_coordinate - “pl”
atmosphere_ertel_potential_vorticity_coordinate - “pv”
atmosphere_altitude_coordinate - “al”
atmosphere_potential_temperature_coordinate - “tl”
flight_level_coordinate -“fl”
The two-letter abbreviation is used for brief identification in the plotting routines in addition to the standard_name of the variable to uniquely identify which data shall be used. The data shall be organized with the dimensions in the order of “time”, “vertical coordinate”, “latitudes”, and “longitudes” (This is important to reduce disk access when generating the plots). Data variables are identified by their standard_name, which is expected to be CF compliant. Data variables must contain a “units” attribute that is by the plotting routines for checking and conversion. The “pint” package is used for parsing the units. Some additional units such as PVU have been added to the package, but failure to parse the unit will cause the server to disregard the variable. Exemplary valid units are ‘dimensionless’, ‘hPa’, ‘m**2’, ‘m.s^-1’, ‘millibar’, ‘knots’, ‘percent’, or ‘ppmv’. Please bear in mind that the vertical axis of all vertical sections is pressure in ‘Pa’.
It is assumed that forecast data is given from one initialisation time onward for several time steps into the future. For each file, the init time is determined by the units attribute of the “time” variable. The time variable is identified by its standard_name being “time”. The date given after “since” is interpreted as the init time such that the numerical value of “0” were the init time (which need not be present in the file). For example, if the units field of “time” contains “hours since 2012-10-17T12:00:00.000Z”, 2012-10-17T12Z would be the init time. Data for different time steps may be contained in one file or split over several ones.
In case a file contains additional dimensions beyond the four required ones, MSS might discard the file, if they are inconsistently used among files or are missing coordinate variables, etc., even though they would not affect the operation of MSS. One may skip checks on these dimensions in the data access class by specifying a list of said dimensions in the “skip_dim_check” constructor parameter.
An exemplary header for a file containing ozone on a vertical pressure coordinate and a 3-D tropopause would look as follows:
netcdf example_ASIA {
dimensions:
press = 13 ;
lat = 51 ;
lon = 141 ;
time = 12 ;
variables:
float press(press) ;
press:units = "Pa" ;
press:positive = "down" ;
press:standard_name = "atmosphere_pressure_coordinate" ;
float lat(lat) ;
lat:units = "degrees_north" ;
lat:standard_name = "latitude" ;
float lon(lon) ;
lon:units = "degrees_east" ;
lon:standard_name = "longitude" ;
float time(time) ;
time:units = "hours since 2012-10-17T12:00:00Z" ;
time:standard_name = "time" ;
float O3(time, press, lat, lon) ;
O3:units = "mol/mol" ;
O3:standard_name = "mole_fraction_of_ozone_in_air" ;
float tropopause(time, lat, lon) ;
tropopause:units = "Pa" ;
tropopause:standard_name = "tropopause_air_pressure" ;
}
Additional plotting layers
The plotting of data is organised via classes following the abstract base class Abstract2DSectionStyle in mslib.mswms.mss_2D_sections. One can define a new class derived from this (respectively the VS_GenericStyle and HS_GenericStyle classes for vertical and horizontal cross-sections) and add them to the configuration as shown in the example mswms server configuration files.
Generic plotting layers
Often a simple plot is sufficient. To facilitate the addition of simple plots, a generic plotting layer class has been defined. The mslib.mswms.generics module offers a ‘register_standard_name’ function that will register a data product with given CF standard_name (including also units to be used and further configuration options). In case that the generics module is imported and a style is registered before the remainder of mslib is imported, plotting classes are automatically generated. This is demonstrated in the following excerpt from a mswms settings file:
# import generics module *first* and register all desired standard_names
import mslib.mswms.generics as generics
generics.register_standard_name(
"mole_fraction_of_CH3Br",
"pmol/mol"
)
# ...
# now import the styles modules and populate the necessary configuration lists
# (see also above for information about the mswms server settings file)
import mslib.mswms.mpl_hsec_styles
register_horizontal_layers = [
(mslib.mswms.mpl_hsec_styles.HS_GenericStyle_PL_mole_fraction_of_CH3Br, ["mydata"])]
This would register the new standard_name ‘mole_fraction_of_CH3Br’, which cause an associated generic plotting class to be instantiated, which can be used later on in the configuration file. Such classes are generated for all registered standard_names, many of which are already preconfigured. The naming scheme for the new classes are (HS|VS)_GenericStyle_(PL|AL|TL|ML)_<standard_name>, whereby HS/VS denotes horizontal and vertical cross-section, respectively, PL/AL/TL/ML specifies the vertical coordinate of the plot (for which the corresponding data must be available), and the last part of the class name is the CF standard_name itself. In the example above, the registering of the “mole_fraction_of_CH3Br” standard_name would cause a series of classes to be generated, amongst them the used “mslib.mswms.mpl_hsec_styles.HS_GenericStyle_PL_mole_fraction_of_CH3Br”, which offers a horizontal cross-section plot of mole_fraction_of_CH3Br on pressure levels.
In case these simple plots are insufficient, the make_generic_class functions from the mslib.mswms.mss_hsec_styles and mslib.mswms.mss_vsec_styles modules used to generate the generic plots offers additional options for further configuration to simply add, e.g., user defined contours of other variables on top or use user defined plotting styles to, e.g., change color maps. More information about the features can be found in the docstrings of these functions.
Custom plotting layers
If the generic plotting layers are not sufficient, a dedicated class can be defined, which allows the use of all matplotlib features. These classes must be derived from the appropriate abstract base classes and implement the relevant methods.
Here is a simple example for horizontal cross-sections:
import numpy as np import matplotlib import mslib.mswms.mpl_hsec_styles import mpl_toolkits.axes_grid1.inset_locator class HS_BrightnessTemperature(mslib.mswms.mpl_hsec_styles.MPLBasemapHorizontalSectionStyle): name = "BT01" entity = "brightness_temperature_residual" dataname = entity title = "brightness temperature residual (K)" long_name = entity abstract = "Brightness temperature residual. Altitude range: " \ "4µm 34-42km, 15µm-high 42-44km, 15µm-low 24-26km" required_datafields = [ ("sfc", "brightness_temperature_residual_4mu", "K"), ("sfc", "brightness_temperature_residual_15mu_high", "K"), ("sfc", "brightness_temperature_residual_15mu_low", "K"), ] styles = [ ("4mu", "4µm"), ("15mu_low", "15µm low"), ("15mu_high", "15µm high"), ] def _plot_style(self): lonmesh_, latmesh_ = np.meshgrid(self.lons, self.lats) lonmesh, latmesh = self.bm(lonmesh_, latmesh_) clev = np.asarray([-7, -4, -2, -1, -0.5, 0.5, 1.0, 2, 4.0, 7.0]) ticks = [-4, -2, -1, -0.5, 0.5, 1, 2, 4] cmap = matplotlib.pyplot.cm.Spectral_r norm = matplotlib.colors.BoundaryNorm(clev, cmap.N) pv = self.bm.pcolormesh( lonmesh, latmesh, self.data[f"brightness_temperature_residual_{self.style}"], cmap=cmap, norm=norm) if not self.noframe: cbar = self.fig.colorbar(pv, fraction=0.05, pad=0.08, shrink=0.7) cbar.set_label("BT") else: axins1 = mpl_toolkits.axes_grid1.inset_locator.inset_axes( self.ax, width="3%", height="40%", loc=4) cbar = self.fig.colorbar(pv, cax=axins1, orientation="vertical", ticks=ticks, extend="both") axins1.yaxis.set_ticks_position("left")
This plotting layer offers several 2-D data products, which can be selected using the style. More examples can be found within the source code of the mswms component.
A taste of WSGI
(MS)WMS is a WSGI application based on Flask. You need a WSGI server to run the application, this converts incoming HTTP requests to the WSGI environ, and outgoing WSGI responses to HTTP responses.
For self hosting have a look on these platforms.
We describe two examples, Waitress for a pure Python Server and Apache2 using mod_wsgi. On long running systems you may want to use Apache2 and have a lot features included in the package. With a nginx proxy also a waitress server can use certificates and supervisord can be used to monitor and control the waitress process.
Waitress
Waitress is a production-quality pure-Python WSGI server.
Installing
It is easy to configure and runs on CPython on Unix and Windows.
mamba install waitress
wms.wsgi
A file /home/mss/INSTANCE/wsgi/wsgi_setup.py with the content
import sys
sys.path.insert(0, '/home/mss/INSTANCE/config/where_mswms_settings.py_is/')
import logging
logging.basicConfig(stream=sys.stderr)
from mslib.mswms.wms import app
Running the waitress server
This runs the wms server on port 5000. If you use a certificate and proxy by e.g. nginx use –url-scheme=https
PYTHONPATH=~/INSTANCE/wsgi/ waitress-serve --host 127.0.0.1 --port 5000 --url-scheme=http wsgi_setup:app
Further documentations:
Apache server setup
Install mod_wsgi
On some distributions an old mod_wsgi is shipped and have to become replaced by a version compatible to the conda environment. This procedure may need the package apache2-dev on your server.
At current state we have to use pip to install mod_wsgi into the INSTANCE environment:
# Install `mod_wsgi`
$ pip install mod_wsgi
# Find the full path to installed `mod_wsgi`
$ which mod_wsgi-express
# Install and register the `mod_wsgi` module with Apache
$ sudo /full/path/to/installed/mod_wsgi-express install-module
This command outputs two lines:
LoadModule wsgi_module "/usr/lib/apache2/modules/mod_wsgi-py310.cpython-310-x86_64-linux-gnu.so"
WSGIPythonHome "/home/mss-demo/miniforge/envs/mssenv"
You have to add the lines into your wsgi_express.conf and wsgi_express.load
Setup a /etc/apache2/mods-available/wsgi_express.conf:
WSGIPythonHome "/home/mss-demo/miniforge/envs/mssenv/"
Setup a /etc/apache2/mods-available/wsgi_express.load:
LoadModule wsgi_module "/usr/lib/apache2/modules/mod_wsgi-py310.cpython-310-x86_64-linux-gnu.so"
Enable the new module by a2enmod and reload the apache2 server
Configuration of apache mod_wsgi.conf
One possibility to setup the PYTHONPATH environment variable is by adding it to your mod_wsgi.conf. Alternatively you could add it also to wms.wsgi.
WSGIPythonPath /home/mss/INSTANCE/config:/home/mss/miniforge/envs/mssenv/lib/python3.X/site-packages
By this setting you override the PYTHONPATH environment variable. So you have also to add the site-packes directory of your miniforge installation besides the config file path.
If your server hosts different instances by different users you want to setup this path in mswms_setting.py.
One Instance
Our examples are based on the following directories located in the home directory of the mss user. INSTANCE is a placeholder for your service name:
.
├── INSTANCE
│ ├── config
│ │ └── mswms_settings.py
│ │ └── mswms_auth.py
│ ├── log
│ │ └── mss_error.log
│ └── wsgi
│ ├── auth.wsgi
│ └── wms.wsgi
├── miniforge
│ ├── bin
│ ├── cmake
│ ├── compiler_compat
│ ├── _conda
│ ├── condabin
│ ├── conda-bld
│ ├── conda_build_config.yaml
│ ├── conda-meta
│ ├── envs
│ │ └── mssenv
│ ├── etc
│ ├── include
│ ├── lib
│ ├── libexec
│ ├── LICENSE.txt
│ ├── locks
│ ├── man
│ ├── pkgs
│ ├── sbin
│ ├── share
│ ├── shell
│ ├── ssl
│ ├── x86_64-conda_cos6-linux-gnu
│ └── x86_64-conda-linux-gnu
Configuration of wsgi for wms
You can setup a vhost for this service.
/home/mss/INSTANCE/wsgi/wms.wsgi
import os import sys sys.path.insert(0, '/home/mss/INSTANCE/config') sys.path.insert(0, '/home/mss/miniconda3/envs/instance/lib/python3.7/site-packages/mslib') sys.path.insert(0, '/home/mss/INSTANCE/wsgi') if os.getenv("PROJ_LIB") is None or os.getenv("PROJ_LIB") == "PROJ_LIB": os.environ["PROJ_LIB"] = '/home/mss/miniconda3/envs/instance/share/proj' sys.stdout = sys.stderr import logging logging.basicConfig(stream=sys.stderr) from mslib.mswms.wms import app as application
Configuration of wsgi auth
As long as you have only one instance of the server running you can use this method to restrict access.
To restrict access to your data use this script.
/home/mss/INSTANCE/wsgi/auth.wsgi
import sys sys.path.extend(['/home/mss/INSTANCE/config']) import mswms_auth import hashlib def check_password(environ, username, password): for u, p in mswms_auth.allowed_users: if (u == username) and (p == hashlib.md5(password).hexdigest()): return True return False
This needs also a configuration /home/mss/INSTANCE/config/mswms_auth.py script.
# -*- coding: utf-8 -*- """ mswms_auth ~~~~~~~~~~~~ Configuration module for authentication to the MSS server. This file is part of mss. :copyright: 2008-2014 Deutsches Zentrum fuer Luft- und Raumfahrt e.V. :copyright: 2011-2014 Marc Rautenhaus :copyright: Copyright 2016-2024 by the MSS team, see AUTHORS. :license: APACHE-2.0, see LICENSE for details. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. """ # # HTTP Authentication ### # # # Use the following code to create a new md5 digest of a password (e.g. in # ipython): # import hashlib; hashlib.md5("my_new_password").hexdigest() allowed_users = [("mswms", "add_md5_digest_of_PASSWORD_here"), ("add_new_user_here", "add_md5_digest_of_PASSWORD_here")]
At the moment you have many different instances with different users or different versions of mss you have to use basic auth of your webserver configuration.
Configuration of your site as vhost
You have to setup a webserver server site configuration file
/etc/apache2/sites-available/mss.yourserver.de.conf
<VirtualHost *:80> ServerName mss.yourserver.de #ServerAlias localwms LogLevel debug ServerAdmin webmaster@yourserver.de ErrorLog /home/mss/log/mss_error.log TransferLog /var/log/apache2/mss_access.log CustomLog /var/log/apache2/mss_custom.log combined #all wsgi scripts are in /home/mss/wsgi <Directory /home/mss/INSTANCE/wsgi> AuthType Basic AuthName "mss" AuthBasicProvider wsgi WSGIAuthUserScript /home/mss/INSTANCE/wsgi/auth.wsgi AuthDigestDomain / http://mss.yourserver.de/ Require valid-user </Directory> #alternative without authentication #<Directory /home/mss/INSTANCE/wsgi> # Require all granted #</Directory> WSGIScriptAlias / /home/mss/INSTANCE/wsgi/wms.wsgi WSGIDaemonProcess MSS python-home=/home/mss/miniconda3/envs/instance user=mss group=mss home=/home/mss/INSTANCE/config processes=2 threads=1 deadlock-timeout=25 display-name=MSS # WSGI Options # python-home: path where your environment python bin is located # home: where your config scripts are located # Please see the documentation here : http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives WSGIProcessGroup MSS # Python Simplified GIL State API ( ISSUE ) # http://code.google.com/p/modwsgi/wiki/ApplicationIssues WSGIApplicationGroup %{GLOBAL} DocumentRoot /home/mss/htdocs # don't loose time with IP address lookups HostnameLookups Off # needed for named virtual hosts UseCanonicalName Off # configures the footer on server-generated documents ServerSignature On </VirtualHost>
Enable it with a2ensite mss.yourserver.de.conf
Many Instances
If you want to setup many instances we suggest to use a similar proxy based configuration
<VirtualHost *:80> ProxyRequests Off ProxyPreserveHost On RewriteEngine On RequestHeader add X-SSL off RewriteRule ^/demo/(.*) http://127.0.0.1/demo/$1 [P,L] ServerName proxy_demo.yourserver.de DocumentRoot /var/www/html ProxyPreserveHost On ProxyPass /demo http://127.0.0.1/demo </VirtualHost>
and if you need authentication then use a Location based AuthType Basic
<VirtualHost 127.0.0.1:80> #<Directory /home/mss/DEMO/wsgi> # Require all granted #</Directory> <Location /demo> AuthType Basic AuthName "mss" AuthDigestDomain / AuthUserFile /home/mss/DEMO/config/apache_users <Limit GET> Require valid-user </Limit> </Location> # You can define different hostname for the WMS VirtualHost here # For example, you can add localwms to your /etc/hosts file along with localhost # and use the 'localwms' name like in the template here. ServerName proxy_demo.yourserver.de RemoteIPHeader X-Forwarded-For RemoteIPInternalProxy 127.0.0.0/8 LogLevel error ServerAdmin admin@email ErrorLog /home/mss/DEMO/log/mss_error.log TransferLog /var/log/apache2/mss_access.log CustomLog /var/log/apache2/mss_custom.log combined # WSGI Options # home: Initial working directory of the script, make sure you change it. # python-path : Directories to search for Modules, make sure you change it as well. # Please see the documentation here : http://code.google.com/p/modwsgi/wiki/ConfigurationDirectives WSGIScriptAlias /demo /home/mss/DEMO/wsgi/wms.wsgi WSGIDaemonProcess MSSDEMO python-home="/home/mss/miniconda3/envs/demo" python-path="/home/mss/miniconda3/envs/demo/lib/python3.7/site-packages/" home="/home/mss/DEMO/config" user=mss group=mss processes=2 threads=1 deadlock-timeout=25 display-name=MSSDEMO WSGIProcessGroup MSSDEMO # Python Simplified GIL State API ( ISSUE ) # http://code.google.com/p/modwsgi/wiki/ApplicationIssues WSGIApplicationGroup %{GLOBAL} # don't loose time with IP address lookups HostnameLookups Off # needed for named virtual hosts UseCanonicalName Off # configures the footer on server-generated documents ServerSignature On </VirtualHost>
For further information on apache2 server setup read https://httpd.apache.org/docs/2.4/howto/