MESSAGEix-Nexus (model.water)

message_ix_models.model.water adds water usage and demand related representation to the MESSAGEix-GLOBIOM global model. The resulting model is referred to as “MESSAGEix-Nexus”. This work extends the water sector linkage described by Parkinson et al. (2019) [72].

CLI usage

Use the CLI command mix-data water to invoke the commands defined in water.cli. Example: mix-models --url=ixmp://ixmp_dev/ENGAGE_SSP2_v4.1.7/baseline_clone_test water cooling model and scenario specifications can be either set manually in cli.py or specified in the --url option

Usage: mix-models water [OPTIONS] COMMAND [ARGS]...

Options:
--regions [ISR|R11|R12|R14|R32|RCP|ZMB]
                                 Code list to use for 'node' dimension.
--help                          Show this message and exit.

Commands:
cooling  Build and solve model with new cooling technologies.
nexus    Add basin structure connected to the energy sector and water...
report   function to run the water report_full from cli to the scenario...

Country vs Global implementation

The message_ix_models.model.water is designed to being able to add water components to either a global R11 (or R12) model or any country model designed with the MESSAGEix single country model prototype. For any of the region configuration a shapefile is needed to run the pre-processing part, while, once the data is prepared, only a .csv file similar to those in message_ix_models.data.water.delineation is needed.

To work with a country model please ensure that:

  1. country model and scenario are specified either in --url or in the cli.py script

  2. the option --regions is used with the ISO3 code of the country (e.g. for Israel --regions=ISR)

  3. Following the Israel example add a ‘country’.yaml file in message_ix_models.data.node for the specific country

  4. Following the Israel example add the country ISO3 code in the ‘regions’ options in message_ix_models.utils.click

Annual vs sub-annual implementation

if a sub-annual timestep is defined (e.g. seasons or months), the water module will automatically generate the water system with seasonality, using the time set components that is not year (assuming that the data has been prepared accordingly).

In the case you want to add seasonality in the water sector to a model with only annual timesteps, the best way is to pre-define the required sets (e.g. time and map_time). Then, running the water nexus module will automatically build the sub-annual water module.

In the case there are already multiple sub-annual time-steps levels already defined and not all are relevant to the water module, the components of the set time that are of interest for the water module should be manually added as a cli option (e.g. –time=year or –time=[1,2])

Code reference

message_ix_models.model.water.read_config(context=None)[source]

Read the water model configuration / metadata from file.

Numerical values are converted to computation-ready data structures.

Returns:

The current Context, with the loaded configuration.

Return type:

Context

Build and run

message_ix_models.model.water.build.get_spec(context) → Mapping[str, ScenarioInfo][source]

Return the specification for nexus implementation

Parameters:

context (Context) – The key regions determines the regional aggregation used.

message_ix_models.model.water.build.main(context, scenario, **options)[source]

Set up MESSAGEix-Nexus on scenario.

See also

add_data, apply_spec, get_spec

message_ix_models.model.water.build.map_basin(context) → Mapping[str, ScenarioInfo][source]

Return specification for mapping basins to regions

The basins are spatially consolidated from HydroSHEDS basins delineation database.This delineation is then intersected with MESSAGE regions to form new water sector regions for the nexus module. The nomenclature for basin names is <basin_id>|<MESSAGEregion> such as R1|AFR

Data preparation

Generate input data.

message_ix_models.model.water.data.add_data(scenario, context, dry_run=False)[source]

Populate scenario with MESSAGEix-Nexus data.

Prepare data for water use for cooling & energy technologies.

message_ix_models.model.water.data.water_for_ppl.cool_tech(context)[source]

Process cooling technology data for a scenario instance. The input values of parent technologies are read in from a scenario instance and then cooling fractions are calculated by using the data from tech_water_performance_ssp_msg.csv. It adds cooling technologies as addons to the parent technologies.The nomenclature for cooling technology is <parenttechnologyname>__<coolingtype>. E.g: coal_ppl__ot_fresh :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_for_ppl.non_cooling_tec(context)[source]

Process data for water usage of power plants (non-cooling technology related). Water withdrawal values for power plants are read in from tech_water_performance_ssp_msg.csv :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["transport build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

Prepare data for adding demands

message_ix_models.model.water.data.demands.add_irrigation_demand(context)[source]

Adds endogenous irrigation water demands from GLOBIOM emulator :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.add_sectoral_demands(context)[source]

Adds water sectoral demands :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.add_water_availability(context)[source]

Adds water supply constraints :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.get_basin_sizes(basin, node)[source]

Returns the sizes of developing and developed basins for a given node

message_ix_models.model.water.data.demands.read_water_availability(context)[source]

Reads water availability data and bias correct it for the historical years and no climate scenario assumptions.

Parameters:

context (Context) –

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.set_target_rate(df, node, year, target)[source]

Sets the target value for a given node and year

message_ix_models.model.water.data.demands.set_target_rate_developed(df, node, target)[source]

Sets target rate for a developed basin

message_ix_models.model.water.data.demands.set_target_rate_developing(df, node, target)[source]

Sets target rate for a developing basin

message_ix_models.model.water.data.demands.set_target_rates(df, basin, val)[source]

Sets target rates for all nodes in a given basin

message_ix_models.model.water.data.demands.target_rate(df, basin, val)[source]

Sets target connection and sanitation rates for SDG scenario. The function filters out the basins as developing and developed based on the countries overlapping basins. If the number of developing countries in the basins are more than basin is categorized as developing and vice versa. If the number of developing and developed countries are equal in a basin, then the basin is assumed developing. For developed basins, target is set at 2030. For developing basins, the access target is set at 2040 and 2035 target is the average of 2030 original rate and 2040 target. :returns: Data frame with updated value column. :rtype: df (pandas.DataFrame)

message_ix_models.model.water.data.demands.target_rate_trt(df, basin)[source]

Sets target treatment rates for SDG scenario. The target value for developed and developing region is making sure that the amount of untreated wastewater is halved beyond 2030 & 2040 respectively. :returns: data :rtype: dict of (str -> pandas.DataFrame)

Prepare data for adding techs related to water distribution, treatment in urban & rural

message_ix_models.model.water.data.infrastructure.add_desalination(context)[source]

Add desalination infrastructure Two types of desalination are considered; 1. Membrane 2. Distillation :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.infrastructure.add_infrastructure_techs(context)[source]

Process water distribution data for a scenario instance. :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

Prepare data for water use for cooling & energy technologies.

message_ix_models.model.water.data.water_supply.add_e_flow(context)[source]

Add environmental flows This function bounds the available water and allocates the environmental flows.Environmental flow bounds are calculated using Variable Monthly Flow (VMF) method. The VMF method is applied to wet and dry seasonal runoff values. These wet and dry seasonal values are then aggregated to annual values.Environmental flows in the model will be incorporated as bounds on ‘return_flow’ technology. The lower bound on this technology will ensure that certain amount of water remain :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_supply.add_water_supply(context)[source]

Add Water supply infrastructure This function links the water supply based on different settings and options. It defines the supply linkages for freshwater, groundwater and salinewater. :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_supply.map_basin_region_wat(context)[source]

Calculate share of water avaialbility of basins per each parent region.

The parent region could be global message regions or country

Parameters:

context (Context) –

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().

Return type:

dict of (str -> pandas.DataFrame)

Prepare data for water use for cooling & energy technologies.

message_ix_models.model.water.data.irrigation.add_irr_structure(context)[source]

Add irrigation withdrawal infrastructure The irrigation demands are added in :param context: :type context: Context

Returns:

data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.

Return type:

dict of (str -> pandas.DataFrame)

Utilities and CLI

message_ix_models.model.water.utils.map_add_on(rtype=<class 'sdmx.model.common.Code'>)[source]

Map addon & type_addon in sets.yaml.

message_ix_models.model.water.utils.map_yv_ya_lt(periods: Tuple[int, ...], lt: int, ya: int | None = None) → DataFrame[source]

All meaningful combinations of (vintage year, active year) given periods.

Parameters:

  • labels (pandas.DataFrame) – Each column (dimension) corresponds to one in df. Each row represents one matched set of labels for those dimensions.

  • lt (int, lifetime) –

message_ix_models.model.water.cli.cooling(context, regions, rcps, rels)[source]

Build and solve model with new cooling technologies.

Use the –url option to specify the base scenario.

Parameters:

  • context (class:message.Context) – Information about target Scenario.

  • regions (str (if not defined already in context.regions)) – Specifies what region definition is used [‘R11’,’R12’,’ISO3’]

  • RCP (str) – Specifies the climate scenario used [‘no_climate’,’6p0’,’2p6’]

message_ix_models.model.water.cli.nexus(context, regions, rcps, sdgs, rels, macro=False)[source]

Add basin structure connected to the energy sector and water balance linking different water demands to supply.

Use the –url option to specify the base scenario.

Parameters:

  • context (class:message.Context) – Information about target Scenario.

  • regions (str (if not defined already in context.regions)) – Specifies what region definition is used [‘R11’,’R12’,’ISO3’]

  • RCP (str) – Specifies the climate scenario used [‘no_climate’,’6p0’,’2p6’]

  • SDG (Str) – Defines if and what water SDG measures are activated

  • REL (str) – Specifies the reliability of hydrological data [‘low’,’mid’,’high’]

message_ix_models.model.water.cli.water_ini(context, regions, time)[source]

Add components of the MESSAGEix-Nexus module

This function modifies model name & scenario name and verifies the region setup :param context: Information about target Scenario. :type context: class:message.Context :param regions: Specifies what region definition is used [‘R11’,’R12’,’ISO3’] :type regions: str (if not defined already in context.regions)

Reporting

Warning

The current reporting features only work for the global model.

message_ix_models.model.water.reporting.multiply_electricity_output_of_hydro(elec_hydro_var, report_iam)[source]

Function to multiply electricity output of hydro to get withdrawals :param elec_hydro_var: List of variables with electricity output of hydro :type elec_hydro_var: list :param report_iam: Report in pyam format :type report_iam: pyam.IamDataFrame

Returns:

report_iam – Report in pyam format

Return type:

pyam.IamDataFrame

message_ix_models.model.water.reporting.report(sc=False, reg='', sdgs=False)[source]

Report nexus module results

message_ix_models.model.water.reporting.report_full(sc=False, reg='', sdgs=False)[source]

Combine old and new reporting workflows

message_ix_models.model.water.reporting.report_iam_definition(sc, rep, df_dmd, rep_dm, report_df, suban)[source]

Function to define the report iam dataframe :param sc: Scenario to report :type sc: ixmp.Scenario :param rep: Reporter object :type rep: Reporter :param suban: True if subannual, False if annual :type suban: bool :param df_dmd: Dataframe with demands :type df_dmd: pd.DataFrame :param rep_dm: Reporter object for demands :type rep_dm: Reporter :param report_df: Dataframe with report :type report_df: pd.DataFrame

Returns:

report_iam – Report in pyam format

Return type:

pyam.IamDataFrame

Data, metadata, and config files

See also Data and configuration files.

  • data/water/: contains input data used for building the Nexus module

    • delineation/: contains geospatial files for basin mapping and MESSAGE regions. These spatial files are created through intersecting HydroSHEDS basin and the MESSAGE region shapefile. The scripts and processing data at ‘P:ene.modelNESTdelineation’

    • ppl_cooling_tech/: contains cooling technology shares, costs and water intensities for different regional definitions

    • water_demands/: contains water sectoral demands, connection rates for basins

    • water_dist/: contains water infrastructure (distribution, treatment mapping) and historical and projected capacities of desalination technologies

    • technology.yaml: metadata for the ‘technology’ dimension.

    • set.yaml: metadata for other sets.

Pre-processing

  • data/water/: contains scripts used in pre_processing source data for the water sector implementation

    • calculate_ppl_cooling_technology_shares.r: contains script for processing cooling technology shares at global level for different regional specifications.

    • groundwater_harmonize.r: contains workflow to calculate historical capacity of renewable groundwater, table depth and energy consumption

    • generate_water_constraints.r: contains function to calculate municipal, manufacturing, rural water demands, water access and sanitation rates

    • desalination.r: contains script for assessing the historical and possible future desalination capacity of a region or country

    • hydro_agg_raster.py: contains workflow for processing the hydrological data in NC4 and adjust the unit conversions, daily to monthly aggregation.

    • hydro_agg_spatial.R: contains workflow for spatially aggregating monthly hydrological data onto basin using appropriate raster masking onto shapefiles

    • hydro_agg_basin.py: contains workflow for aggregating monthly data to 5 yearly averages using appropriate statistical methods (quantiles, averages etc.). It also calculates e flows based on Variable MF method.

Deprecated R Code

  • data/water/deprecated: contains R scripts from the older water sector implementation

    • Figures.R: R script for producing figures

    • cooling_tech_av.R: contains similar code as in the above-mentioned scripts, but this was originated from another workstream.

    • add_water_infrastructure.R: contains spatially-explicit analysis of gridded demands and socioeconomic indicators to develop pathways for sectoral water withdrawals, return flows and infrastructure penetration rates in each MESSAGE region. The pathways feature branching points reflecting a specific water sector development narrative (e.g., convergence towards achieving specific SDG targets).

Reference