MESSAGEix-Nexus (`model.water`)

Note

The current version of model.water is configured to work with scenarios that have technology set elements including csp_sm1_ppl and csp_sm3_ppl instead of solar_th_ppl. Previous versions (in message_ix_models v2024.8.6 and earlier) were configured to work with scenarios including the technology solar_th_ppl. See further discussion at PR #242.

message_ix_models.model.water adds water usage and demand related representation to the MESSAGEix-GLOBIOM global model. The resulting model is referred to as “MESSAGEix-Nexus”. This work extends the water sector linkage described by Parkinson et al. (2019) [72].

CLI usage 

Use the CLI command mix-data water to invoke the commands defined in water.cli. Example: mix-models --url=ixmp://ixmp_dev/ENGAGE_SSP2_v4.1.7/baseline_clone_test water cooling model and scenario specifications can be either set manually in cli.py or specified in the --url option

Usage: mix-models water [OPTIONS] COMMAND [ARGS]...

Options:
--regions [ISR|R11|R12|R14|R32|RCP|ZMB]
                                 Code list to use for 'node' dimension.
--help                          Show this message and exit.

Commands:
cooling  Build and solve model with new cooling technologies.
nexus    Add basin structure connected to the energy sector and water...
report   function to run the water report_full from cli to the scenario...

Country vs Global implementation 

The message_ix_models.model.water is designed to being able to add water components to either a global R11 (or R12) model or any country model designed with the MESSAGEix single country model prototype. For any of the region configuration a shapefile is needed to run the pre-processing part, while, once the data is prepared, only a .csv file similar to those in message_ix_models.data.water.delineation is needed.

To work with a country model please ensure that:

country model and scenario are specified either in --url or in the cli.py script
the option --regions is used with the ISO3 code of the country (e.g. for Israel --regions=ISR)
Following the Israel example add a ‘country’.yaml file in message_ix_models.data.node for the specific country
Following the Israel example add the country ISO3 code in the ‘regions’ options in message_ix_models.utils.click

Annual vs sub-annual implementation 

if a sub-annual timestep is defined (e.g. seasons or months), the water module will automatically generate the water system with seasonality, using the time set components that is not year (assuming that the data has been prepared accordingly).

In the case you want to add seasonality in the water sector to a model with only annual timesteps, the best way is to pre-define the required sets (e.g. time and map_time). Then, running the water nexus module will automatically build the sub-annual water module.

In the case there are already multiple sub-annual time-steps levels already defined and not all are relevant to the water module, the components of the set time that are of interest for the water module should be manually added as a cli option (e.g. –time=year or –time=[1,2])

Code reference 

message_ix_models.model.water.read_config(context: Context | None = None)[source]

Read the water model configuration / metadata from file.

Numerical values are converted to computation-ready data structures.

Returns:: The current Context, with the loaded configuration.
Return type:: Context

Build and run 

message_ix_models.model.water.build.get_spec(context: Context) → Mapping[str, ScenarioInfo][source]

Return the specification for nexus implementation

Parameters:: context (Context) – The key regions determines the regional aggregation used.

message_ix_models.model.water.build.main(context: Context, scenario, **options)[source]: Set up MESSAGEix-Nexus on scenario.

See also

add_data, apply_spec, get_spec

message_ix_models.model.water.build.map_basin(context: Context) → Mapping[str, ScenarioInfo][source]

Return specification for mapping basins to regions

The basins are spatially consolidated from HydroSHEDS basins delineation database.This delineation is then intersected with MESSAGE regions to form new water sector regions for the nexus module. The nomenclature for basin names is <basin_id>|<MESSAGEregion> such as R1|AFR

Data preparation 

Generate input data.

message_ix_models.model.water.data.add_data(scenario, context: Context, dry_run=False)[source]: Populate scenario with MESSAGEix-Nexus data.

Prepare data for water use for cooling & energy technologies.

message_ix_models.model.water.data.water_for_ppl.cool_tech(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Process cooling technology data for a scenario instance. The input values of parent technologies are read in from a scenario instance and then cooling fractions are calculated by using the data from tech_water_performance_ssp_msg.csv. It adds cooling technologies as addons to the parent technologies. The nomenclature for cooling technology is <parenttechnologyname>__<coolingtype>. E.g: coal_ppl__ot_fresh

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_for_ppl.cooling_fr(x: Series) → float[source]

Calculate cooling fraction

Returns:

The calculated cooling fraction after for two categories;
1. Technologies that produce heat as an output – cooling_fraction(h_cool) = input value(hi) - 1
Simply subtract 1 from the heating value since the rest of the part is already
accounted in the heating value
2. Rest of technologies – h_cool = hi -Hi* h_fg - 1, where:

h_fg (flue gasses losses) = 0.1 (10% assumed losses)

message_ix_models.model.water.data.water_for_ppl.hist_act(x: Series, context: Context, hold_cost: DataFrame) → list[source]

Calculate historical activity of cooling technology. The data for shares is read from cooltech_cost_and_shares_ssp_msg.csv

Returns:

hist_activity(cooling_tech) = hist_activitiy(parent_technology) * share
*cooling_fraction

message_ix_models.model.water.data.water_for_ppl.hist_cap(x: Series, context: Context, hold_cost: DataFrame) → list[source]

Calculate historical capacity of cooling technology. The data for shares is read from cooltech_cost_and_shares_ssp_msg.csv

Returns:

hist_new_capacity(cooling_tech) = historical_new_capacity(parent_technology)*
share * cooling_fraction

message_ix_models.model.water.data.water_for_ppl.missing_tech(x: Series) → Series[source]: Assign values to missing data. It goes through the input data frame and extract the technologies which don’t have input values and then assign manual values to those technologies along with assigning them an arbitrary level i.e dummy supply

message_ix_models.model.water.data.water_for_ppl.non_cooling_tec(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Process data for water usage of power plants (non-cooling technology related). Water withdrawal values for power plants are read in from tech_water_performance_ssp_msg.csv

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["transport build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_for_ppl.relax_growth_constraint(ref_hist: DataFrame, scen, cooling_df: DataFrame, g_lo: DataFrame, constraint_type: Union[ForwardRef('activity'), ForwardRef('new_capacity'), ForwardRef('total_capacity')]]) → DataFrame[source]

Checks if the parent technologies are shut down and require relaxing the growth constraint.

Parameters:

ref_hist (pd.DataFrame) – Historical data in the reference scenario.
scen (Scenario) – Scenario object to retrieve necessary parameters.
cooling_df (pd.DataFrame) – DataFrame containing information on cooling technologies and their parent technologies.
g_lo (pd.DataFrame) – DataFrame containing growth constraints for each technology.
constraint_type ({"activity", "new_capacity"}) – Type of constraint to check, either “activity” for operational limits or “new_capacity” for capacity expansion limits.

Returns:

Updated g_lo DataFrame with relaxed growth constraints.

Return type:

pd.DataFrame

message_ix_models.model.water.data.water_for_ppl.shares(x: Series, context: Context, search_cols_cooling_fraction: list, hold_df: DataFrame, search_cols: list) → Series[source]

Process share and cooling fraction.

Returns:

Product of value of shares of cooling technology types of regions with
corresponding cooling fraction

Prepare data for adding demands

message_ix_models.model.water.data.demands.add_irrigation_demand(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Adds endogenous irrigation water demands from GLOBIOM emulator

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.add_sectoral_demands(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Adds water sectoral demands

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.add_water_availability(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Adds water supply constraints

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par().
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.demands.get_basin_sizes(basin: DataFrame, node: str) → Sequence[Series | Literal[0]][source]: Returns the sizes of developing and developed basins for a given node

message_ix_models.model.water.data.demands.read_water_availability(context: Context) → Sequence[DataFrame][source]

Reads water availability data and bias correct it for the historical years and no climate scenario assumptions.

Parameters:: context (Context) –
Returns:: data
Return type:: (pd.DataFrame, pd.DataFrame)

message_ix_models.model.water.data.demands.set_target_rate(df: DataFrame, node: str, year: int, target: float) → None[source]: Sets the target value for a given node and year

message_ix_models.model.water.data.demands.set_target_rate_developed(df: DataFrame, node: str, target: float) → None[source]: Sets target rate for a developed basin

message_ix_models.model.water.data.demands.set_target_rate_developing(df: DataFrame, node: str, target: float) → None[source]: Sets target rate for a developing basin

message_ix_models.model.water.data.demands.set_target_rates(df: DataFrame, basin: DataFrame, val: float) → None[source]: Sets target rates for all nodes in a given basin

message_ix_models.model.water.data.demands.target_rate(df: DataFrame, basin: DataFrame, val: float) → DataFrame[source]

Sets target connection and sanitation rates for SDG scenario. The function filters out the basins as developing and developed based on the countries overlapping basins. If the number of developing countries in the basins are more than basin is categorized as developing and vice versa. If the number of developing and developed countries are equal in a basin, then the basin is assumed developing. For developed basins, target is set at 2030. For developing basins, the access target is set at 2040 and 2035 target is the average of 2030 original rate and 2040 target.

Returns:: df (pandas.DataFrame)
Return type:: Data frame with updated value column.

message_ix_models.model.water.data.demands.target_rate_trt(df: DataFrame, basin: DataFrame) → DataFrame[source]

Sets target treatment rates for SDG scenario. The target value for developed and developing region is making sure that the amount of untreated wastewater is halved beyond 2030 & 2040 respectively.

Returns:: data
Return type:: pandas.DataFrame

Prepare data for adding techs related to water distribution, treatment in urban & rural

message_ix_models.model.water.data.infrastructure.add_desalination(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Add desalination infrastructure Two types of desalination are considered; 1. Membrane 2. Distillation

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.infrastructure.add_infrastructure_techs(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Process water distribution data for a scenario instance.

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.infrastructure.start_creating_input_dataframe(sdg: str, df_node: DataFrame, df_non_elec: DataFrame, df_dist: DataFrame, year_wat: tuple, first_year: int, sub_time) → DataFrame[source]: Creates an input pd.DataFrame and adds some data to it.

Prepare data for water use for cooling & energy technologies.

message_ix_models.model.water.data.water_supply.add_e_flow(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Add environmental flows This function bounds the available water and allocates the environmental flows.Environmental flow bounds are calculated using Variable Monthly Flow (VMF) method. The VMF method is applied to wet and dry seasonal runoff values. These wet and dry seasonal values are then aggregated to annual values.Environmental flows in the model will be incorporated as bounds on ‘return_flow’ technology. The lower bound on this technology will ensure that certain amount of water remain

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_supply.add_water_supply(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Add Water supply infrastructure This function links the water supply based on different settings and options. It defines the supply linkages for freshwater, groundwater and salinewater.

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

message_ix_models.model.water.data.water_supply.map_basin_region_wat(context: Context) → DataFrame[source]

Calculate share of water availability of basins per each parent region.

The parent region could be global message regions or country

Parameters:: context (Context) –
Returns:: data
Return type:: pandas.DataFrame

Prepare data for water use for cooling & energy technologies.

message_ix_models.model.water.data.irrigation.add_irr_structure(context: Context) → dict[str, pandas.core.frame.DataFrame][source]

Add irrigation withdrawal infrastructure The irrigation demands are added in

Parameters:: context (Context) –
Returns:: data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for add_par(). Years in the data include the model horizon indicated by context["water build info"], plus the additional year 2010.
Return type:: dict of (str -> pandas.DataFrame)

Utilities and CLI 

message_ix_models.model.water.utils.map_add_on(rtype=<class 'sdmx.model.common.Code'>)[source]: Map addon & type_addon in sets.yaml.

message_ix_models.model.water.utils.map_yv_ya_lt(periods: tuple[int, ...], lt: int, ya: int | None = None) → DataFrame[source]

All meaningful combinations of (vintage year, active year) given periods.

Parameters:

periods (tuple[int, ]) – A sequence of years.
lt (int, lifetime) –
ya (int, active year) – The first active year.

Returns:

A DataFrame with columns ‘year_vtg’ and ‘year_act’.

Return type:

pd.DataFrame

message_ix_models.model.water.cli.cooling(context, regions, rcps, rels)[source]

Build and solve model with new cooling technologies.

Use the –url option to specify the base scenario.

Parameters:

context (class:message.Context) – Information about target Scenario.
regions (str (if not defined already in context.regions)) – Specifies what region definition is used [‘R11’,’R12’,’ISO3’]
RCP (str) – Specifies the climate scenario used [‘no_climate’,’6p0’,’2p6’]

message_ix_models.model.water.cli.nexus(context: Context, regions, rcps, sdgs, rels, macro=False)[source]

Add basin structure connected to the energy sector and water balance linking different water demands to supply.

Use the –url option to specify the base scenario.

Parameters:

context (class:message.Context) – Information about target Scenario.
regions (str (if not defined already in context.regions)) – Specifies what region definition is used [‘R11’,’R12’,’ISO3’]
RCP (str) – Specifies the climate scenario used [‘no_climate’,’6p0’,’2p6’]
SDG (Str) – Defines if and what water SDG measures are activated
REL (str) – Specifies the reliability of hydrological data [‘low’,’mid’,’high’]

message_ix_models.model.water.cli.water_ini(context: Context, regions, time)[source]

Add components of the MESSAGEix-Nexus module

This function modifies model name & scenario name and verifies the region setup :param context: Information about target Scenario. :type context: class:message.Context :param regions: Specifies what region definition is used [‘R11’,’R12’,’ISO3’] :type regions: str (if not defined already in context.regions)

data/water/: contains scripts used in pre_processing source data for the water sector implementation
- calculate_ppl_cooling_technology_shares.r: contains script for processing cooling technology shares at global level for different regional specifications.
- groundwater_harmonize.r: contains workflow to calculate historical capacity of renewable groundwater, table depth and energy consumption
- generate_water_constraints.r: contains function to calculate municipal, manufacturing, rural water demands, water access and sanitation rates
- desalination.r: contains script for assessing the historical and possible future desalination capacity of a region or country
- hydro_agg_raster.py: contains workflow for processing the hydrological data in NC4 and adjust the unit conversions, daily to monthly aggregation.
- hydro_agg_spatial.R: contains workflow for spatially aggregating monthly hydrological data onto basin using appropriate raster masking onto shapefiles
- hydro_agg_basin.py: contains workflow for aggregating monthly data to 5 yearly averages using appropriate statistical methods (quantiles, averages etc.). It also calculates e flows based on Variable MF method.

Deprecated R Code 

data/water/deprecated: contains R scripts from the older water sector implementation
- Figures.R: R script for producing figures
- cooling_tech_av.R: contains similar code as in the above-mentioned scripts, but this was originated from another workstream.
- add_water_infrastructure.R: contains spatially-explicit analysis of gridded demands and socioeconomic indicators to develop pathways for sectoral water withdrawals, return flows and infrastructure penetration rates in each MESSAGE region. The pathways feature branching points reflecting a specific water sector development narrative (e.g., convergence towards achieving specific SDG targets).

Reference 

Data and configuration files

MESSAGEix-Nexus (`model.water`)

CLI usage 

Country vs Global implementation 

Annual vs sub-annual implementation 

Code reference 

Build and run 

Data preparation 

Utilities and CLI 

Reporting 

Data, metadata, and config files 

Pre-processing 

Deprecated R Code 

Reference 

MESSAGEix-Nexus (model.water)

MESSAGEix-Nexus (`model.water`)