MESSAGEix-Nexus (model.water
)
message_ix_models.model.water
adds water usage and demand related representation to the MESSAGEix-GLOBIOM global model.
The resulting model is referred to as “MESSAGEix-Nexus”.
This work extends the water sector linkage described by Parkinson et al. (2019) [72].
CLI usage
Use the CLI command mix-data water
to invoke the commands defined in water.cli
. Example:
mix-models --url=ixmp://ixmp_dev/ENGAGE_SSP2_v4.1.7/baseline_clone_test water cooling
model and scenario specifications can be either set manually in cli.py
or specified in the --url
option
Usage: mix-models water [OPTIONS] COMMAND [ARGS]...
Options:
--regions [ISR|R11|R12|R14|R32|RCP|ZMB]
Code list to use for 'node' dimension.
--help Show this message and exit.
Commands:
cooling Build and solve model with new cooling technologies.
nexus Add basin structure connected to the energy sector and water...
report function to run the water report_full from cli to the scenario...
Country vs Global implementation
The message_ix_models.model.water
is designed to being able to add water components to either a global R11 (or R12) model or any country model designed with the MESSAGEix single country model prototype.
For any of the region configuration a shapefile is needed to run the pre-processing part, while, once the data is prepared, only a .csv file similar to those in message_ix_models.data.water.delineation is needed.
To work with a country model please ensure that:
country model and scenario are specified either in
--url
or in thecli.py
scriptthe option
--regions
is used with the ISO3 code of the country (e.g. for Israel--regions=ISR
)Following the Israel example add a ‘country’.yaml file in message_ix_models.data.node for the specific country
Following the Israel example add the country ISO3 code in the ‘regions’ options in message_ix_models.utils.click
Annual vs sub-annual implementation
if a sub-annual timestep is defined (e.g. seasons or months), the water module will automatically generate the water system with seasonality, using the time set components that is not year (assuming that the data has been prepared accordingly).
In the case you want to add seasonality in the water sector to a model with only annual timesteps, the best way is to pre-define the required sets (e.g. time and map_time). Then, running the water nexus module will automatically build the sub-annual water module.
In the case there are already multiple sub-annual time-steps levels already defined and not all are relevant to the water module, the components of the set time that are of interest for the water module should be manually added as a cli option (e.g. –time=year or –time=[1,2])
Code reference
- message_ix_models.model.water.read_config(context: Context | None = None)[source]
Read the water model configuration / metadata from file.
Numerical values are converted to computation-ready data structures.
- Returns:
The current Context, with the loaded configuration.
- Return type:
Build and run
- message_ix_models.model.water.build.get_spec(context: Context) Mapping[str, ScenarioInfo] [source]
Return the specification for nexus implementation
- Parameters:
context (
Context
) – The keyregions
determines the regional aggregation used.
- message_ix_models.model.water.build.main(context: Context, scenario, **options)[source]
Set up MESSAGEix-Nexus on scenario.
See also
add_data
,apply_spec
,get_spec
- message_ix_models.model.water.build.map_basin(context: Context) Mapping[str, ScenarioInfo] [source]
Return specification for mapping basins to regions
The basins are spatially consolidated from HydroSHEDS basins delineation database.This delineation is then intersected with MESSAGE regions to form new water sector regions for the nexus module. The nomenclature for basin names is <basin_id>|<MESSAGEregion> such as R1|AFR
Data preparation
Generate input data.
- message_ix_models.model.water.data.add_data(scenario, context: Context, dry_run=False)[source]
Populate scenario with MESSAGEix-Nexus data.
Prepare data for water use for cooling & energy technologies.
- message_ix_models.model.water.data.water_for_ppl.cool_tech(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Process cooling technology data for a scenario instance. The input values of parent technologies are read in from a scenario instance and then cooling fractions are calculated by using the data from
tech_water_performance_ssp_msg.csv
. It adds cooling technologies as addons to the parent technologies. The nomenclature for cooling technology is <parenttechnologyname>__<coolingtype>. E.g: coal_ppl__ot_fresh- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["water build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
- message_ix_models.model.water.data.water_for_ppl.cooling_fr(x: Series) float [source]
Calculate cooling fraction
- Returns:
The calculated cooling fraction after for two categories;
1. Technologies that produce heat as an output
– cooling_fraction(h_cool) = input value(hi) - 1Simply subtract 1 from the heating value since the rest
ofthe part is already
accounted in the heating value
2. Rest
oftechnologies
– h_cool = hi -Hi* h_fg - 1, where:h_fg (flue gasses losses) = 0.1 (10% assumed losses)
- message_ix_models.model.water.data.water_for_ppl.hist_act(x: Series, context: Context, hold_cost: DataFrame) list [source]
Calculate historical activity of cooling technology. The data for shares is read from
cooltech_cost_and_shares_ssp_msg.csv
- Returns:
hist_activity(cooling_tech) = hist_activitiy(parent_technology) * share
*cooling_fraction
- message_ix_models.model.water.data.water_for_ppl.hist_cap(x: Series, context: Context, hold_cost: DataFrame) list [source]
Calculate historical capacity of cooling technology. The data for shares is read from
cooltech_cost_and_shares_ssp_msg.csv
- Returns:
hist_new_capacity(cooling_tech) = historical_new_capacity(parent_technology)*
share * cooling_fraction
- message_ix_models.model.water.data.water_for_ppl.missing_tech(x: Series) Series [source]
Assign values to missing data. It goes through the input data frame and extract the technologies which don’t have input values and then assign manual values to those technologies along with assigning them an arbitrary level i.e dummy supply
- message_ix_models.model.water.data.water_for_ppl.non_cooling_tec(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Process data for water usage of power plants (non-cooling technology related). Water withdrawal values for power plants are read in from
tech_water_performance_ssp_msg.csv
- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["transport build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
Process share and cooling fraction.
- Returns:
Product
ofvalue
ofshares
ofcooling technology types
ofregions with
corresponding cooling fraction
Prepare data for adding demands
- message_ix_models.model.water.data.demands.add_irrigation_demand(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Adds endogenous irrigation water demands from GLOBIOM emulator
- message_ix_models.model.water.data.demands.add_sectoral_demands(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Adds water sectoral demands
- message_ix_models.model.water.data.demands.add_water_availability(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Adds water supply constraints
- message_ix_models.model.water.data.demands.get_basin_sizes(basin: DataFrame, node: str) Sequence[Series | Literal[0]] [source]
Returns the sizes of developing and developed basins for a given node
- message_ix_models.model.water.data.demands.read_water_availability(context: Context) Sequence[DataFrame] [source]
Reads water availability data and bias correct it for the historical years and no climate scenario assumptions.
- Parameters:
context (
Context
) –- Returns:
data
- Return type:
(pd.DataFrame
,pd.DataFrame)
- message_ix_models.model.water.data.demands.set_target_rate(df: DataFrame, node: str, year: int, target: float) None [source]
Sets the target value for a given node and year
- message_ix_models.model.water.data.demands.set_target_rate_developed(df: DataFrame, node: str, target: float) None [source]
Sets target rate for a developed basin
- message_ix_models.model.water.data.demands.set_target_rate_developing(df: DataFrame, node: str, target: float) None [source]
Sets target rate for a developing basin
- message_ix_models.model.water.data.demands.set_target_rates(df: DataFrame, basin: DataFrame, val: float) None [source]
Sets target rates for all nodes in a given basin
- message_ix_models.model.water.data.demands.target_rate(df: DataFrame, basin: DataFrame, val: float) DataFrame [source]
Sets target connection and sanitation rates for SDG scenario. The function filters out the basins as developing and developed based on the countries overlapping basins. If the number of developing countries in the basins are more than basin is categorized as developing and vice versa. If the number of developing and developed countries are equal in a basin, then the basin is assumed developing. For developed basins, target is set at 2030. For developing basins, the access target is set at 2040 and 2035 target is the average of 2030 original rate and 2040 target.
- Returns:
df (pandas.DataFrame)
- Return type:
Data frame with updated value column.
- message_ix_models.model.water.data.demands.target_rate_trt(df: DataFrame, basin: DataFrame) DataFrame [source]
Sets target treatment rates for SDG scenario. The target value for developed and developing region is making sure that the amount of untreated wastewater is halved beyond 2030 & 2040 respectively.
- Returns:
data
- Return type:
Prepare data for adding techs related to water distribution, treatment in urban & rural
- message_ix_models.model.water.data.infrastructure.add_desalination(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Add desalination infrastructure Two types of desalination are considered; 1. Membrane 2. Distillation
- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["water build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
- message_ix_models.model.water.data.infrastructure.add_infrastructure_techs(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Process water distribution data for a scenario instance.
- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["water build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
- message_ix_models.model.water.data.infrastructure.start_creating_input_dataframe(sdg: str, df_node: DataFrame, df_non_elec: DataFrame, df_dist: DataFrame, year_wat: tuple, first_year: int, sub_time) DataFrame [source]
Creates an input pd.DataFrame and adds some data to it.
Prepare data for water use for cooling & energy technologies.
- message_ix_models.model.water.data.water_supply.add_e_flow(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Add environmental flows This function bounds the available water and allocates the environmental flows.Environmental flow bounds are calculated using Variable Monthly Flow (VMF) method. The VMF method is applied to wet and dry seasonal runoff values. These wet and dry seasonal values are then aggregated to annual values.Environmental flows in the model will be incorporated as bounds on ‘return_flow’ technology. The lower bound on this technology will ensure that certain amount of water remain
- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["water build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
- message_ix_models.model.water.data.water_supply.add_water_supply(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Add Water supply infrastructure This function links the water supply based on different settings and options. It defines the supply linkages for freshwater, groundwater and salinewater.
- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["water build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
- message_ix_models.model.water.data.water_supply.map_basin_region_wat(context: Context) DataFrame [source]
Calculate share of water availability of basins per each parent region.
The parent region could be global message regions or country
- Parameters:
context (
Context
) –- Returns:
data
- Return type:
Prepare data for water use for cooling & energy technologies.
- message_ix_models.model.water.data.irrigation.add_irr_structure(context: Context) dict[str, pandas.core.frame.DataFrame] [source]
Add irrigation withdrawal infrastructure The irrigation demands are added in
- Parameters:
context (
Context
) –- Returns:
data – Keys are MESSAGE parameter names such as ‘input’, ‘fix_cost’. Values are data frames ready for
add_par()
. Years in the data include the model horizon indicated bycontext["water build info"]
, plus the additional year 2010.- Return type:
dict
of(str -> pandas.DataFrame)
Utilities and CLI
- message_ix_models.model.water.utils.map_add_on(rtype=<class 'sdmx.model.common.Code'>)[source]
Map addon & type_addon in
sets.yaml
.
- message_ix_models.model.water.utils.map_yv_ya_lt(periods: Tuple[int, ...], lt: int, ya: int | None = None) DataFrame [source]
All meaningful combinations of (vintage year, active year) given periods.
- message_ix_models.model.water.cli.cooling(context, regions, rcps, rels)[source]
Build and solve model with new cooling technologies.
Use the –url option to specify the base scenario.
- Parameters:
context (class:message.Context) – Information about target Scenario.
regions (
str (if not defined already in context.regions)
) – Specifies what region definition is used [‘R11’,’R12’,’ISO3’]RCP (
str
) – Specifies the climate scenario used [‘no_climate’,’6p0’,’2p6’]
- message_ix_models.model.water.cli.nexus(context: Context, regions, rcps, sdgs, rels, macro=False)[source]
Add basin structure connected to the energy sector and water balance linking different water demands to supply.
Use the –url option to specify the base scenario.
- Parameters:
context (class:message.Context) – Information about target Scenario.
regions (
str (if not defined already in context.regions)
) – Specifies what region definition is used [‘R11’,’R12’,’ISO3’]RCP (
str
) – Specifies the climate scenario used [‘no_climate’,’6p0’,’2p6’]SDG (
Str
) – Defines if and what water SDG measures are activatedREL (
str
) – Specifies the reliability of hydrological data [‘low’,’mid’,’high’]
- message_ix_models.model.water.cli.water_ini(context: Context, regions, time)[source]
Add components of the MESSAGEix-Nexus module
This function modifies model name & scenario name and verifies the region setup :param context: Information about target Scenario. :type context: class:message.Context :param regions: Specifies what region definition is used [‘R11’,’R12’,’ISO3’] :type regions:
str (if not defined already in context.regions)
Reporting
Warning
The current reporting features only work for the global model.
Data, metadata, and config files
See also Data and configuration files.
data/water/
: contains input data used for building the Nexus moduledelineation/
: contains geospatial files for basin mapping and MESSAGE regions. These spatial files are created through intersecting HydroSHEDS basin and the MESSAGE region shapefile. The scripts and processing data at ‘P:ene.modelNESTdelineation’ppl_cooling_tech/
: contains cooling technology shares, costs and water intensities for different regional definitionswater_demands/
: contains water sectoral demands, connection rates for basinswater_dist/
: contains water infrastructure (distribution, treatment mapping) and historical and projected capacities of desalination technologiestechnology.yaml
: metadata for the ‘technology’ dimension.set.yaml
: metadata for other sets.
Pre-processing
data/water/
: contains scripts used in pre_processing source data for the water sector implementationcalculate_ppl_cooling_technology_shares.r
: contains script for processing cooling technology shares at global level for different regional specifications.groundwater_harmonize.r
: contains workflow to calculate historical capacity of renewable groundwater, table depth and energy consumptiongenerate_water_constraints.r
: contains function to calculate municipal, manufacturing, rural water demands, water access and sanitation ratesdesalination.r
: contains script for assessing the historical and possible future desalination capacity of a region or countryhydro_agg_raster.py
: contains workflow for processing the hydrological data in NC4 and adjust the unit conversions, daily to monthly aggregation.hydro_agg_spatial.R
: contains workflow for spatially aggregating monthly hydrological data onto basin using appropriate raster masking onto shapefileshydro_agg_basin.py
: contains workflow for aggregating monthly data to 5 yearly averages using appropriate statistical methods (quantiles, averages etc.). It also calculates e flows based on Variable MF method.
Deprecated R Code
data/water/deprecated
: contains R scripts from the older water sector implementationFigures.R
: R script for producing figurescooling_tech_av.R
: contains similar code as in the above-mentioned scripts, but this was originated from another workstream.add_water_infrastructure.R
: contains spatially-explicit analysis of gridded demands and socioeconomic indicators to develop pathways for sectoral water withdrawals, return flows and infrastructure penetration rates in each MESSAGE region. The pathways feature branching points reflecting a specific water sector development narrative (e.g., convergence towards achieving specific SDG targets).