Investment and fixed costs (tools.costs)

tools.costs implements methods to project investment and fixed costs of technologies [1] in MESSAGEix-GLOBIOM family models.

Methods

The tool creates distinct projected cost values for different regions, technologies, and scenarios. The costs are projected based on historical (mostly a base year) data and assumptions about future cost reductions.

The projections use the concept of a reference region [2] and apply distinct methods to the reference and non-reference regions:

Reference region

Costs in the reference region are projected based on the following assumption: given a cost reduction rate, the cost of the technology in the reference region experiences an exponential decay over time.

Non-reference regions

Costs for each technology in all non-reference regions may be calculated using one of three methods, specified using Config.method:

  1. Constant cost reduction rate (Config.method = “constant”): the regional cost ratio (versus the reference region) that is calculated in the base year is held constant and used to project regionally-differentiated costs across all years.

  2. Convergence to reference region costs by a certain year (Config.method = “convergence”): all other regions’ costs exponentially decay until they become they same as the reference region’s cost by a specified year.

  3. GDP-adjusted cost reduction rate (Config.method = “gdp”): this method assumes that regional costs converge not based on a specified year but based on GDP per capita. All non-reference regions’ costs are adjusted based on the ratio of the GDP per capita of the region to the GDP per capita of the reference region.

Modules and model variants

Within the context of the tool, the term module (specified by Config.module) is used to mean input data for particular sets of technologies. These correspond to subsets of all the technologies in MESSAGEix-GLOBIOM models—either the base model or model variants. [3] Currently, tools.costs supports two module module settings:

“energy”

Mostly electric power technologies, as well as a few other supply-side technologies.

This can be considered the “base” module, corresponding to the “base” version of MESSAGEix-GLOBIOM, as it contains the most technologies.

“materials”

Technologies conceived as part of the materials and industry sectors.

Data and files for a particular module can refer to other modules. This allows for values or settings for “materials” and other technologies to be assumed to match the values and settings used for the referenced “energy”-module technologies.

To add a new module, the following steps are required:

  • In message_ix_models/data/costs/, create another subdirectory with the name of the new module, for instance message_ix_models/data/costs/[module]/.

  • Add the following files to the new directory:

    first_year_[module].csv

    A file with a list of technologies and the corresponding first year that the respective technology can start being deployed/modeled. The file should have the following columns:

    • “message_technology”: the technology name.

    • “first_year_original”: the first year the technology can start being deployed.

    tech_map_[module].csv

    A file with the mapping of technologies to a source of base year cost data. The file should have the following columns:

    • “message_technology”: the technology name.

    • “reg_diff_source” and “reg_diff_technology”: the source data for the regional differentiation of costs and the corresponding technology to map to.

      • If “reg_diff_source” is “energy”, then “reg_diff_technology” should be a technology that is present in the “energy” module.

      • If “reg_diff_source” is “weo”, then “reg_diff_technology” should be a technology that is present in the WEO data (refer to tech_map_energy.csv for the names of WEO technologies available, as all energy technologies are mapped to a WEO technology).

      • You can also add another source of regional differentiation (in the case of module="materials", a newly created source called “intratec” is used). However, this method is a little more involved as it requires extending the code to read in new source data.

    • “base_year_reference_region_cost”: the base year cost for the technology in the reference region.

    • “fix_ratio”: the ratio of fixed O&M costs to investment costs for the technology.

  • Add the new module to the allowed values of Config.module.

Please note that the following assumptions are made in technology costs mapping:

  • If a technology is mapped to a technology in the “energy” module, then the cost reduction across scenarios is the same as the cost reduction of the mapped technology.

  • If a “materials” (or any other non-“energy”) technology has reg_diff_source="energy" and the “base_year_reference_region_cost” is not empty, then the “base_year_reference_region_cost” in tech_map_[module].csv is used as the base year cost for the technology in the reference region. If the “base_year_reference_region_cost” is empty, then the cost reduction across scenarios is the same as the cost reduction of the mapped technology.

  • If using the “materials” module, if a technology that is specified in tech_map_materials.csv already exists in tech_map_energy.csv, then the reference region cost is taken from tech_map_materials.csv.

  • If a technology in a module is not mapped to any source of regional differentiation, then no cost reduction over the years is applied to the technology.

  • If a technology has a non-empty “base_year_reference_region_cost” but is not mapped to any source of regional differentiation, then assume no regional differentiation and use the reference region base year cost as the base year cost for all regions.

Data sources

The tool uses the following data sources for the regional differentiation of costs:

  • WEO: the World Energy Outlook data from the International Energy Agency (IEA).

  • Intratec: the Intratec data, which is a database of production costs for chemicals and other materials.

The tool also uses ssp.data (via exo_data.prepare_computer()) to adjust the costs of technologies based on GDP per capita.

Usage

create_cost_projections() is the top-level entry point.

This function takes a single costs.Config object as an argument. The object carries all the settings understood by create_cost_projections() and other functions. Those settings include the following; click each for the full description, allowable values, and defaults:

create_cost_projections() in turn calls the other functions in the module in the correct order, and returns a Python dict with the following keys mapped to pandas.DataFrame.

  • “inv_cost”: the investment costs of the technologies in each region.

  • “fix_cost”: the fixed O&M costs of the technologies in each region.

To use the tool with the default settings, simply create a Config object and pass it as an argument to create_cost_projections():

from message_ix_models.tools.costs import Config, create_cost_projections

# Use default settings
cfg = Config()

# Compute cost projections
costs = create_cost_projections(cfg)

# Show the resulting data
costs["inv_cost"]
costs["fix_cost"]

These data can be further manipulated; for instance, added to a scenario using add_par_data(). See the file message_ix_models/tools/costs/demo.py for multiple examples using various non-default settings to control the methods and data used by create_cost_projections().

Code reference

The top-level function and configuration class:

Config(base_year, convergence_year, ...)

Configuration for costs.

create_cost_projections(config)

Get investment and fixed cost projections.

The other submodules implement the supporting methods, calculations, and data handling, in roughly the following order:

  1. regional_differentiation calculates the regional differentiation of costs for technologies.

  2. decay projects the costs of technologies in a reference region with only a cost reduction rate applied.

  3. gdp adjusts the regional differentiation of costs for technologies based on the GDP per capita of the region.

  4. projections combines all the above steps and returns the projected costs for each technology in each region.

class message_ix_models.tools.costs.Config(base_year: int = 2025, convergence_year: int = 2050, final_year: int = 2110, fom_rate: float = 0, format: ~typing.Literal['iamc', 'message'] = 'message', node: ~typing.Literal['R11', 'R12', 'R20'] = 'R12', method: ~typing.Literal['constant', 'convergence', 'gdp'] = 'gdp', module: ~typing.Literal['energy', 'materials'] = 'energy', use_vintages: bool = False, pre_last_year_rate: float = 0.01, ref_region: str | None = None, scenario_version: ~typing.Literal['original', 'updated', 'all'] = 'updated', scenario: ~typing.Literal['all', 'LED', 'SSP1', 'SSP2', 'SSP3', 'SSP4', 'SSP5'] = 'all', _info: ~message_ix_models.util.scenarioinfo.ScenarioInfo = <factory>)[source]

Configuration for costs.

On creation:

  • If not given, ref_region is set based on node using, for instance, ref_region="R12_NAM" for node="R12".

property Y: List[int]

List of model periods.

base_year: int = 2025

Base year for projected costs. This is the first year for which cost reductions/decay are calculated. If the base year is greater than y0 (first model year), then the costs are assumed to be the same from y0 to base_year.

check()[source]

Validate settings.

convergence_year: int = 2050

Year of convergence; used when method is “convergence”. This is the year by which costs in all regions should converge to the reference region’s costs. See create_projections_converge().

final_year: int = 2110

Final year for projections. Note that the default is the same as the final model year of 2110 commonly used in MESSAGEix-GLOBIOM (Years or time periods (year/*.yaml)).

fom_rate: float = 0

Rate of exponential growth (positive values) or decrease of fixed operating and maintenance costs over time. The default of 0 implies no change over time. If the rate is 0.025, for example, that implies exponential growth at a rate of 2.5% per year; or (1 + 0.025) ** N for a period of length N.

format: Literal['iamc', 'message'] = 'message'

Format of output from create_cost_projections(). One of:

  • “iamc”: IAMC time series data structure.

  • “message”: message_ix parameter data.

method: Literal['constant', 'convergence', 'gdp'] = 'gdp'

Method for projecting costs in non-reference regions. One of:

module: Literal['energy', 'materials'] = 'energy'

Model variant for which to project costs.

node: Literal['R11', 'R12', 'R20'] = 'R12'

Node code list / spatial resolution for which to project costs. This should correspond to the target scenario to which data is to be added.

pre_last_year_rate: float = 0.01

Todo

Document the meaning of this setting.

ref_region: str | None = None

Reference region. If not given, "{node}_NAM"` for a given node. This default must be overridden if there is no such node.

scenario: Literal['all', 'LED', 'SSP1', 'SSP2', 'SSP3', 'SSP4', 'SSP5'] = 'all'

Scenario(s) for which to project costs. “all” implies the set of all the other values, meaning that costs are projected for all scenarios.

scenario_version: Literal['original', 'updated', 'all'] = 'updated'

Set of SSPs referenced by scenario. One of:

  • “original”: SSP_2017

  • “updated”: SSP_2024

  • “all”: both of the above.

property seq_years: List[int]

Similar to Y.

This list of periods differs in that it:

  1. Excludes periods after final_year.

  2. Includes 5-year periods even when these are not in Y.

use_vintages: bool = False

Use vintages.

If True, for each vintage, the fixed O&M costs will be calculated as a ratio of the investment costs, that decreases by the rate of fom_rate. If False, the fix_cost is the ratio of the investment cost for each year_act. In this case, the fix_cost is the same for all vintages of the same year_act.

property y0: int

The first model period.

message_ix_models.tools.costs.create_cost_projections(config: Config) Mapping[str, DataFrame][source]

Get investment and fixed cost projections.

This is the main function to get investment and fixed cost projections. It calls the other functions in this module, and returns the projections in the specified format.

Parameters:

config (Config) – The function responds to, or passes on to other functions, the fields: base_year, convergence_year, fom_rate, format, method, module, node, ref_region, scenario, and scenario_version.

Returns:

Keys are “fix_cost” and “inv_cost”, each mapped to a DataFrame.

If Config.format is “message”, the data frames have the same columns as required by message_ix for the respective parameter—for instance, the columns given by make_df("fix_cost", ...)plus columns named “scenario” and “scenario_version”.

Return type:

dict

Cost reduction of technologies over time (decay)

get_cost_reduction_data(module)

Get cost reduction data from file.

get_technology_reduction_scenarios_data(...)

Read in technology first year and cost reduction scenarios.

project_ref_region_inv_costs_using_reduction_rates(...)

Project investment costs for the reference region using cost reduction rates.

message_ix_models.tools.costs.decay.get_cost_reduction_data(module) DataFrame[source]

Get cost reduction data from file.

Raw data on cost reduction in 2100 for technologies are read from data/[module]/cost_reduction_[module].csv, based on GEA data.

Parameters:

module (str) – Model module

Returns:

DataFrame with columns:

  • message_technology: name of technology in MESSAGEix

  • reduction_rate: the cost reduction rate (either very_low, low, medium, high,

or very_high) - cost_reduction: cost reduction in 2100 (%)

Return type:

pandas.DataFrame

message_ix_models.tools.costs.decay.get_technology_reduction_scenarios_data(first_year: int, module: str) DataFrame[source]

Read in technology first year and cost reduction scenarios.

Raw data on technology first year and reduction scenarios are read from data/costs/[module]/first_year_[module]. The first year the technology is available in MESSAGEix is adjusted to be the base year if the original first year is before the base year.

Raw data on cost reduction scenarios are read from data/costs/[module]/scenarios_reduction_[module].csv.

Assumptions are made for the materials module for technologies’ cost reduction scenarios that are not given.

Parameters:
  • base_year (int, optional) – The base year, by default set to global BASE_YEAR

  • module (str) – Model module

Returns:

DataFrame with columns:

  • message_technology: name of technology in MESSAGEix

  • scenario: scenario (SSP1, SSP2, SSP3, SSP4, SSP5, or LED)

  • first_technology_year: first year the technology is available in MESSAGEix.

  • reduction_rate: the cost reduction rate (either very_low, low, medium, high,

or very_high)

Return type:

pandas.DataFrame

message_ix_models.tools.costs.decay.project_ref_region_inv_costs_using_reduction_rates(regional_diff_df: DataFrame, config: Config) DataFrame[source]

Project investment costs for the reference region using cost reduction rates.

This function uses the cost reduction rates for each technology under each scenario to project the capital costs for each technology in the reference region.

The returned data have the list of periods given by Config.seq_years.

Parameters:
Returns:

DataFrame with columns:

  • message_technology: name of technology in MESSAGEix

  • scenario: scenario (SSP1, SSP2, SSP3, SSP4, SSP5, or LED)

  • reference_region: reference region

  • first_technology_year: first year the technology is available in MESSAGEix.

  • year: year

  • inv_cost_ref_region_decay: investment cost in reference region in year.

Return type:

pandas.DataFrame

GDP-adjusted costs and regional differentiation (gdp)

process_raw_ssp_data(context, config)

Retrieve SSP data as required for tools.costs.

adjust_cost_ratios_with_gdp(region_diff_df, ...)

Calculate adjusted region-differentiated cost ratios.

message_ix_models.tools.costs.gdp.adjust_cost_ratios_with_gdp(region_diff_df, config: Config)[source]

Calculate adjusted region-differentiated cost ratios.

This function takes in a data frame with region-differentiated cost ratios and calculates adjusted region-differentiated cost ratios using GDP per capita data.

Parameters:
Returns:

DataFrame with columns:

  • scenario_version: scenario version

  • scenario: SSP scenario

  • message_technology: message technology

  • region: R11, R12, or R20 region

  • year

  • gdp_ratio_reg_to_reference: ratio of GDP per capita in respective region to GDP per capita in reference region.

  • reg_cost_ratio_adj: adjusted region-differentiated cost ratio

Return type:

pandas.DataFrame

message_ix_models.tools.costs.gdp.process_raw_ssp_data(context: Context, config: Config) DataFrame[source]

Retrieve SSP data as required for tools.costs.

This method uses SSPOriginal and SSPUpdate via exo_data.prepare_computer()

Returns:

with the columns:

  • scenario_version

  • scenario

  • region

  • year

  • total_gdp

  • total_population

  • gdp_ppp_per_capita

  • gdp_ratio_reg_to_reference

Return type:

pandas.DataFrame

Projection of costs given input parameters (projections)

create_projections_constant(config)

Create cost projections using assuming constant regional cost ratios.

create_projections_gdp(config)

Create cost projections using the GDP method.

create_projections_converge(config)

Create cost projections using the convergence method.

create_message_outputs(df_projections, config)

Create MESSAGEix outputs for investment and fixed costs.

create_iamc_outputs(msg_inv, msg_fix)

Create IAMC outputs for investment and fixed costs.

message_ix_models.tools.costs.projections.create_cost_projections(config: Config) Mapping[str, DataFrame][source]

Get investment and fixed cost projections.

This is the main function to get investment and fixed cost projections. It calls the other functions in this module, and returns the projections in the specified format.

Parameters:

config (Config) – The function responds to, or passes on to other functions, the fields: base_year, convergence_year, fom_rate, format, method, module, node, ref_region, scenario, and scenario_version.

Returns:

Keys are “fix_cost” and “inv_cost”, each mapped to a DataFrame.

If Config.format is “message”, the data frames have the same columns as required by message_ix for the respective parameter—for instance, the columns given by make_df("fix_cost", ...)plus columns named “scenario” and “scenario_version”.

Return type:

dict

message_ix_models.tools.costs.projections.create_iamc_outputs(msg_inv: DataFrame, msg_fix: DataFrame) Tuple[DataFrame, DataFrame][source]

Create IAMC outputs for investment and fixed costs.

Parameters:
  • msg_inv (pd.DataFrame) – Dataframe containing investment costs in MESSAGEix format. Output of func:create_message_outputs.

  • msg_fix (pd.DataFrame) – Dataframe containing fixed operating and maintenance costs in MESSAGEix format. Output of func:create_message_outputs.

Returns:

  • iamc_inv (pd.DataFrame) – Dataframe containing investment costs in IAMC format.

  • iamc_fix (pd.DataFrame) – Dataframe containing fixed operating and maintenance costs in IAMC format.

message_ix_models.tools.costs.projections.create_message_outputs(df_projections: DataFrame, config: Config) Tuple[DataFrame, DataFrame][source]

Create MESSAGEix outputs for investment and fixed costs.

The returned data have the model periods given by Config.Y.

Parameters:
  • df_projections (pd.DataFrame) – Dataframe containing the cost projections for each technology. Output of func:create_cost_projections.

  • config (Config) – The function responds to the fields fom_rate and Y.

Returns:

  • inv (pd.DataFrame) – Dataframe containing investment costs.

  • fom (pd.DataFrame) – Dataframe containing fixed operating and maintenance costs.

message_ix_models.tools.costs.projections.create_projections_constant(config: Config)[source]

Create cost projections using assuming constant regional cost ratios.

Parameters:

config (Config) – The function responds to, or passes on to other functions, the fields: base_year, module, node, ref_region, and scenario.

Returns:

df_costs – Dataframe containing the cost projections with the columns: - scenario_version: scenario version (for constant method, only

”Not applicable”)

  • scenario: scenario name (SSP1, SSP2, SSP3, SSP4, SSP5, or LED)

  • message_technology: technology name

  • region: region name

  • year: year

  • inv_cost: investment cost

  • fix_cost: fixed operating and maintenance cost

Return type:

pd.DataFrame

message_ix_models.tools.costs.projections.create_projections_converge(config: Config)[source]

Create cost projections using the convergence method.

Parameters:

config (Config) – The function responds to, or passes on to other functions, the fields: base_year, convergence_year, module, node, ref_region, and scenario.

Returns:

df_costs – Dataframe containing the cost projections with the columns: - scenario_version: scenario version (for convergence method, only “Not

applicable”)

  • scenario: scenario name (SSP1, SSP2, SSP3, SSP4, SSP5, or LED)

  • message_technology: technology name

  • region: region name

  • year: year

  • inv_cost: investment cost

  • fix_cost: fixed operating and maintenance cost

Return type:

pd.DataFrame

message_ix_models.tools.costs.projections.create_projections_gdp(config: Config)[source]

Create cost projections using the GDP method.

Parameters:

config (Config) – The function responds to, or passes on to other functions, the fields: base_year, module, node, ref_region, scenario, and scenario_version.

Returns:

df_costs – Dataframe containing the cost projections with the columns: - scenario_version: scenario version (for gdp method, either “Review (2023)” or

”Previous (2013)”

  • scenario: scenario name (SSP1, SSP2, SSP3, SSP4, SSP5, or LED)

  • message_technology: technology name

  • region: region name

  • year: year

  • inv_cost: investment cost

  • fix_cost: fixed operating and maintenance cost

Return type:

pd.DataFrame

Regional differentiation of costs (regional_differentiation)

get_weo_data()

Read in raw WEO investment/capital costs and O&M costs data.

get_intratec_data()

Read in raw Intratec data.

get_raw_technology_mapping(module)

Retrieve a technology mapping for module.

subset_materials_map(raw_map)

Subset materials mapping for only technologies that have sufficient data.

adjust_technology_mapping(module)

Adjust technology mapping based on sources and assumptions.

get_weo_regional_differentiation(config)

Apply WEO regional differentiation.

get_intratec_regional_differentiation(node, ...)

Apply Intratec regional differentiation.

apply_regional_differentiation(config)

Apply regional differentiation depending on mapping source.

message_ix_models.tools.costs.regional_differentiation.adjust_technology_mapping(module: Literal['energy', 'materials']) DataFrame[source]

Adjust technology mapping based on sources and assumptions.

Parameters:

module (str) – See Config.module.

Returns:

DataFrame with columns:

  • message_technology: MESSAGEix technology name.

  • reg_diff_source: data source to map MESSAGEix technology to (e.g., WEO, Intratec).

  • reg_diff_technology: technology name in the data source.

  • base_year_reference_region_cost: manually specified base year cost of the technology in the reference region (in 2005 USD).

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.apply_regional_differentiation(config: Config) DataFrame[source]

Apply regional differentiation depending on mapping source.

  1. Retrieve an adjusted technology mapping from adjust_technology_mapping().

  2. Based on the value in the reg_diff_source column:

Parameters:

config (Config) – The function responds to, or passes on to other functions, the fields: module, node, and ref_region.

Returns:

DataFrame with columns:

  • message_technology: MESSAGEix technology name

  • reg_diff_source: data source to map MESSAGEix technology to (e.g., WEO, Intratec)

  • reg_diff_technology: technology name in the data source

  • region: MESSAGEix region

  • base_year_reference_region_cost: manually specified base year cost of the technology in the reference region (in 2005 USD)

  • reg_cost_ratio: regional cost ratio relative to reference region

  • fix_ratio: ratio of fixed O&M costs to investment costs

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.get_intratec_data() DataFrame[source]

Read in raw Intratec data.

Returns:

DataFrame with columns:

  • node: Intratec region

  • value: Intratec index value

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.get_intratec_regional_differentiation(node: str, ref_region: str) DataFrame[source]

Apply Intratec regional differentiation.

  1. Retrieve Intratec data using get_intratec_data().

  2. Map data to MESSAGEix-GLOBIOM regions according to the Config.node.

  3. Calculate cost ratios for each region relative to the ref_region.

Parameters:
  • node (str) – See :attr`.Config.node`.

  • ref_region (str) – See :attr`.Config.ref_region`.

Returns:

DataFrame with columns:

  • message_technology: MESSAGEix technology name

  • region: MESSAGEix region

  • intratec_ref_region_cost: Intratec cost in reference region

  • reg_cost_ratio: regional cost ratio relative to reference region

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.get_raw_technology_mapping(module: Literal['energy', 'materials']) DataFrame[source]

Retrieve a technology mapping for module.

The data are read from a CSV file at data/module/tech_map_module.csv. The file must have the following columns:

  • message_technology: MESSAGEix-GLOBIOM technology code

  • reg_diff_source: data source to map MESSAGEix technology to. A string like “weo”, “energy”, or possibly others.

  • reg_diff_technology: Technology code in the source data.

  • base_year_reference_region_cost: manually specified base year cost of the technology in the reference region (in 2005 USD).

  • fix_ratio: ???

Parameters:

module (str) – See Config.module.

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.get_weo_data() DataFrame[source]

Read in raw WEO investment/capital costs and O&M costs data.

Returns:

DataFrame with columns:

  • cost_type: investment or fixed O&M cost

  • weo_technology: WEO technology name

  • weo_region: WEO region

  • year: year

  • value: cost value

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.get_weo_region_map(regions: str) Mapping[str, str][source]

Return a mapping from MESSAGE node IDs to WEO region names.

The mapping is constructed from the iea-weo-region annotations on the Node code lists.

message_ix_models.tools.costs.regional_differentiation.get_weo_regional_differentiation(config: Config) DataFrame[source]

Apply WEO regional differentiation.

  1. Retrieve WEO data using get_weo_data().

  2. Map data to MESSAGEix-GLOBIOM regions according to the Config.node.

  3. Calculate cost ratios for each region relative to the ref_region.

Parameters:

config (Config) – The function responds to the fields: base_year, node, and ref_region.

Returns:

DataFrame with columns:

  • message_technology: MESSAGEix technology name

  • region: MESSAGEix region

  • weo_ref_region_cost: WEO cost in reference region

  • reg_cost_ratio: regional cost ratio relative to reference region

Return type:

pandas.DataFrame

message_ix_models.tools.costs.regional_differentiation.subset_materials_map(raw_map)[source]

Subset materials mapping for only technologies that have sufficient data.

Parameters:

raw_map (pandas.DataFrame) – Output of get_raw_technology_mapping()

Returns:

DataFrame with columns:

  • message_technology: MESSAGEix technology name

  • reg_diff_source: data source to map MESSAGEix technology to (e.g., WEO)

  • reg_diff_technology: technology name in the data source

  • base_year_reference_region_cost: manually specified base year cost of the technology in the reference region (in 2005 USD)

Return type:

pandas.DataFrame