Add model years to an existing Scenario

Description

This tool adds new modeling years to an existing message_ix.Scenario (hereafter “reference scenario”). For instance, in a scenario define with:

history = [690]
model_horizon = [700, 710, 720]
sc_ref.add_horizon(
    year=history + model_horizon,
    firstmodelyear=model_horizon[0]
)

…additional years can be added after importing the add_year function:

from message_ix.tools.add_year import add_year
sc_new = message_ix.Scenario(mp, sc_ref.model, sc_ref.scenario,
                             version='new')
add_year(sc_ref, sc_new, [705, 712, 718, 725])

At this point, sc_new will have the years [700, 705, 710, 712, 718, 720, 725], and original or interpolated data for all these years in all parameters.

The tool operates by creating a new empty Scenario (hereafter “new scenario”) and:

  • Copying all sets from the reference scenario, adding new time steps to relevant sets (e.g., adding 2025 between 2020 and 2030 in the set year)

  • Copying all parameters from the reference scenario, adding new years to relevant parameters, and calculating missing values for the added years.

Features

  • It can be used for any MESSAGE scenario, from tutorials, country-level, and global models.

  • The new years can be consecutive, between existing years, and/or after the model horizon.

  • The user can define for what regions and parameters the new years should be added. This saves time when adding the new years to only one parameter of the reference scenario, when other parameters have previously been successfully added to the new scenario.

Usage

The tool can be used either:

  1. Directly from the command line:

    $ message-ix \
    --platform default
    --model MESSAGE_Model \
    --scenario baseline \
    add-years
    --years_new 2015,2025,2035,2045

    For the full list of input arguments, run:

    $ message-ix add-years --help

  2. By calling the function add_year() from a Python script.

Technical details

  1. An existing scenario is loaded and the desired new years are specified.

  2. A new (empty) scenario is created for adding the new years.

  3. The new years are added to the relevant sets, year and type_year.

    • The sets firstmodelyear, lastmodelyear, baseyear_macro, and initializeyear_macro are modified, if needed.

    • The set cat_year is modified for the new years.

  4. The new years are added to the index sets of relevant parameters, and the missing data for the new years are calculated based on interpolation of adjacent data points. The following steps are applied:

    1. Each non-empty parameter is loaded from the reference scenario.

    2. The year-related indexes (0, 1, or 2) of the parameter are identified.

    3. The new years are added to the parameter, and the missing data is calculated based on the number of year-related indexes. For example:

      • The parameter inv_cost has index year_vtg, to which the new years are added.

      • The parameter output has indices year_act and year_vtg. The new years are added to both of these dimensions.

    4. Missing data is calculated by interpolation.

    5. For parameters with 2 year-related indices (e.g. output), a final check is applied so ensure that the vintaging is correct. This step is done based on the lifetime of each technology.

  5. The changes are committed and saved to the new scenario.

Warning

The tool does not ensure that the new scenario will solve after adding the new years. The user needs to load the new scenario, check some key parameters (like bounds) and solve the new scenario.

API reference

Add model years to an existing Scenario.

message_ix.tools.add_year.add_year(sc_ref: Scenario, sc_new: Scenario, years_new: list[int], firstyear_new: int | None = None, lastyear_new: int | None = None, macro: bool = False, baseyear_macro: int | None = None, parameter: list[str] | Literal['all'] = 'all', region: list[str] | Literal['all'] = 'all', rewrite: bool = True, unit_check: bool = True, extrapol_neg: float | None = None, bound_extend: bool = True) → None[source]

Add years to sc_ref to produce sc_new.

add_year() does the following:

  1. calls add_year_set() to add and modify required sets.

  2. calls add_year_par() to add new years and modifications to each parameter if needed.

Parameters:

  • sc_ref (ixmp.Scenario) – Reference scenario.

  • sc_new (ixmp.Scenario) – New scenario.

  • yrs_new (list of int) – New years to be added.

  • firstyear_new (int, optional) – New first model year for new scenario.

  • macro (bool) – Add new years to parameters of the MACRO model.

  • baseyear_macro (int) – New base year for the MACRO model.

  • parameter (list of str or 'all') – Parameters for adding new years.

  • rewrite (bool) – Permit rewriting a parameter in new scenario when adding new years.

  • check_unit (bool) – Harmonize the units for each commodity, if there is inconsistency across model years.

  • extrapol_neg (float) – When extrapolation produces negative values, replace with a multiple of the value for the previous timestep.

  • bound_extend (bool) – Duplicate data from the previous timestep when there is only one data point for interpolation (e.g., permitting the extension of a bound to 2025, when there is only one value in 2020).

message_ix.tools.add_year.add_year_par(sc_ref: Scenario, sc_new: Scenario, yrs_new: list[int], parname: str, reg_list: list[str], firstyear_new: int, extrapolate: bool = False, rewrite: bool = True, unit_check: bool = True, extrapol_neg: float | None = None, bound_extend: bool = True) → None[source]

Add new years to parameters.

This function adds additional years to a parameter. The value of the parameter for additional years is calculated mainly by interpolating and extrapolating data from existing years.

See add_year() for parameter descriptions.

message_ix.tools.add_year.add_year_set(sc_ref: Scenario, sc_new: Scenario, years_new: list[int], firstyear_new: int | None = None, lastyear_new: int | None = None, baseyear_macro: int | None = None) → None[source]

Add new years to sets.

add_year_set() adds additional years to an existing scenario, by starting to make a new scenario from scratch. After modification of the year-related sets, all other sets are copied from sc_ref to sc_new.

See add_year() for parameter descriptions.

message_ix.tools.add_year.interpolate_1d(df: DataFrame, yrs_new: list[int], horizon: list[int], year_col: str, value_col: str = 'value', extrapolate: bool = False, extrapol_neg: float | None = None, bound_extend: bool = True)[source]

Interpolate data with one year dimension.

This function receives a parameter data as a dataframe, and adds new data for the additonal years by interpolation and extrapolation.

Parameters:

  • df (pandas.DataFrame) – The dataframe of the parameter to which new years to be added.

  • yrs_new (list of int) – New years to be added.

  • horizon (list of int) – The horizon of the reference scenario.

  • year_col (str) – The header of the column to which the new years should be added, e.g. ‘year_act’.

  • value_col (str) – The header of the column containing values.

  • extrapolate (bool) – Allow extrapolation when a new year is outside the parameter years.

  • extrapol_neg (bool) – Allow negative values obtained by extrapolation.

  • bound_extend (bool) – Allow extrapolation of bounds for new years

message_ix.tools.add_year.interpolate_2d(df: DataFrame, yrs_new: list[int], horizon: list[int], year_ref: str, year_col: str, tec_list: list[str], par_tec: DataFrame, value_col: str = 'value', extrapolate: bool = False, extrapol_neg: float | None = None, year_diff: list[int] | None = None, bound_extend: bool = True)[source]

Interpolate parameters with two dimensions related year.

This function receives a dataframe that has 2 time-related columns (e.g., “input” or “relation_activity”), and adds new data for the additonal years in both time-related columns by interpolation and extrapolation.

Parameters:

  • df (pandas.DataFrame) – The dataframe of the parameter to which new years to be added.

  • yrs_new (list of int) – New years to be added.

  • horizon (list of int) – The horizon of the reference scenario.

  • year_ref (str) – The header of the first column to which the new years should be added, e.g. ‘year_vtg’.

  • year_col (str) – The header of the column to which the new years should be added, e.g. ‘year_act’.

  • tec_list (list of str) – List of technologies in the parameter technical_lifetime.

  • par_tec (pandas.DataFrame) – Parameter technical_lifetime.

  • value_col (str) – The header of the column containing values.

  • extrapolate (bool) – Allow extrapolation when a new year is outside the parameter years.

  • extrapol_neg (bool) – Allow negative values obtained by extrapolation.

  • year_diff (list of int) – List of model years with different time intervals before and after them

  • bound_extend (bool) – Allow extrapolation of bounds for new years based on one data point

message_ix.tools.add_year.intpol(y1: float | Series | DataFrame, y2: float | Series | DataFrame, x1: int, x2: int, x: int) → float | Series | DataFrame[source]

Interpolate between (x1, y1) and (x2, y2) at x.

Parameters:
message_ix.tools.add_year.mask_df(df: DataFrame, index: str, count: int, value) → None[source]

Create a mask for removing extra values from df.

message_ix.tools.add_year.slice_df(df: DataFrame, idx: list[str], level: str, locator: list, value: int | str | None) → DataFrame[source]

Slice a MultiIndex DataFrame and set a value to a specific level.

Parameters:
message_ix.tools.add_year.unit_uniform(df: DataFrame) → DataFrame[source]

Make units in df uniform.