What’s New

v3.9.0 (2024-06-04)

All changes

  • Split installation instructions to a basic Quick install and detailed Advanced installation guide (PR #843).

  • Ensure compatibility with pandas upcoming 3.0 Copy-on-Write behaviour (PR #842).

  • Improve tutorial Westeros baseline for correct lifetime, units, and vintage-activity years (PR #815).

  • Update tutorial Westeros multinode to include code-based hints for in-depth questions (PR #798).

  • make_df() can now create partly-filled DataFrames for indexed sets; not only parameters (PR #784).

  • New function util.copy_model() that exposes the behaviour of the message-ix copy-model CLI command to other Python code (PR #784).

  • New test fixture tmp_model_dir() (PR #784).

  • Bug fix: Scenario.rename() would not rename keys where the index set and index name differed (#601, PR #791).

  • Increase minimum requirement for genno dependency to 1.20 (PR #783).

v3.8.0 (2024-01-12)

Migration notes

Update code that imports from the following modules:

Code that imports from the old locations will continue to work, but will raise DeprecationWarning.

All changes

v3.7.0 (2023-05-16)

Migration notes

  • The default lpmethod has been changed from “Dual Simplex” (lpmethod=2) to “Barrier” (lpmethod=4). NOTE: this may result in changes to the solution. In order to use the previous default lpmethod, the user-specific default setting can be set through the user’s ixmp configuration file. Alternatively, the lpmethod can be specified directly as an argument when solving a scenario. Both of these configuration methods are further explained here.

  • The dimensionality of one set and two parameters (map_tec_storage, storage_initial, and storage_self_discharge) are extended to allow repesentation of the mode of operation of storage technologies and the temporal level of storage containers. If these items are already populated with data in a Scenario, this data will be incompatible with the MESSAGE GAMS implementation in this release; a UserWarning will be emitted when the Scenario is instantiated, and solve() will raise a ValueError. (If these items are empty, their dimensions will be updated automatically. New Scenarios are unaffected.)

    Users must update data for these items, specifically:

    Existing parameter or set

    Dimension(s) to add

    map_tec_storage

    mode, storage_mode, lvl_temporal

    storage_initial

    mode

    storage_self_discharge

    mode

    For the set map_tec_storage, values for the new dimensions represent, respectively, the mode of operation for charge/discharge technologies, and the storage_mode and lvl_temporal for the corresponding storage device. For the two parameters, expand_dims() is provided to help:

    from message_ix import Scenario
from message_ix.util import expand_dims

scen, platform = Scenario.from_url("…")

# Re-use the existing data in `scen`, adding the `mode` dimension
expand_dims(scen, "storage_initial", mode="an existing mode")

All changes

  • Add a tutorial for Westeros multi-node and different trade possibilities (PR #683).

  • Add additional oscillation detection mechanism for macro iterations (PR #645, PR #676)

  • Adjust default lpmethod from “Dual Simplex” (2) to “Barrier” (4); do NOT remove cplex.opt file(s) after solving workflow completes (PR #657).

  • Adjust Scenario.add_macro() calculations for pandas 1.5.0 (PR #656).

  • Ensure levelized_cost are also calculated for technologies with only variable costs (PR #653).

  • Correct calculation of COST_NODAL_NET for standalone MESSAGE (PR #648)

  • Account for difference in period-length in equations NEW_CAPACITY_CONSTRAINT_LO and NEW_CAPACITY_CONSTRAINT_UP (PR #654)

  • Extend functionality of storage solutions to include “mode” and temporal level (PR #633)

  • Introduce a citation file CITATION.cff with citation information (PR #695).

  • Correct GAMS for the assignment of “capacity_factor” at “year” (PR #705).

v3.6.0 (2022-08-17)

Migration notes

  • The in_horizon argument to vintage_and_active_years() is deprecated, and will be removed in message_ix 4.0 or later. At the same time, the behaviour will change to be the equivalent of providing in_horizon = False, i.e. the method will no longer filter to the scenario time horizon by default. To prepare for this change, user code that expects values confined to the time horizon can be altered to use pandas.DataFrame.query():

    df = scen.vintage_and_active_years().query(f"{scen.y0} <= year_vtg")

  • The default reports (tables in IAMC format) available in a Reporter have changed keys to e.g. message::default with two colons. Code using e.g. message:default (one colon) should be updated to use the current keys.

    This matches fixed behaviour upstream in genno version 1.12 to avoid unintended confusion with keys like A:i: i (after the first colon) is the name for the sole dimension of a 1-dimensional quantity, whereas default in message::default is a tag.

All changes

v3.5.0 (2022-05-06)

Migration notes

The format of input data files for MACRO calibration has been changed in PR #327. Files compatible with v3.4.0 and earlier will not work with this version and should be updated; see details of the current data file format in the documentation.

PR #561 corrected the model internal logic for handling zero values in the capacity_factor parameter. Before this change, the GAMS code inserted a capacity_factor value of 1.0 where such zero values appeared; now, zeros are preserved, so the technologies may be created (CAP) but none of their capacity will be usable at the \((n, t, y^V, y, h)\) where zero values appear. This is consistent with the general concept of a “capacity factor”: for instance, a solar photovoltaic technology for electricity generation may have a non-zero capacity with a capacity factor of 0 at \(h=\text{night}\). This may cause changes in model output for scenarios where such zero values appear; see #591 for discussion, including methods to check for and adjust/remove such values.

All changes

v3.4.0 (2022-01-27)

All changes

  • Expand the documentation with an outlook of the MESSAGEix usage (PR #520).

  • Adjust test suite for pyam v1.1.0 compatibility (PR #499).

  • Add Westeros tutorial on historical parameters (PR #478).

  • Update reference for activity and capacity soft constraints (PR #474).

  • Update years_active() to use sorted results (PR #491).

  • Adjust the Westeros reporting tutorial to pyam 1.0 deprecations (PR #492).

  • Change precision of GAMS check for parameter “duration_time” (PR #513).

  • Update light and historic demand in Westeros baseline tutorial (PR #523).

  • Enhance mathematical formulation to represent sub-annual time slices consistently (PR #509).

v3.3.0 (2021-05-28)

Migration notes

rmessageix (and rixmp) are deprecated and removed, as newer versions of the R reticulate package allow direct import and use of the Python modules with full functionality. See the updated page for Usage in R via reticulate, and the updated instructions on how to Install R and reticulate.

All changes

v3.2.0 (2021-01-24)

Migration notes

  • Code that uses make_df() can be adjusted in one of two ways. See the function documentation for details. The function should be imported from the top level:

    from message_ix import make_df

All changes

v3.1.0 (2020-08-28)

message_ix v3.1.0 coincides with ixmp v3.1.0.

For citing message_ix, distinct digital object identifiers (DOIs) are available for every release from v3.1.0 onwards; see the user guidelines and notice for more information and how to cite.

All changes

v3.0.0 (2020-06-07)

message_ix v3.0.0 coincides with ixmp v3.0.0.

Migration notes

The generic storage formulation introduces new ixmp items (sets, parameters, variables, and equations) to the MESSAGE model scheme. When loading a Scenario created with a version of message_ix older than 3.0.0, MESSAGE.initialize() will initialized these items (and leave them empty), using at most one call to commit().

See also the migration notes for ixmp 3.0.0.

All changes

v2.0.0 (2020-01-14)

message_ix v2.0.0 coincides with ixmp v2.0.0.

Migration notes

Support for Python 2.7 is dropped as it has reached end-of-life, meaning no further releases will be made even to fix bugs. See PEP-0373 and https://python3statement.org. message_ix users must upgrade to Python 3.

Command-line interface (CLI). Use message-ix as the program for all command-line operations:

  • message-ix copy-model replaces messageix-config.

  • message-ix dl replaces messageix-dl.

  • message-ix also provides all the features of the ixmp CLI.

Configuration. ixmp adds a streamlined system for storing information about different platforms, backends, and databases that store Scenario data. See the ixmp release notes for migration notes.

All changes

v1.2.0 (2019-06-25)

MESSAGEix 1.2.0 adds an option to set the commodity balance to strict equality, rather than a supply > demand inequality. It also improves the support for models with non-equidistant years.

Other improvements include an experimental reporting module, support for CPLEX solver options via solve(), and a reusable message_ix.testing module.

Release 1.2.0 coincides with ixmp release 0.2.0, which provides full support for clone() across platforms (database instances), e.g. from a remote database to a local HSQL database; as well as other improvements. See the ixmp release notes for further details.

All changes

  • PR #161: A feature for adding new periods to a scenario.

  • PR #205: Implement required changes related to timeseries-support and cloning across platforms (see ixmp#142).

  • PR #196: Improve testing by re-using ixmp apparatus.

  • PR #187: Test for cumulative bound on emissions.

  • PR #182: Fix cross-platform cloning.

  • PR #178: Bugfix of the PRICE_EMISSION variable in models with non-equidistant period durations.

  • PR #176: Add message_ix.report module.

  • PR #173: The meth:~.Scenario.solve command now takes additional arguments when solving with CPLEX. The cplex.opt file is now generated on the fly during the solve command and removed after successfully solving.

  • PR #172: Add option to set COMMODITY_BALANCE to equality.

  • PR #154: Enable documentation build on ReadTheDocs.

  • PR #138: Update documentation and tutorials.

  • PR #131: Update clone function argument scen to scenario with planned deprecation of the former.

v1.1.0 (2018-11-21)

Migration notes

This patch introduces a few backwards-incompatible changes to database management.

Database Migration

If you see an error message like:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
usr/local/lib/python2.7/site-packages/ixmp/core.py:81: in __init__
    self._jobj = java.ixmp.Platform("Python", dbprops)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <jpype._jclass.at.ac.iiasa.ixmp.Platform object at 0x7ff1a8e98410>
args = ('Python', '/tmp/kH07wz/test.properties')

    def _javaInit(self, *args):
        object.__init__(self)

        if len(args) == 1 and isinstance(args[0], tuple) \
           and args[0][0] is _SPECIAL_CONSTRUCTOR_KEY:
            self.__javaobject__ = args[0][1]
        else:
            self.__javaobject__ = self.__class__.__javaclass__.newClassInstance(
>               *args)
E           org.flywaydb.core.api.FlywayExceptionPyRaisable: org.flywaydb.core.api.FlywayException: Validate failed: Migration checksum mismatch for migration 1
E           -> Applied to database : 1588531206
E           -> Resolved locally    : 822227094

Then you need to update your local database. There are two methods to do so:

  1. Delete it (you will lose all data and need to regenerate it). The default location is ~/.local/ixmp/localdb/.

  2. Manually apply the underlying migrations. This is not particularly easy, but allows you to save all your data. If you want help, feel free to get in contact on the listserv.

New Property File Layout

If you see an error message like:

usr/local/lib/python2.7/site-packages/jpype/_jclass.py:111: at.ac.iiasa.ixmp.exceptions.IxExceptionPyRaisable
---------------------------- Captured stdout setup -----------------------------
2018-11-13 08:15:17,410 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.config' in /tmp/hhvE1o/test.properties
2018-11-13 08:15:17,412 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.password' in /tmp/hhvE1o/test.properties
2018-11-13 08:15:17,412 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.username' in /tmp/hhvE1o/test.properties
2018-11-13 08:15:17,413 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.url' in /tmp/hhvE1o/test.properties
------------------------------ Captured log setup ------------------------------
core.py                     80 INFO     launching ixmp.Platform using config file at '/tmp/hhvE1o/test.properties'
_________________ ERROR at setup of test_add_spatial_multiple __________________

    @pytest.fixture(scope="session")
    def test_mp():
        test_props = create_local_testdb()

        # start jvm
        ixmp.start_jvm()

        # launch Platform and connect to testdb (reconnect if closed)
>       mp = ixmp.Platform(test_props)

Then you need to update your property configuration file. The old file looks like:

config.name = message_ix_test_db@local
jdbc.driver.1 = org.hsqldb.jdbcDriver
jdbc.url.1 = jdbc:hsqldb:file:/path/to/database
jdbc.user.1 = ixmp
jdbc.pwd.1 = ixmp
jdbc.driver.2 = org.hsqldb.jdbcDriver
jdbc.url.2 = jdbc:hsqldb:file:/path/to/database
jdbc.user.2 = ixmp
jdbc.pwd.2 = ixmp

The new file should look like:

config.name = message_ix_test_db@local
jdbc.driver = org.hsqldb.jdbcDriver
jdbc.url = jdbc:hsqldb:file:/path/to/database
jdbc.user = ixmp
jdbc.pwd = ixmp

All changes

  • PR #202: Added the “Development rule of thumb” section from the wiki and the Tutorial style guide to the Contributor guidelines. Tweaked some formatting to improve readibility.

  • PR #113: Upgrading to MESSAGEix 1.1: improved representation of renewables, share constraints, etc.

  • PR #109: MACRO module added for initializing models to be solved with MACRO. Added scenario-based CI on circleci.

  • PR #99: Fixing an error in the compuation of the auxiliary GAMS reporting variable PRICE_EMISSION.

  • PR #89: Fully implementing system reliability and flexibity considerations (cf. Sullivan).

  • PR #88: Reformulated capacity maintainance constraint to ensure that newly installed capacity cannot be decommissioned within the same model period as it is built in.

  • PR #84: message_ix.Scenario.vintage_active_years() now limits active years to those after the first model year or the years of a certain technology vintage.

  • PR #82: Introducing “add-on technologies” for mitigation options, etc.

  • PR #81: Share constraints by mode added.

  • PR #80: Share constraints by commodity/level added.

  • PR #78: Bugfix: message_ix.Scenario.solve() uses ‘MESSAGE’ by default, but can be provided other model names.

  • PR #77: rename() function can optionally keep old values in the model (i.e., copy vs. copy-with-replace).

  • PR #74: Activity upper and lower bounds can now be applied to all modes of a technology.

  • PR #67: Use of advanced basis in cplex.opt turned off by default to avoid conflicts with barrier method.

  • PR #65: Bugfix for downloading tutorials. Now downloads current installed version by default.

  • PR #60: Add basic ability to write and read model input to/from Excel.

  • PR #59: Added MacOSX CI support.