What’s new

Next release

• Incorporate the MESSAGEix-GLOBIOM global model documentation (PR #107, PR #110). This documentation formerly lived at https://docs.messageix.org/global/ and in a separate repository at iiasa/message_doc.

• Convert Intratec data for Investment and fixed costs (tools.costs) from Excel to a simpler CSV format. (PR #167).

• New module for Investment and fixed costs (tools.costs) (PR #99).

• Migrate ‘Legacy’ reporting (report.legacy) to provide post-processing functionality for the global model snapshot (PR #159).

• Migrate and improve code for four sources of exogenous data (PR #162): project.gea, project.shape, tools.gfei, and tools.iea.eei.

• Expand Data, metadata, and configuration (PR #161).

• Add an explicit Upstream version policy (PR #162).

v2024.4.2

• The SSPUpdate data provider pulls data from the SSP 2024 “Release 3.0” data files, and handles both the earlier and current structures (PR #156).

• Improve ExoDataSource with raise_on_extra_kw() utility method, automatic copy of source keyword arguments (PR #156).

• Expose node.nodes_ex_world() for use as a genno (reporting) operator.

• Raise DeprecationWarning from util.sdmx.eval_anno(); remove internal usage of this deprecated method (PR #156).

• Reduce verbosity when using the mix-models CLI when message_data is not installed (GH #37, PR #156).

• Improve logging (PR #156).

  • Use multi-threaded logging for better performance. Logging to stdout and file is on a separate thread and does not block operations on the main thread.

  • Add automatic file logging. Log versions of packages to file when using workflow.make_click_command().

  • New CLI command mix-models last-log to retrieve the location of the latest log file.

• Update Command-line interface (PR #156).

• Improve performance in disutility.data_conversion() (PR #156).

• Use platformdirs.user_cache_path() in more places; remove cache-path handling code (PR #156).

• Add util.datetime_now_with_tz() (PR #156).

• Add util.show_versions(), wrapping ixmp.util.show_versions() and returning its output as str (PR #156).

• util.private_data_path() returns an alternate, local data path if message_data is not installed (PR #156).

• Annotate c="transport" in the commodity code list with associated IEA (E)WEB flows (PR #153).

v2024.1.29

• Add (Extended) World Energy Balances (tools.iea.web) for handling data from the International Energy Agency (IEA) Extended World Energy Balances (GH #25, PR #75).

• Add World Bank structures (tools.wb) and assign_income_groups() to assign MESSAGE regions to World Bank income groups (PR #144).

• Adjust report.compat for genno version 1.22 (GH #141, PR #142).

• Raise informative exception from ScenarioInfo.io_units() (PR #151).

v2023.11.24

Migration notes

Update code that imports from the following modules:

• message_ix_models.report.computations → use message_ix_models.report.operator.

Code that imports from the old locations will continue to work, but will raise DeprecationWarning.

Data for MESSAGEix-Nexus (model.water) is no longer included in the PyPI distributions for message_ix_models. This reduces the package size from >20 MB to <5 MB. To automatically download and unpack these data into a local directory, use mix-models fetch MESSAGEix-Nexus.

All changes

• Improve ExoDataSource (PR #137):

  • New attributes name, extra_dims.

  • New method transform() that can be overridden by subclasses.

  • New arguments archive_member, non_iso_3166 to iamc_like_data_for_query().

• New provider for exogenous data from the ADVANCE project (PR #137). This module, project.advance, supersedes tools.advance and its idiosyncratic API, which are deprecated.

• New CLI commands (PR #137):

  • mix-models testing fuzz-private-data, superseding mix-models ssp make-test-data.

  • mix-models fetch, superseding mix-models snapshot fetch.

• New utility functions (PR #137).

  • tools.iamc.describe() to generate SDMX code lists that describe the structure of particular IAMC-format data (PR #137).

  • workflow.make_click_command() to generate click commands for any Workflow.

  • util.minimum_version() to ensure compatibility with upstream packages and aid test writing.

  • util.iter_keys() to generate keys for chains of genno computations.

• Add message_ix_models.report.compat for emulating legacy reporting (PR #134).

• Rename message_ix_models.report.operator (PR #137).

• Deprecate iter_parameters() in favour of ixmp.Scenario.par_list() with indexed_by=... argument from ixmp v3.8.0 (PR #137).

v2023.10.16

• New providers for exogenous data from the SSPOriginal and SSPUpdate (PR #125) sources.

• Improved ScenarioInfo (PR #125):

  • New attributes model, scenario, version, and (settable) url; class method from_url() to allow storing Scenario identifiers on ScenarioInfo objects.

  • New property path, giving a valid path name for scenario-specific file I/O.

• Improvements to report (PR #125):

  • New report.Config class collecting recognized settings for the module.

  • context["report"] always exists as an instance of report.Config.

  • New submodule report.plot with base class and 5 plots of time-series data stored on Scenarios.

  • Submodule report.sim provides add_simulated_solution() for testing reporting configuration.

  • New operator filter_ts().

• New reusable command-line option --urls-from-file in util.click (PR #125).

• Add pyarrow to dependencies (PR #125).

v2023.9.12

All changes

• New module project.ssp (PR #122) to generate SDMX codelists for the 2017/original SSPs and the 2024 update, and provide these as Enum to other code.

• New module tools.exo_data to retrieve exogenous data for, among others, population and GDP (PR #122). This module has a general API that can be implemented by provider classes.

• New function model.emissions.get_emission_factors() and associated data file to provide data from this table in the MESSAGEix-GLOBIOM documentation (PR #122).

• New functions in util.sdmx (PR #122):

  • read(), write() to retrieve/store package data in SDMX-ML.

  • make_enum() to make pure-Python Enum (or subclass) data structures based on SDMX code lists.

• same_node() also fills “node_shares”, “node_loc”, and “node”, as appropriate (PR #122).

Deprecations

• eval_anno() is deprecated; code should instead use sdmx.model.common.AnnotableArtefact.eval_annotation(), which provides the same functionality.

v2023.9.2

• New module message_ix_models.report for reporting (PR #116). Use of this module requires ixmp and message_ix version 3.6.0 or greater.

• Add documentation on Migrating individual modules using git filter-repo and helper scripts (PR #89).

v2023.7.26

• Add code and CLI commands to fetch and load MESSAGEix-GLOBIOM snapshots (PR #102). Use of this module requires ixmp and message_ix version 3.5.0 or greater.

• Add util.pooch.fetch(), a thin wrapper for using Pooch (PR #102).

• New module message_ix_models.model.macro with utilities for calibrating message_ix.macro (PR #104).

• New method Workflow.guess_target() (PR #104).

• Change in behaviour of Workflow.add_step(): the method now returns the name of the newly-added workflow step, rather than the WorkflowStep object added to carry out the step (PR #104). The former is more frequently used in code that uses Workflow.

• Add the 17-region aggregation (R17) node code list (PR #109).

• Add the 20-region aggregation (R20) node code list (PR #109).

v2023.5.31

• Adjust sdmx usage for version 2.10.0 (PR #101).

v2023.5.13

• Adjust generate_product() for pandas 2.0.0 (PR #98).

2023.4.2

• Add MESSAGEix-Nexus (model.water) (PR #88, PR #91).

• New utility function replace_par_data() (PR #90).

• disutility.get_spec() preserves all Annotations attached to the Code object used as a template for usage technologies (PR #90).

• Add CO2_Emission_Global_Total to the “A” relation codelist (PR #90).

• Adapter and MappingAdapter can be imported from message_ix_models.util (PR #90).

• Bump sdmx requirement from v2.2.0 to v2.8.0 (PR #90).

2023.2.8

• Codelists for the relation MESSAGEix set (PR #85):

  • Add three relation codelists.

  • The “bare” reference energy system now includes relations from the codelist indicated by model.Config.relations; default “A”.

• Commodities (commodity.yaml) (PR #85):

  • Add “biomass”, “non-comm”, “rc_spec”, and “rc_therm”.

  • Add “report” annotations for some items. These include string fragments to be used in variable names when reporting data in the IAMC data structure.

• generate_product() (and generate_set_elements()) can handle a regular expression to select a subset of codes for the Cartesian product (PR #85).

• New utility method Context.write_debug_archive() writes a ZIP archive containing files listed by Config.debug_paths (PR #85).

• WorkflowStep can store and apply keyword options for the optional clone() step at the start of the step execution (PR #85).

• Bugfix: WorkflowStep.__call__() ensures that Config.scenario_info on the Context instance passed to its callback matches the target scenario (PR #85).

2022.11.7

• Add the Zambia (ZMB) node code list (PR #83).

• Add the utility same_time(), to copy the set time in parameters (PR #83).

• New Config and model.Config dataclasses for clearer description/handling of recognized settings stored on Context (PR #82). ConfigHelper for convenience/utility functionality in message_ix_models-based code.

• New functions generate_product(), generate_set_elements(), get_region_codes() in model.structure (PR #82).

• Revise and improve the Workflow API (PR #82).

• Adjust for pandas 1.5.0 (PR #81).

2022.8.17

• Add nodes_ex_world() and use this in disutility.data_conversion() instead of expected a “World” node ID to be the first element in ScenarioInfo.N (PR #78).

• Add example files and documentation for IIASA Scenario Explorer metadata (PR #78).

• Expand ~ (i.e. $HOME) in the "message local data" configuration setting (PR #78).

2022.7.25

• Add get_advance_data(), and related tools for data from the ADVANCE project, including the node codelist for the data (PR #76).

• Add unit annotations to Commodities (commodity.yaml) (PR #76).

• New utility methods ScenarioInfo.io_units() to derive units for input and output parameters from units_for() commodity stocks and technology activities (PR #76).

• Transfer add_tax_emission() from message_data, improve, and add tests (PR #76).

• Unit annotations on commodity and technology codes are copied to child codes using process_units_anno() (PR #76).

• make_matched_dfs() accepts pint.Quantity to set both magnitude and units in generated data (PR #76).

• strip_par_data() also removes the set element for which data is being stripped (PR #76).

• The common CLI options --verbose and --dry-run are stored on Context automatically (PR #76).

• New utility method Context.set_scenario() (PR #76).

• iam_units.registry is the default unit registry even when message_data is not installed (PR #76).

• Expand broadcast() to allow DataFrame with multiple dimensions as input (PR #74).

2022.5.6

• Bump minimum required version of message_ix to v3.4.0 from v3.2.0 (PR #71).

• Add a documentation page on Distributed computing (PR #59).

• Add testing.not_ci() for marking tests not to be run on continuous integration services; improve session_context() (PR #62).

• apply_spec() also adds elements of the “node” set using ixmp.Platform.add_region() (PR #62).

• Add new logo the documentation (PR #68).

• Add Workflow; see Multi-scenario workflows (workflow) (PR #60).

2022.3.30

• Add adapt_R11_R12, a function for adapting data from the 11-region aggregation (R11) to the 12-region aggregation (R12) node lists (PR #56).

• Work around iiasa/ixmp#425 in disutility.data_conversion() (docs, PR #55).

2022.3.3

• Change the node name in R12.yaml from R12_CPA to R12_RCPA (PR #49).

• Register “message local data” ixmp configuration file setting and use to set the .Context.local_path when provided. See (4) Other, system-specific (“local”) directories (PR #47)

2022.1.26

• New Spec class for easier handling of specifications of model (or model variant) structure (PR #39)

• New utility function util.local_data_path() (PR #39).

• repr() of Context no longer prints a (potentially very long) list of all keys and settings (PR #39).

• as_codes() accepts a dict with Code values (PR #39).

Earlier releases

2021.11.24

• Add --years and --nodes to common_params() (PR #35).

• New utility function structure.codelists() (PR #35).

2021.7.27

• Improve caching using mod:genno v1.8.0 (PR #29).

2021.7.22

• Migrate utilities cached(), check_support(), convert_units(), maybe_query(), series_of_pint_quantity() (PR #27)

• Add testing.NIE.

• Add the --jvmargs option to pytest (see pytest_addoption()).

• Remove .Context.get_config_file(), .get_path(), .load_config(), and .units, all deprecated since 2021-02-28.

2021.7.6

• Add identify_nodes(), a function for identifying a Node code lists based on a Scenario (PR #24).

• Add adapt_R11_R14, a function for adapting data from the 11-region aggregation (R11) to the 14-region aggregation (R14) node lists (PR #24).

• Add export_test_data() and mix-models export-test-data command (PR #16). See Prepare data for testing.

• Allow use of pytest’s persistent cache across test sessions (PR #23). See Reproducibility.

• Add the 12-region aggregation (R12) node code list (PR #14).

2021.4.7

• Add model.disutility, code for setting up structure and data for generalized consumer disutility (PR #13)

2021.3.24

• Add Years or time periods (year/*.yaml), YAML data files, ScenarioInfo.year_from_codes() and associated tests (GH #11, PR #12)

2021.3.22

• Migrate model.bare, model.build, model.cli, and associated documentation (PR #9)

• Migrate utilities: ScenarioInfo, add_par_data(), eval_anno(), iter_parameters(), and strip_par_data().

2021.3.3

• Migrate util.click, .util.logging; expand documentation (PR #8:).

• Context.clone_to_dest() method replaces clone_to_dest() function.

• Build PDF documentation on ReadTheDocs.

• Allow CLI commands from both message_ix_models and message_data via mix-models.

• Migrate mix-models techs CLI command.

2021.2.28

• Migrate Context class and testing module from message_data (PR #5:).

• Add load_private_data(), package_data_path(), private_data_path().

• Document: Data, metadata, and configuration and Command-line interface.

• Update node codelists to ensure they contain both current and former ISO 3166 codes for countries that have changed status (PR #6:). For instance, ANT dissolved into BES, CUW, and SXM in 2010; all four are included in R11_LAM so this list can be used to handle data from either before or after 2010.

2021.2.26

• Add get_codes() and related code lists (PR #2:).

• Add MessageDataFinder and document Migrating from message_data (PR #3:).

2021.2.23

Initial release.