What’s new
Next release
message_ix_models
is tested and compatible with Python 3.13 (PR #250).Support for Python 3.8 is dropped (PR #250), as it has reached end-of-life.
Expand Reproducibility with sections on Documentation and Versioning and naming, including a list of external model names and ‘versions’ like “MESSAGEix-GLOBIOM 2.0” (GH #224, PR #226).
New code list Emission species (emission.yaml) (PR #225).
By topic:
Materials
Improve and extend MESSAGEix-Materials (model.material) (PR #218, PR #253).
Release of MESSAGEix-Materials 1.1.0 (Version 1.1.0 (August 27, 2024)).
Transport
Update MESSAGEix-Transport (model.transport) (PR #213, PR #225).
Adopt consistent terms “F RAIL” and “F ROAD” for freight service/modes.
New technologies: “f rail {electr,lightoil}”, “f road electr”.
Extend unit annotations to all transport commodities.
New input files
pdt-cap.csv
andload-factor-ldv.csv
.Add mix-models ssp transport CLI command to postprocess aviation emissions data.
New SDMX Codelist
IIASA:CL_TRANSPORT_SCENARIO
to distinguish “Low energy demand (LED)” and EDITS (project.edits) scenarios from Shared Socioeconomic Pathways (project.ssp) baseline scenarios.Adjust R12 baseline settings:
mode-share/default.csv
: adjustR12_NAM
values for AIR and LDV.pdt-cap-ref.csv
: adjustR12_NAM
value.
Implement LED scenarios via
load-factor-ldv.csv
andpdt-cap.csv
.Use y=2019 data from IEA EWEB 2024 edition to align MESSAGEix-Transport with base model calibration.
New
genno
operators:broadcast()
,broadcast_wildcard()
,broadcast_t_c_l()
,freight_usage_output()
.New utility methods
get_cl_scenario()
,wildcard()
.Replace
DataSourceConfig.LDV
setting withConfig.dummy_LDV
.
Water/Nexus
Update water availability data and major code editing to allow a new test suite for doc:/water/index (PR #106).
Fix the nexus/cooling function and add test for checking some input data (PR #236).
Make setup of constraints for cooling technologies flexible and update solar CSP technology name (PR #242).
Connect
model.water
totools.costs
for cooling technologies (PR #245).Introduce an SSP configuration for cooling technologies with share constraints (PR #256).
Change SSP parameters and add missing CSP data (PR #262).
Investment and fixed costs
Fix naming of GDP and population columns in SSP data aggregation for Investment and fixed costs (tools.costs) (PR #219).
Edit inputs for storage, CSP, hydrogen, and industry technologies (PR #206).
Replace solar and wind technologies with new ones (PR #206).
Reorganize input files and incorporate first_year.csv data into tech_map.csv (PR #221).
Reconfigure use and implementation of technology variants/modules to be more agnostic (PR #221).
Change cost decay to reach reduction percentage specified on the year 2100 (PR #227).
Add cooling technology variant/module (PR #222).
Add functionality to specify cost reduction values and cost reduction scenarios in a module (GH #251, PR #255).
Fix technology mapping of ammonia CCS technologies in materials module (PR #258).
Remove unused technology_type column from tool (PR #269).
Others
Add “LED”, “SSP4”, and “SSP5” as values for the --ssp=… option in
common_params()
(PR #233).Add CircEUlar (project.circeular) project code and documentation (PR #232).
iea.web
handles the 2024 edition and fixed-width file format published by the IEA directly (PR #225).Code lists for the
COUNTRY
,FLOW
, andPRODUCT
concepts are included withmessage_ix_models
.Add
COUNTRY_NAME
to map particular labels appearing in these data.Fuzzed data and tests for this functionality.
Add
.types.ParameterData
and.types.MutableParameterData
to type the common internal data structure in which adict
maps from MESSAGE parameter names tomessage_ix
-structuredpandas.DataFrame
(PR #225).message_ix_models.Config
—the “core” configuration class—gains methods specific to its settings (PR #225). These were formerly onContext
.Context
is no longer a subclass ofdict
(PR #225). This avoids its mishandling bydask
version 2024.11.0 or later when Context is used withixmp.Reporter
.Add
MODULE_WITH_CONFIG_DATACLASS
; add attributes such asContext.model
that are typed to the respective class such asmodel.Config
.Add
Context.asdict()
for serialization.Former methods such as
Context.get_scenario()
are aliased to their new locations, e.g.Config.get_scenario()
.
Improve
ScenarioInfo
(PR #225):Implement the
|
(logical OR/union) operator: si_a | si_b is a new ScenarioInfo instance with the union of the contents of the operands.New method
substitute_codes()
to replace string codes (for instance, determined from a Scenario object) with Codes from the corresponding code list, including all annotations.
MappingAdapter
skips missing labels in the input data without raising an exception (PR #225).Workflow.visualize()
displays in left-to-right rank direction by default (PR #225).convert_units()
can handle MESSAGE-schemepandas.DataFrame
(PR #225).util.sdmx.make_enum()
usesURNLookupEnum
by default (PR #225). This allows to call, for instance,SSP_2024.by_urn("…Code=ICONICS:SSP(2024).2")
to retrieve an enumeration number.make_matched_dfs()
acceptsdict
for itspar_value
arg, allowing replacement of values for particular dimensions of output data frames (PR #225).New reporting operator
quantity_from_iamc()
(PR #225).same_node()
andsame_time()
can handle.ParameterData
(PR #225).report.register()
becomesreport.Config.register()
which populatesreport.Config.callback
(PR #225). The set of callback functions used to configure a class:.Reporter instance is thus now particular to areport.Config
instance attached to aContext
instance. This allows better isolation of code/tests that use different sets of callbacks.
v2024.8.6
Add MESSAGEix-Materials (model.material) (PR #188, PR #189).
Update MESSAGEix-Materials (model.material) (PR #201).
Add MESSAGEix-Transport (model.transport) (PR #207, PR #208, GH #210, PR #212).
Add EDITS (project.edits) project code and documentation (PR #204).
Reduce log verbosity of
apply_spec()
(PR #202).Fix and update Investment and fixed costs (tools.costs) (PR #186, PR #187, PR #190, PR #195).
Fix jumps in cost projections for technologies with first technology year that’s after than the first model year (PR #186).
Change the use of base_year to mean the year to start modeling cost changes (PR #186).
Update cost assumptions for certain CCS technologies (PR #186).
Change the default fixed O&M reduction rate to 0 (PR #186).
Modify to use 2023 release of IEA WEO data and to use 2022 historic data for the base year (PR #187).
Change the default final year to 2110 (PR #190).
Add
use_vintages
to control whether vintages are used in computing fixed O&M costs (PR #195).
v2024.4.22
Incorporate the MESSAGEix-GLOBIOM global model documentation (PR #107, PR #110). This documentation formerly lived at https://docs.messageix.org/global/ and in a separate repository at iiasa/message_doc.
Improve tool for Migrate individual modules using git filter-repo (PR #174); expand documentation.
New module for Investment and fixed costs (tools.costs) (PR #99).
Update investment cost assumptions in Investment and fixed costs (tools.costs) for wind and solar technologies (PR #176).
Remove manually specified base year reference region costs for solar_pv_ppl and solar_pv_RC so that 2021 IEA WEO costs are used for these technologies.
Fix the manually specified base year reference region cost for wind_ppf.
Update cost reduction rates and scenarios for wind_ppf so that it follows the same narratives as wind_ppl.
Convert Intratec data for Investment and fixed costs (tools.costs) from Excel to a simpler CSV format. (PR #167).
Migrate ‘Legacy’ reporting (report.legacy) to provide post-processing functionality for the global model snapshot (PR #159).
Migrate and improve code for four sources of exogenous data (PR #162):
project.gea
,project.shape
,tools.gfei
, andtools.iea.eei
.Add a Quick-start guide to running a MESSAGEix-GLOBIOM baseline model (PR #157).
Expand Data, metadata, and configuration (PR #161).
Add an explicit Upstream version policy (PR #162).
v2024.4.2
The
SSPUpdate
data provider pulls data from the SSP 2024 “Release 3.0” data files, and handles both the earlier and current structures (PR #156).Improve
ExoDataSource
withraise_on_extra_kw()
utility method, automatic copy of source keyword arguments (PR #156).Expose
node.nodes_ex_world()
for use as a genno (reporting) operator.Raise DeprecationWarning from
util.sdmx.eval_anno()
; remove internal usage of this deprecated method (PR #156).Reduce verbosity when using the mix-models CLI when
message_data
is not installed (GH #37, PR #156).Improve logging (PR #156).
Use multi-threaded logging for better performance. Logging to stdout and file is on a separate thread and does not block operations on the main thread.
Add automatic file logging. Log versions of packages to file when using
workflow.make_click_command()
.New CLI command mix-models last-log to retrieve the location of the latest log file.
Update Command-line interface (PR #156).
Improve performance in
disutility.data_conversion()
(PR #156).Use
platformdirs.user_cache_path()
in more places; remove cache-path handling code (PR #156).Add
util.show_versions()
, wrappingixmp.util.show_versions()
and returning its output asstr
(PR #156).util.private_data_path()
returns an alternate, local data path ifmessage_data
is not installed (PR #156).Annotate
c="transport"
in the commodity code list with associated IEA (E)WEB flows (PR #153).
v2024.1.29
Add (Extended) World Energy Balances (tools.iea.web) for handling data from the International Energy Agency (IEA) Extended World Energy Balances (GH #25, PR #75).
Add World Bank structures (tools.wb) and
assign_income_groups()
to assign MESSAGE regions to World Bank income groups (PR #144).Adjust
report.compat
for genno version 1.22 (GH #141, PR #142).Raise informative exception from
ScenarioInfo.io_units()
(PR #151).
v2023.11.24
Migration notes
Update code that imports from the following modules:
message_ix_models.report.computations
→ usemessage_ix_models.report.operator
.
Code that imports from the old locations will continue to work, but will raise DeprecationWarning
.
Data for MESSAGEix-Nexus (model.water) is no longer included in the PyPI distributions for message_ix_models
.
This reduces the package size from >20 MB to <5 MB.
To automatically download and unpack these data into a local directory, use mix-models fetch MESSAGEix-Nexus.
All changes
Improve
ExoDataSource
(PR #137):New attributes
name
,extra_dims
.New method
transform()
that can be overridden by subclasses.New arguments
archive_member
,non_iso_3166
toiamc_like_data_for_query()
.
New provider for exogenous data from the
ADVANCE
project (PR #137). This module,project.advance
, supersedestools.advance
and its idiosyncratic API, which are deprecated.New CLI commands (PR #137):
mix-models testing fuzz-private-data, superseding mix-models ssp make-test-data.
mix-models fetch, superseding mix-models snapshot fetch.
New utility functions (PR #137).
tools.iamc.describe()
to generate SDMX code lists that describe the structure of particular IAMC-format data (PR #137).workflow.make_click_command()
to generateclick
commands for anyWorkflow
.util.minimum_version()
to ensure compatibility with upstream packages and aid test writing.util.iter_keys()
to generate keys for chains ofgenno
computations.
Add
message_ix_models.report.compat
for emulating legacy reporting (PR #134).Rename
message_ix_models.report.operator
(PR #137).Deprecate
iter_parameters()
in favour ofixmp.Scenario.par_list()
withindexed_by=...
argument from ixmp v3.8.0 (PR #137).
v2023.10.16
New providers for exogenous data from the
SSPOriginal
andSSPUpdate
(PR #125) sources.Improved
ScenarioInfo
(PR #125):Improvements to
report
(PR #125):New
report.Config
class collecting recognized settings for the module.context["report"]
always exists as an instance ofreport.Config
.New submodule
report.plot
with base class and 5 plots of time-series data stored on Scenarios.Submodule
report.sim
providesadd_simulated_solution()
for testing reporting configuration.New operator
filter_ts()
.
New reusable command-line option --urls-from-file in
util.click
(PR #125).
v2023.9.12
All changes
New module
project.ssp
(PR #122) to generate SDMX codelists for the 2017/original SSPs and the 2024 update, and provide these asEnum
to other code.New module
tools.exo_data
to retrieve exogenous data for, among others, population and GDP (PR #122). This module has a general API that can be implemented by provider classes.New function
model.emissions.get_emission_factors()
and associated data file to provide data from this table in the MESSAGEix-GLOBIOM documentation (PR #122).New functions in
util.sdmx
(PR #122):make_enum()
to make pure-PythonEnum
(or subclass) data structures based on SDMX code lists.
same_node()
also fills “node_shares”, “node_loc”, and “node”, as appropriate (PR #122).
Deprecations
eval_anno()
is deprecated; code should instead usesdmx.model.common.AnnotableArtefact.eval_annotation()
, which provides the same functionality.
v2023.9.2
New module
message_ix_models.report
for reporting (PR #116). Use of this module requires ixmp and message_ix version 3.6.0 or greater.Add documentation on Migrate individual modules using git filter-repo using git filter-repo and helper scripts (PR #89).
v2023.7.26
Add code and CLI commands to fetch and load MESSAGEix-GLOBIOM snapshots (PR #102). Use of this module requires ixmp and message_ix version 3.5.0 or greater.
Add
util.pooch.fetch()
, a thin wrapper for using Pooch (PR #102).New module
message_ix_models.model.macro
with utilities for calibratingmessage_ix.macro
(PR #104).New method
Workflow.guess_target()
(PR #104).Change in behaviour of
Workflow.add_step()
: the method now returns the name of the newly-added workflow step, rather than theWorkflowStep
object added to carry out the step (PR #104). The former is more frequently used in code that usesWorkflow
.Add the 17-region aggregation (R17) node code list (PR #109).
Add the 20-region aggregation (R20) node code list (PR #109).
v2023.5.31
v2023.5.13
Adjust
generate_product()
for pandas 2.0.0 (PR #98).
2023.4.2
New utility function
replace_par_data()
(PR #90).disutility.get_spec()
preserves allAnnotations
attached to theCode
object used as a template for usage technologies (PR #90).Add
CO2_Emission_Global_Total
to the “A” relation codelist (PR #90).Adapter
andMappingAdapter
can be imported frommessage_ix_models.util
(PR #90).
2023.2.8
Codelists for the
relation
MESSAGEix set (PR #85):The “bare” reference energy system now includes relations from the codelist indicated by
model.Config.relations
; default “A”.
Commodities (commodity.yaml) (PR #85):
Add “biomass”, “non-comm”, “rc_spec”, and “rc_therm”.
Add “report” annotations for some items. These include string fragments to be used in variable names when reporting data in the IAMC data structure.
generate_product()
(andgenerate_set_elements()
) can handle a regular expression to select a subset of codes for the Cartesian product (PR #85).New utility method
Context.write_debug_archive()
writes a ZIP archive containing files listed byConfig.debug_paths
(PR #85).WorkflowStep
can store and apply keyword options for the optionalclone()
step at the start of the step execution (PR #85).Bugfix:
WorkflowStep.__call__()
ensures thatConfig.scenario_info
on theContext
instance passed to its callback matches the target scenario (PR #85).
2022.11.7
Add the Zambia (ZMB) node code list (PR #83).
Add the utility
same_time()
, to copy the set time in parameters (PR #83).New
Config
andmodel.Config
dataclasses
for clearer description/handling of recognized settings stored onContext
(PR #82).ConfigHelper
for convenience/utility functionality inmessage_ix_models
-based code.New functions
generate_product()
,generate_set_elements()
,get_region_codes()
inmodel.structure
(PR #82).Revise and improve the Workflow API (PR #82).
Adjust for pandas 1.5.0 (PR #81).
2022.8.17
Add
nodes_ex_world()
and use this indisutility.data_conversion()
instead of expected a “World” node ID to be the first element inScenarioInfo.N
(PR #78).Add example files and documentation for IIASA Scenario Explorer metadata (PR #78).
Expand
~
(i.e.$HOME
) in the"message local data"
configuration setting (PR #78).
2022.7.25
Add
get_advance_data()
, and related tools for data from the ADVANCE project, including the node codelist for the data (PR #76).Add unit annotations to Commodities (commodity.yaml) (PR #76).
New utility methods
ScenarioInfo.io_units()
to derive units forinput
andoutput
parameters fromunits_for()
commodity stocks and technology activities (PR #76).Transfer
add_tax_emission()
frommessage_data
, improve, and add tests (PR #76).Unit annotations on commodity and technology codes are copied to child codes using
process_units_anno()
(PR #76).make_matched_dfs()
acceptspint.Quantity
to set both magnitude and units in generated data (PR #76).strip_par_data()
also removes the set element for which data is being stripped (PR #76).The common CLI options --verbose and --dry-run are stored on
Context
automatically (PR #76).New utility method
Context.set_scenario()
(PR #76).iam_units.registry
is the default unit registry even whenmessage_data
is not installed (PR #76).Expand
broadcast()
to allowDataFrame
with multiple dimensions as input (PR #74).
2022.5.6
Bump minimum required version of
message_ix
to v3.4.0 from v3.2.0 (PR #71).Add a documentation page on Distributed computing (PR #59).
Add
testing.not_ci()
for marking tests not to be run on continuous integration services; improvesession_context()
(PR #62).apply_spec()
also adds elements of the “node” set usingixmp.Platform.add_region()
(PR #62).Add new logo the documentation (PR #68).
Add
Workflow
; see Multi-scenario workflows (workflow) (PR #60).
2022.3.30
Add
adapt_R11_R12
, a function for adapting data from the 11-region aggregation (R11) to the 12-region aggregation (R12) node lists (PR #56).Work around iiasa/ixmp#425 in
disutility.data_conversion()
(docs, PR #55).
2022.3.3
Change the node name in R12.yaml from R12_CPA to R12_RCPA (PR #49).
Register “message local data” ixmp configuration file setting and use to set the
.Context.local_path
when provided. See (4) Other, system-specific (“local”) directories (PR #47)
2022.1.26
New
Spec
class for easier handling of specifications of model (or model variant) structure (PR #39)New utility function
util.local_data_path()
(PR #39).repr()
ofContext
no longer prints a (potentially very long) list of all keys and settings (PR #39).as_codes()
accepts adict
withCode
values (PR #39).
Earlier releases
2021.11.24
Add --years and --nodes to
common_params()
(PR #35).New utility function
structure.codelists()
(PR #35).
2021.7.27
Improve caching using mod:genno v1.8.0 (PR #29).
2021.7.22
Migrate utilities
cached()
,check_support()
,convert_units()
,maybe_query()
,series_of_pint_quantity()
(PR #27)Add
testing.NIE
.Add the
--jvmargs
option to pytest (seepytest_addoption()
).Remove
.Context.get_config_file()
,.get_path()
,.load_config()
, and.units
, all deprecated since 2021-02-28.
2021.7.6
Add
identify_nodes()
, a function for identifying a Node code lists based on aScenario
(PR #24).Add
adapt_R11_R14
, a function for adapting data from the 11-region aggregation (R11) to the 14-region aggregation (R14) node lists (PR #24).Add
export_test_data()
and mix-models export-test-data command (PR #16). See Prepare data for testing.Allow use of pytest’s persistent cache across test sessions (PR #23). See Reproducibility.
Add the 12-region aggregation (R12) node code list (PR #14).
2021.4.7
Add
model.disutility
, code for setting up structure and data for generalized consumer disutility (PR #13)
2021.3.24
Add Years or time periods (year/*.yaml), YAML data files,
ScenarioInfo.year_from_codes()
and associated tests (GH #11, PR #12)
2021.3.22
Migrate
model.bare
,model.build
,model.cli
, and associated documentation (PR #9)Migrate utilities:
ScenarioInfo
,add_par_data()
,eval_anno()
,iter_parameters()
, andstrip_par_data()
.
2021.3.3
Migrate
util.click
,.util.logging
; expand documentation (PR #8:).Context.clone_to_dest()
method replacesclone_to_dest()
function.Build PDF documentation on ReadTheDocs.
Allow CLI commands from both
message_ix_models
andmessage_data
via mix-models.Migrate mix-models techs CLI command.
2021.2.28
Migrate
Context
class andtesting
module frommessage_data
(PR #5:).Add
load_private_data()
,package_data_path()
,private_data_path()
.Document: Data, metadata, and configuration and Command-line interface.
Update node codelists to ensure they contain both current and former ISO 3166 codes for countries that have changed status (PR #6:). For instance, ANT dissolved into BES, CUW, and SXM in 2010; all four are included in R11_LAM so this list can be used to handle data from either before or after 2010.
2021.2.26
Add
get_codes()
and related code lists (PR #2:).Add
MessageDataFinder
and document Migrating from message_data (PR #3:).
2021.2.23
Initial release.