What’s New
Next release
ixmp
locates GAMS API libraries needed for the Java code underlyingJDBCBackend
based on the system GAMS installation (PR #532). As a result:JDBCBackend
is usable on MacOS with newer,arm64
-architecture processors and Python/GAMS compiled forarm64
(#473, #531).GAMS API libraries are no longer (re-)packaged with ixmp in the directory
ixmp/backend/jdbc/
.
v3.9.0 (2024-06-04)
All changes
v3.8.0 (2024-01-12)
Migration notes
Update code that imports from the following modules:
ixmp.reporting
→ useixmp.report
.ixmp.reporting.computations
→ useixmp.report.operator
.ixmp.utils
→ useixmp.util
.
Code that imports from the old locations will continue to work, but will raise DeprecationWarning
.
All changes
ixmp
is tested and compatible with Python 3.12 (PR #504).Support for Python 3.7 is dropped (PR #492), as it has reached end-of-life.
Rename
ixmp.report
andixmp.util
(PR #500).New option
record_version_packages
toGAMSModel
(PR #502). Versions of the named Python packages are recorded in a special set in GDX-format input and output files to help associate these files with generating code.New reporting operators
from_url()
,get_ts()
, andremove_ts()
(PR #500).New CLI command ixmp platform copy and CLI documentation (PR #500).
New argument
indexed_by=...
toScenario.items()
(thusScenario.par_list()
and similar methods) to iterate over (or list) only items that are indexed by a particular set (#402, PR #500).New
util.discard_on_error()
and matching argument toTimeSeries.transact()
to avoid lockingTimeSeries
/Scenario
on failed operations withJDBCBackend
(PR #488).Work around limitations of
JDBCBackend
(PR #500):Unit
""
cannot be added with the Oracle driver (#425).Certain items (variables) could not be initialized when providing
idx_sets=...
, even if those match the sets fixed by the underlying Java code. With this fix, a matching list is silently accepted; a different list raisesNotImplementedError
.When a
GAMSModel
is solved with an LP status of 5 (optimal, but with infeasibilities after unscaling),JDBCBackend
would attempt to read the output GDX file and fail, leading to an uninformative error message (#98). NowModelError
is raised describing the situation.
Improved type hinting for static typing of code that uses
ixmp
(#465, PR #500).ixmp
requires JPype1 1.4.0 or earlier, for Python 3.10 and earlier (PR #504). With JPype1 1.4.1 and later, memory management inCachingBackend
may not function as intended (#463), which could lead to high memory use where many, largeScenario
objects are created and used in a single Python program. (For Python 3.11 and later, any version of JPype1 from the prior minimum (1.2.1) to the latest is supported.)
v3.7.0 (2023-05-17)
All changes
ixmp
is tested and compatible with Python 3.11 (PR #481).ixmp
is tested and compatible with pandas 2.0.0 (PR #471). Note that pandas 1.4.0 dropped support for Python 3.7: thus whileixmp
still supports Python 3.7 this is achieved with pandas 1.3.x, which may not receive further updates (the last patch release was in December 2021). Support for Python 3.7 will be dropped in a future version ofixmp
, and users are encouraged to upgrade to a newer version of Python.Bugfix: year argument to
TimeSeries.timeseries()
acceptsint
orlist
ofint
(#440, PR #469).Adjust to pandas 1.5.0 (PR #458).
New module
util.sphinx_linkcode_github
to link documentation to source code on GitHub (PR #459).
v3.6.0 (2022-08-17)
All changes
Optionally tolerate failures to add individual items in
store_ts()
reporting computation (PR #451); usetimeseries_only=True
in check-out to function withScenario
with solution data stored.Bugfix:
Config
squashed configuration values read fromconfig.json
, if the respective keys were registered in downstream packages, e.g.message_ix
. Allow the values loaded from file to persist (PR #451).Adjust to genno 1.12 and set this as the minimum required version for
ixmp.reporting
(PR #451).Add
enforce()
to theModel
API for enforcing structure/data consistency beforeModel.run()
(PR #450).
v3.5.0 (2022-05-06)
All changes
Add new logo and diagram to the documentation (PR #446).
Raise an informative
ValueError
when adding infinite values withadd_timeseries()
; this is unsupported onJDBCBackend
when connected to an Oracle database (PR #443, #442).New attribute
url
for convenience in constructingTimeSeries
/Scenario
URLS (PR #444).New
store_ts()
reporting computation for storing time-series data on aTimeSeries
/Scenario
(PR #444).Minimum requirements are increased for dependencies (PR #435):
Python 3.7 or greater. Python 3.6 reached end-of-life on 2021-12-31.
Pandas 1.2 (2020-12-26) or greater, the oldest version with a minimum Python version of 3.7.
Improvements to configuration (PR #435):
The jvmargs argument to
JDBCBackend
can be set via the command line (ixmp platform add …) orConfig.add_platform()
; see Configuration (#408).Bug fix: user config file values from downstream packages (e.g.
message_ix
) are respected (#415).
Security: upgrade Log4j to 2.17.1 in Java code underlying
JDBCBackend
to address CVE-2021-44228, a.k.a. “Log4Shell” (PR #445).The ixmp Python package is not network-facing per se (unless exposed as such by user code; we are not aware of any such applications), so remote code execution attacks are not a significant concern. However, users should still avoid running unknown or untrusted code provided by third parties with versions of ixmp prior to 3.5.0, as such code could be deliberately crafted to exploit the vulnerability.
v3.4.0 (2022-01-24)
Migration notes
ixmp.util.isscalar()
is deprecated.
Code should use numpy.isscalar()
.
All changes
Add
TimeSeries.transact()
, for wrapping data manipulations incheck_out()
andcommit()
operations (PR #422).Add Data model, a documentation page giving a complete description of the
ixmp
data model (PR #422).Add the pytest --user-config command-line option, to use user’s local configuration when testing (PR #422).
Adjust
format_scenario_list()
for changes inpandas
1.3.0 (PR #421).
v3.3.0 (2021-05-28)
Migration notes
rixmp
is deprecated, though not yet removed, as newer versions of the R reticulate package allow direct import and use of the Python modules with full functionality.
See the updated page for Usage in R via reticulate.
All changes
Add
ixmp config show
CLI command (PR #416).Add
genno
andmessage_ix_models
to the output ofshow_versions()
/ixmp show-versions
(PR #416).Clean up test suite, improve performance, increase coverage (PR #416).
Adjust documentation for deprecation of
rixmp
(PR #416).Deprecate
util.logger()
(PR #399).Add a quiet option to
GAMSModel
and use in testing (PR #399).Fix
GAMSModel
would try to write GDX data to filenames containing invalid characters on Windows (PR #398).Format user-friendly exceptions when GAMSModel errors (#383, PR #398).
Adjust
ixmp.reporting
to usegenno
(PR #397).Fix two minor bugs in reporting (PR #396).
v3.2.0 (2021-01-24)
All changes
Increase JPype minimum version to 1.2.1 (PR #394).
Adjust test suite for pandas v1.2.0 (PR #391).
Raise clearer exceptions from
add_par()
for incorrect parameters; silently handle empty data (PR #374).Depend on
openpyxl
instead ofxlrd
andxlsxwriter
for Excel I/O;xlrd
versions 2.0.0 and later do not support.xlsx
(PR #389).Add a parameter for exporting all model+scenario run versions to
Platform.export_timeseries_data()
, and fix a bug where exporting all runs happens uninteneded (PR #367).Silence noisy output from ignored exceptions on JDBCBackend/JVM shutdown (PR #378).
Add a utility method,
gams_version()
, to check the installed version of GAMS (PR #376). The result is displayed by theixmp show-versions
CLI command/show_versions()
.init_par()
and related methods accept any sequence (not merelylist
) ofstr
for the idx_sets and idx_names arguments (PR #376).
v3.1.0 (2020-08-28)
All changes
ixmp v3.1.0 coincides with message_ix v3.1.0.
Fix a bug in
read_excel()
when parameter data is spread across multiple sheets (PR #345).Expand documentation and revise installation instructions (PR #363).
Raise Python exceptions from
JDBCBackend
(PR #362).Add
Scenario.items()
,util.diff()
, and allow using filters in CLI commandixmp export
(PR #354).Add functionality for storing ‘meta’ (annotations of model names, scenario names, versions, and some combinations thereof) (PR #353).
Add
Backend.add_model_name()
,add_scenario_name()
,get_model_names()
,get_scenario_names()
,get_meta()
,set_meta()
,remove_meta()
.Allow these to be called from
Platform
instances.Remove
Scenario.delete_meta()
.
Avoid modifying indexers dictionary in
AttrSeries.sel
(PR #349).Add region/unit parameters to
Platform.export_timeseries_data()
(PR #343).Preserve dtypes of index columns in
data_for_quantity()
(PR #347).ixmp show-versions
includes the path to the default JVM used by JDBCBackend/JPype (PR #339).Make
reporting.Quantity
classes interchangeable (PR #317).Use GitHub Actions for continuous testing and integration (PR #330).
v3.0.0 (2020-06-05)
ixmp v3.0.0 coincides with message_ix v3.0.0.
Migration notes
- Excel input/output (I/O)
The file format used by
Scenario.to_excel()
andread_excel()
is now fully specified; see File formats and input/output.ixmp writes and reads items with more elements than the ~10⁶ row maximum of the Excel data format, by splitting these across multiple sheets.
The I/O code now explicitly checks for situations where the index sets and names for an item are ambiguous; see this example for how to initialize and read these items.
- Updated dependencies
The minimum versions of the following dependencies are increased:
JPype1 0.7.5
pandas 1.0
dask 2.14 (for reporting)
- Deprecations and deprecation policy
The following items, marked as deprecated in ixmp 2.0, are removed (PR #254):
$HOME/.local/ixmp/
as a configuration location. Configuration files are now placed in the standard$HOME/.local/share/ixmp/
.positional and
dbtype=
arguments toPlatform
/JDBCBackend
.first_model_year=
,keep_sol=
, andscen=
arguments toclone()
. Use shift_first_model_year, keep_solution, and scenario, respectively.rixmp.legacy
, an earlier version of the R interface that did not use reticulate.
Newly deprecated is:
cache keyword argument to
Scenario
. Caching is controlled at thePlatform
/Backend level, using the same keyword argument.
Starting with ixmp v3.0, arguments and other features marked as deprecated will follow a standard deprecation policy: they will be removed no sooner than the second major release following the release in which they are marked deprecated. For instance, a feature marked deprecated in ixmp version “10.5” would be retained in ixmp versions “11.x”, and removed only in version “12.0” or later.
All changes
Bump JPype dependency to 0.7.5 (PR #327).
Improve memory management in
JDBCBackend
(PR #298).Raise user-friendly exceptions from
Reporter.get
in Jupyter notebooks and other read–evaluate–print loops (REPLs) (PR #316).Ensure
Model.initialize()
is always called for new and cloned objects (PR #315).Add CLI command ixmp show-versions to print ixmp and dependency versions for debugging (PR #320).
Bulk saving for metadata and exposing documentation AP (PR #314)I
Add
apply_units()
,select()
reporting operators; expandReporter.add
(PR #312).Reporter.add_product
accepts aKey
with a tag;aggregate()
preservesQuantity
attributes (PR #310).Add CLI command
ixmp solve
to run model solver (PR #304).Add dims and units arguments to
Reporter.add_file
; removeReporter.read_config()
(redundant withReporter.configure
) (PR #303).Add option to include subannual column in dataframe returned by
TimeSeries.timeseries()
(PR #295).Add
Scenario.to_excel()
andread_excel()
; this functionality is transferred to ixmp frommessage_ix
and enhanced for dealing with maximum row limits in Excel (PR #286, PR #297, PR #309).Include all tests in the ixmp package (PR #270).
Add
Model.initialize()
API to help populate new Scenarios according to a model scheme (PR #212).Apply units to reported quantities (PR #267).
Increase minimum pandas version to 1.0; adjust for API changes and deprecations (PR #261).
Add
export_timeseries_data()
to write data for multiple scenarios to CSV (PR #243).Implement methods to get and create new subannual timeslices (PR #264).
v2.0.0 (2020-01-14)
ixmp v2.0.0 coincides with message_ix v2.0.0.
Migration notes
Support for Python 2.7 is dropped as it has reached end-of-life, meaning no further releases will be made even to fix bugs.
See PEP-0373 and https://python3statement.org.
ixmp
users must upgrade to Python 3.
Configuration for ixmp and its storage backends has been streamlined.
See the ref:Configuration section of the documentation for complete details on how to use ixmp platform add
register local and remote databases.
To migrate from pre-2.0 settings:
- DB_CONFIG_PATH
…pointed to a directory containing database properties (.properties) files.
All Platform configuration is stored in one ixmp configuration file, config.json, and manipulated using the
ixmp platform
command and subcommands.The
Platform
constructor accepts the name of a stored platform configuration.Different storage backends may accept relative or absolute paths to backend-specific configuration files.
- DEFAULT_DBPROPS_FILE
…gave a default backend via a file path.
On the command line, use
ixmp platform add default NAME
to setNAME
as the default platform.This platform is loaded when
ixmp.Platform()
is called without any arguments.
- DEFAULT_LOCAL_DB_PATH
…pointed to a default local database.
ixmp.config
always contains a platform named ‘local’ that is located below the configuration path, in the directory ‘localdb/default’.To change the location for this platform, use e.g.:
ixmp platform add local jdbc hsqldb PATH
.
All changes
Add
ixmp list
command-line tool (PR #240).Ensure filters are always converted to string (PR #225).
Identify and load Scenarios using URLs (PR #189).
Add new Backend, Model APIs and CachingBackend, JDBCBackend, GAMSModel classes (PR #182, PR #200, PR #213, PR #217, PR #230, PR #245, PR #246).
Set convertStrings=True for JPype >= 0.7; see the JPype changelog (PR #174).
Make AppVeyor CI more robust; support pandas 0.25.0 (PR #173).
Add support for handling geodata (PR #165).
Fix exposing whole config file to log output (PR #232).
v0.2.0 (2019-06-25)
ixmp 0.2.0 provides full support for clone()
across platforms (database instances), e.g. from a remote database to a local HSQL database.
IAMC-style timeseries data is better supported, and can be used to store processed results, together with model variables and equations.
Other improvements include a new, dedicated ixmp.testing
module, and user-supplied callbacks in solve()
.
The retixmp
package using reticulate to access the ixmp API is renamed to rixmp
and now has its own unit tests (the former rixmp
package can be accessed as rixmp.legacy
).
Release 0.2.0 coincides with MESSAGEix release 1.2.0.
All changes
Test
rixmp
(formerretixmp
) using the Rtestthat
package (PR #135).Cloning across platforms, better support of IAMC_style timeseries data, preparations for MESSAGEix release 1.2 in Java core (PR #142).
Support iterating with user-supplied callbacks (PR #115).
Recognize
IXMP_DATA
environment variable for configuration and local databases (PR #130).Fully implement
clone()
across platforms (databases) (PR #129, PR #132).New module
ixmp.testing
for reuse of testing utilities (PR #128, PR #137).Add functions to view and add regions for IAMC-style timeseries data (PR #125).
Return absolute path from
find_dbprops()
(PR #123).Switch to RTD Sphinx theme (PR #118).
Bugfix and extend functionality for working with IAMC-style timeseries data (PR #116).
Add functions to check if a Scenario has an item (set, par, var, equ) (PR #111).
Generalize the internal functions to format index dimensions for mapping sets and parameters (PR #110).
Improve documentation (PR #108).
Replace deprecated pandas
.ix
indexer with.iloc
(PR #105).Specify dependencies in setup.py (PR #103).
v0.1.3 (2018-11-21)
Connecting to multiple databases, updating MESSAGE-scheme scenario specifications to version 1.1 (PR #88).
Can now set logging level which is harmonized between Java and Python (PR #80).
Adding a deprecated-warning for ixmp.Scenario with scheme==’MESSAGE’ (PR #79).
Changing the API from
mp.Scenario(...)
toixmp.Scenario(mp, ...)
(PR #76).Adding a function
has_solution()
, rename kwargs to …_solution (PR #73).Bring retixmp available to other users (PR #69).
Support writing multiple sheets to Excel in utils.pd_write (PR #64).
Now able to connect to multiple databases (Platforms) (PR #61).
Add MacOSX support in CI (PR #58).
Add ability to load all scenario data into memory for fast subsequent computation (PR #52).