Advanced installation guide

This guide includes detailed instructions for a variety of ways of installing MESSAGEix in cases beyond the simple case covered by the Quick install instructions.

Again, be sure that you have the prerequisites skills and knowledge; these include specific points of knowledge that are necessary to understand these instructions and choose among different installation options.

Install system dependencies

Python (required)

MESSAGEix requires Python version 3.8 or greater. We recommend the latest version; currently Python 3.12. Common ways to install Python include:

• Use the official Python releases.

• Use a third-party Python distribution, such as Miniconda or Anaconda. [1] [2]

• Use a version of Python bundled with your operating system (for example, Ubuntu and other Linux distributions).

Before making this choice, see Choose pip or conda, below, for further considerations.

[1]

See the conda glossary for the differences between Anaconda and Miniconda, and the definitions of the terms ‘channel’ and ‘environment’ here.

[2]

On newer macOS systems with Apple M-series processors: the Miniconda or Anaconda installers provided for the arm64 architecture lead to errors in ixmp. Currently, we recommend to instead use the macOS installers for x86_64 architecture on these systems. See ixmp issues 473 and 531.

Java (required)

A Java Runtime Environment (JRE) is required to use the current default ixmp.JDBCBackend in the ixmp package that handles data storage for message_ix.

Common ways to install Java include:

• Use the official releases of Java.

• Use a version of Java bundled or packaged for your operating system (for example, Ubuntu and other Linux distributions).

If using Anaconda or Miniconda, installing Java manually is not required. Tis is because the message-ix conda-forge package depends on the openjdk package, so the latter is automatically installed with the former.

GAMS (required)

MESSAGEix requires GAMS.

1. Download GAMS for your operating system; either the latest version or, for users not familiar with GAMS licenses, version 29 (see note below).

2. Run the installer.

3. Ensure that the PATH environment variable on your system includes the path to the GAMS program:

   • on Windows, in the GAMS installer…

     • Check the box labeled “Use advanced installation mode.”

     • Check the box labeled “Add GAMS directory to PATH environment variable” on the Advanced Options page.

   • on macOS, in the GAMS installer…

     • When prompted to specify the “Installation Type” (step 3 of the installation process), select “Customise”.

     • Check the box labeled “Add GAMS to PATH”.

     If this option is not available see instructions below.

   • on other platforms (macOS or Linux), add the following line to a file such as ~/.bash_profile (macOS), ~/.bashrc, or ~/.profile:

     $ export PATH=$PATH:/path/to/gams-directory-with-gams-binary
     

   Run gams in a terminal/command prompt to confirm this step has taken effect.

Note

MESSAGE-MACRO and MACRO require GAMS 24.8.1 or later (see MACRO.GAMS_min_version) The latest version is recommended.

GAMS is proprietary software and requires a license to solve optimization problems. To run both the message_ix and ixmp tutorials and test suites, a “free demonstration” license is required; the free license is suitable for these small models. Versions of GAMS up to version 29 include such a license with the installer; since version 30, the free demo license is no longer included, but may be requested via the GAMS website.

Note

If you only have a license for an older version of GAMS, install both the older and the latest versions.

Graphviz (optional)

Reporter.visualize() uses Graphviz, a program for graph visualization. Installing message_ix causes the graphviz Python package to be installed. If you want to use visualize() or run the test suite, the Graphviz program itself must also be installed; otherwise it is optional.

If you install MESSAGEix using conda, Graphviz is installed automatically via its conda-forge package. For other methods of installation (such as pip) see the Graphviz download page for downloads and instructions for your system.

Install MESSAGEix

4. Open a terminal/command prompt.

   Windows users who have installed Python using Anaconda/Miniconda should use the “Anaconda Prompt” to avoid issues with permissions and environment variables. This program is available in the Windows Start menu after installing Anaconda.

Choose pip or conda

We recommend that new users install MESSAGEix using pip (user guide), the package manager recommended by the Python Software Foundation. pip can be used when Python is installed directly, or it can be installed using conda. [3]

If you are more comfortable with Anaconda, you can also install MESSAGEix using conda.

Advanced users may choose to install from source code, to benefit from the latest features or to test features that have not been merged. For this purpose pip must be used; while it is possible to do this within an initial install made using conda, [3] it is usually simpler not to mix the two and instead use pip from the start.

Whichever option you choose, please skip the other sections.

[3] (1,2)

If you intend to use pip in a conda environment, please read conda’s guide to using pip in an environment. In particular, please make sure to use conda only to install pip into an environment, and then use that environment-specific pip for all further package installation.

Create and activate a virtual environment

See Prerequisite knowledge and skills > Basic usage > Scientific computing skills > #6. In particular, the two links given for venv module documentation explain the general concept of virtual environments.

For MESSAGEix usage, many users choose to create one virtual environment for each project, and switch between those environments in order to switch between project-specific versions of message_ix, ixmp, message_ix_models, and any other dependencies.

It is also possible to use MESSAGEix without a virtual environment, but we strongly recommend that you create and use one. The way of doing so depends on whether you chose:

• pip —then the steps further depend on which virtual environment tool you choose. This guide gives examples for the first-party venv and third-party virtualenv; for others, see their documentation.

• conda —this program handles both virtual environment and package management.

See the respective sections below.

Use pip

5. Create a virtual environment. Using venv, per the documentation:

   python -m venv message_env
   

   or using virtualenv:

   virtualenv message_env
   

   These examples store the environment files in a directory named message_env under the current working directory, but you can also place these anywhere else on your system.

6. Activate the environment with:

   # On Linux or macOS
   source message_env/bin/activate
   
   # On Windows
   .\message_env\Scripts\activate
   

   These examples use the directory created in the previous step. If you stored your virtual environment elsewhere, use the appropriate path.

7. Ensure pip is installed:

   pip --version
   

   If not, see the installation instructions for pip.

Choose optional dependencies

When installing using pip (but not conda), there is a distinction between required and optional dependencies. For example ixmp is a required dependency of message_ix. Whenever the latter is installed, a compatible version of the former will also be installed.

Optional dependencies (also called “extra requirements”) are gathered in groups. The example commands below include a string like [docs,report,tests,tutorial]. This implies four groups of extra requirements:

• docs includes packages required to build this documentation locally,

• report includes packages required to use the built-in reporting features of message_ix,

• tests includes packages required to run the test suite, and

• tutorial includes packages required to run the tutorials.

The set of extras used can be freely adjusted according to your needs.

Install the latest release from PyPI

8. Install MESSAGEix [4]:

   pip install message_ix[docs,report,tests,tutorial]
   

[4] (1,2)

If using the (non-standard) zsh shell, note or recall that [...] is a glob operator, so the argument to pip must be quoted appropriately: pip install -e '.[docs,tests,tutorial]'.

At this point, installation is complete. Next, you can Check that installation was successful.

Install from GitHub

The above installs the latest release of MESSAGEix. If you are instead interested in installing a specific version of the code such as a branch of the message_ix GitHub repository, instead:

8. Run the following. Replace <ref> with a specific Git reference such as a branch name (for instance, the main development branch, or a branch associated with a pull request), a tag, or a commit hash:

   pip install git+ssh://git@github.com:iiasa/message_ix.git@<ref>[docs,report,tests,tutorial]
   

   git+ssh:// assumes that you use SSH to authenticate to GitHub, which we recommend. If you instead use personal access tokens, then run:

   pip install git+https://github.com/iiasa/message_ix.git@<ref>[docs,report,tests,tutorial]
   

At this point, installation is complete. Next, you can Check that installation was successful.

Install from a git clone of the source code

Note

If you want to install MESSAGEix from source, but already have an install from pip, please make sure to first pip uninstall message-ix. Otherwise, Python might not recognize your new install correctly. A symptom of this error is a message like “’message_ix’ has no attribute ‘Scenario’”.

8. Install ixmp, either also from source, or from PyPI. Use the same combination of major and minor versions: for instance, if installing message_ix version 3.9.x from source, install ixmp version 3.9.x.

9. (Optional) If you intend to contribute changes to MESSAGEix, first register a GitHub account, and fork the message_ix repository. This will create a new repository <user>/message_ix. (Please also see Contributing to development.)

10. Clone either the main repository, or your fork; using the Github Desktop client, or the command line:

    git clone git@github.com:iiasa/message_ix.git
    
    # or:
    git clone git@github.com:USER/message_ix.git
    

11. (Optional) If you cloned your fork, add the main repository as a remote git repository. This allows to stay up to date with changes there and to import tags, which also must be done for the install tests to succeed:

    git remote add upstream git@github.com:iiasa/message_ix.git
    git fetch upstream --tags
    

12. Navigate to the message_ix directory created by git clone. Run [4]:

    pip install --editable .[docs,report,tests,tutorial]
    

    The --editable flag ensures that changes to the source code are picked up every time import message_ix is used in Python code.

At this point, installation is complete. Next, you can Check that installation was successful.

Use conda

Note

An earlier version of the instructions from this section are available as a narrated video on the IIASA YouTube channel. If you are a beginner, you may want to watch the video before attempting the installation yourself.

5. Configure conda to install message_ix from the conda-forge channel:

   conda config --prepend channels conda-forge
   

6. Install and configure the mamba solver, which is faster and more reliable than conda’s default solver:

   conda install conda-libmamba-solver
   conda config --set solver libmamba
   

7. Create a new conda environment and activate it. This step is required if using Anaconda, but optional if using Miniconda. This example uses the name message_env, but you can use any name of your choice:

   conda create --name message_env
   conda activate message_env
   

8. Install the message-ix package into the current environment (either message_env, or another name from the previous step) [5]:

   conda install message-ix
   

At this point, installation is complete. Next, you can Check that installation was successful.

[5]

Notice that conda uses the hyphen (‘-’) in package names, different from the underscore (‘_’) used in Python when importing the package.

Note

When using Anaconda (not Miniconda), steps (5) through (8) can also be performed using the graphical Anaconda Navigator. See the Anaconda Navigator documentation for how to perform the various steps.

Check that installation was successful

Verify that the version installed corresponds to the latest release by running the following commands on the command line:

# Show versions of message_ix, ixmp, and key dependencies
message-ix show-versions

# Show the list of platforms (~databases) that have been configured
# and the path to the ixmp config file. By default, only the "local"
# platform, backed by a local database, should appear in the list
message-ix platform list

The above commands will work as of message_ix 3.0 and in subsequent versions. If an error occurs, this may mean that an older version has been installed unintentionally. To check the installed version directly:

# If installed using pip
pip show message-ix

# If installed using conda
conda list message-ix

For an install from source, it is possible to run the built-in test suite to check that MESSAGEix functions correctly on your system. This requires that the [tests] extra dependencies were installed. In the directory created by git clone, run:

pytest

Install R and reticulate

You only need to install R if you want to use message_ix and ixmp from R, rather than from Python.

First, install message_ix using one of the three methods above. Then:

1. Install R.

   Warning

   Ensure the the R version installed is either 32- or 64-bit (and >= 3.5.0), consistent with GAMS and Java. Having both 32- and 64-bit versions of R, or mixed 32- and 64-bit versions of different packages, can cause errors.

2. Install reticulate.

3. (Optional) Install IRkernel, which allows running R code in Jupyter notebooks (see the link for instructions).

Next:

• See Usage in R via reticulate for further details.

• If you installed message_ix from source, check that the R interface works by using the built-in test suite to run the R tutorial notebooks:

  $ pytest -m rmessageix
  

Common issues

If you run into an issue during installation that is not listed below, check the MESSAGEix issue tracker for an existing report, workaround, and/or solution.

“No JVM shared library file (jvm.dll) found”

Error messages like this when running message-ix --platform=default list or when creating a ixmp.Platform object (for instance, ixmp.Platform() in Python) indicate that message_ix (via ixmp and JPype) cannot find Java on your machine, in particular the Java Virtual Machine (JVM). There are multiple ways to resolve this issue:

1. If you have installed Java manually, ensure that the JAVA_HOME environment variable is set system-wide; see for example these instructions for Windows users.

2. If using Anaconda, install the openjdk package in the same environment as the message-ix package. When the Windows Anaconda Prompt is opened, conda activate then ensures the JAVA_HOME variable is correctly set.

To check which JVM will be used by ixmp, run the following in any prompt or terminal:

python -c "import jpype; print(jpype.getDefaultJVMPath())"

“No module named ‘pyam’”

The package pyam-iamc is one of the “report” extra dependencies of message_ix. These extra dependencies are not installed automatically, but can be installed using:

# If message_ix is installed using pip
pip install message_ix[report]
# or
pip install pyam-iamc

# If message_ix is installed using Anaconda (see note below)
conda install pyam

Note that this package has the different name on conda-forge versus PyPI: pyam.

The package listed as pyam on PyPI (and not available via Anaconda) is unrelated to message_ix, not compatible with it, and will produce other error messages. If you installed this package accidentally, remove it using:

# If installed using pip
pip uninstall pyam

Copy GAMS model files for editing

By default, the GAMS files containing the mathematical model core are installed with message_ix (e.g., in your Python site-packages directory). Many users will simply want to run MESSAGEix, or use the Python or R APIs to manipulate data, parameters and scenarios. For these uses, direct editing of the GAMS files is not necessary.

To edit the files directly—to change the mathematical formulation, such as adding new types of parameters, constraints, etc.—use the message-ix command-line program to copy the model files to a directory of your choice:

message-ix copy-model /path/for/model/files

You can also set the message model dir configuration key so that this copy of the files is used by default:

message-ix config set "message model dir" /path/for/model/files

…or do both in one step:

message-ix copy-model --set-default /path/for/model/files

Ignore local changes to .gms files

If you will be using MESSAGE_master.gms outside of the message_ix Python API to run MESSAGEix, you will likely modify this file, but will not want to commit these changes to Git. Set the Git “assume unchanged” bit for this file:

git update-index --assume-unchanged message_ix/model/MESSAGE_master.gms

To unset the bit, use --no-assume-unchanged. See the Git documentation for more details.