General purpose modeling tools

“Tools” can include, inter alia:

  • Codes for retrieving data from specific data sources and adapting it for use with message_ix_models.

  • Codes for modifying scenarios; although tools for building models should go in message_ix_models.model.

On this page:

ADVANCE data (tools.advance)

get_advance_data([query])

Return data from the ADVANCE Work Package 2 data snapshot at LOCATION.

advance_data(variable[, query])

Return a single ADVANCE data variable as a genno.Quantity.
message_ix_models.tools.advance.LOCATION = ('advance', 'advance_compare_20171018-134445.csv.zip')

Expected location of the ADVANCE WP2 data snapshot.

This is a location relative to a parent directory. The specific parent directory depends on whether message_data is available:

Without message_data:

The code finds the data within (3) Other, system-specific (“local”) directories (see discussion there for how to configure this location). Users should:

  1. Visit https://tntcat.iiasa.ac.at/ADVANCEWP2DB/dsd?Action=htmlpage&page=about and register for access to the data.

  2. Log in.

  3. Download the snapshot with the file name given in LOCATION to a subdirectory advance/ within their local data directory.

With message_data:

The code finds the data within (2) data/ directory in the message_data repo. The snapshot is stored directly in the repository using Git LFS.

Handle data from the ADVANCE project.

message_ix_models.tools.advance.DIMS = ['model', 'scenario', 'region', 'variable', 'unit', 'year']

Standard dimensions for data produced as snapshots from the IIASA ENE Program “WorkDB”.

Todo

Move to a common location for use with other snapshots in the same format.

message_ix_models.tools.advance.NAME = 'advance_compare_20171018-134445.csv'

Name of the data file within the archive.

message_ix_models.tools.advance._fuzz_data(size=100.0, include: List[Tuple[str, str]] = [])[source]

Select a subset of the data for use in testing.

Parameters:

  • size (numeric) – Number of rows to include.

  • include (sequence of 2-tuple (str, str)) – (variable name, unit) to include. The data will be partly duplicated to ensure the given variable name(s) are included.

message_ix_models.tools.advance._read_workdb_snapshot(path: Path, name: str) → Series[source]

Read the data file.

The expected format is a ZIP archive at path containing a member at name in CSV format, with columns corresponding to DIMS, except for “year”, which is stored as column headers (‘wide’ format). (This corresponds to an older version of the “IAMC format,” without more recent additions intended to represent sub-annual time resolution using a separate column.)

Todo

Move to a general location for use with other files in the same format.

Data returned by this function is cached using cached(); see also SKIP_CACHE.

message_ix_models.tools.advance.advance_data(variable: str, query: str | None = None) → Quantity[source]

Return a single ADVANCE data variable as a genno.Quantity.

Parameters:

query (str, optional) – Passed to get_advance_data().

Returns:

with the dimensions DIMS and name variable. If the units of the data for variable are consistent and parseable by pint, the returned Quantity has these units; otherwise units are discarded and the returned Quantity is dimensionless.

Return type:

genno.Quantity

message_ix_models.tools.advance.get_advance_data(query: str | None = None) → Series[source]

Return data from the ADVANCE Work Package 2 data snapshot at LOCATION.

Parameters:

query (str, optional) – Passed to pandas.DataFrame.query() to limit the returned values.

Returns:

with a pandas.MultiIndex having the levels DIMS.

Return type:

pandas.Series

Data returned by this function is cached using cached(); see also SKIP_CACHE.