General purpose modeling tools
“Tools” can include, inter alia:
Codes for retrieving data from specific data sources and adapting it for use with
message_ix_models
.Codes for modifying scenarios; although tools for building models should go in
message_ix_models.model
.
On this page:
ADVANCE data (tools.advance
)
|
Return data from the ADVANCE Work Package 2 data snapshot at |
|
Return a single ADVANCE data variable as a |
- message_ix_models.tools.advance.LOCATION = ('advance', 'advance_compare_20171018-134445.csv.zip')
Expected location of the ADVANCE WP2 data snapshot.
This is a location relative to a parent directory.
The specific parent directory depends on whether message_data
is available:
- Without
message_data
: The code finds the data within (3) Other, system-specific (“local”) directories (see discussion there for how to configure this location). Users should:
Visit https://tntcat.iiasa.ac.at/ADVANCEWP2DB/dsd?Action=htmlpage&page=about and register for access to the data.
Log in.
Download the snapshot with the file name given in
LOCATION
to a subdirectoryadvance/
within their local data directory.
- With
message_data
: The code finds the data within (2) data/ directory in the message_data repo. The snapshot is stored directly in the repository using Git LFS.
Handle data from the ADVANCE project.
- message_ix_models.tools.advance.DIMS = ['model', 'scenario', 'region', 'variable', 'unit', 'year']
Standard dimensions for data produced as snapshots from the IIASA ENE Program “WorkDB”.
Todo
Move to a common location for use with other snapshots in the same format.
- message_ix_models.tools.advance.NAME = 'advance_compare_20171018-134445.csv'
Name of the data file within the archive.
- message_ix_models.tools.advance._fuzz_data(size=100.0, include: List[Tuple[str, str]] = [])[source]
Select a subset of the data for use in testing.
- message_ix_models.tools.advance._read_workdb_snapshot(path: Path, name: str) Series [source]
Read the data file.
The expected format is a ZIP archive at path containing a member at name in CSV format, with columns corresponding to
DIMS
, except for “year”, which is stored as column headers (‘wide’ format). (This corresponds to an older version of the “IAMC format,” without more recent additions intended to represent sub-annual time resolution using a separate column.)Todo
Move to a general location for use with other files in the same format.
Data returned by this function is cached using
cached()
; see alsoSKIP_CACHE
.
- message_ix_models.tools.advance.advance_data(variable: str, query: str | None = None) Quantity [source]
Return a single ADVANCE data variable as a
genno.Quantity
.- Parameters:
query (str, optional) – Passed to
get_advance_data()
.- Returns:
with the dimensions
DIMS
and name variable. If the units of the data for variable are consistent and parseable bypint
, the returned Quantity has these units; otherwise units are discarded and the returned Quantity is dimensionless.- Return type:
- message_ix_models.tools.advance.get_advance_data(query: str | None = None) Series [source]
Return data from the ADVANCE Work Package 2 data snapshot at
LOCATION
.- Parameters:
query (str, optional) – Passed to
pandas.DataFrame.query()
to limit the returned values.- Returns:
with a
pandas.MultiIndex
having the levelsDIMS
.- Return type:
Data returned by this function is cached using
cached()
; see alsoSKIP_CACHE
.