Storage back ends (ixmp.backend)

ixmp includes ixmp.backend.jdbc.JDBCBackend, which can store data in many types of relational database management systems (RDBMS) that have Java DataBase Connector (JDBC) interfaces—hence its name.

ixmp is extensible to support other methods of storing data: in non-JDBC RDBMS, non-relational databases, local files, memory, or other ways. Developers wishing to add such capabilities may subclass ixmp.backend.base.Backend and implement its methods.

Provided backends

ixmp.backend.BACKENDS: Dict[str, Type] = {'jdbc': <class 'ixmp.backend.jdbc.JDBCBackend'>}[source]

Mapping from names to available backends. To register additional backends, add entries to this dictionary.

class ixmp.backend.jdbc.JDBCBackend(jvmargs=None, **kwargs)[source]

Backend using JPype/JDBC to connect to Oracle and HyperSQL databases.

This backend is based on the third-party JPype Python package that allows interaction with Java code.

Parameters:
  • driver ('oracle' or 'hsqldb') – JDBC driver to use.

  • path (os.PathLike, optional) – Path to the HyperSQL database.

  • url (str, optional) – Partial or complete JDBC URL for the Oracle or HyperSQL database, e.g. database-server.example.com:PORT:SCHEMA. See Configuration.

  • user (str, optional) – Database user name.

  • password (str, optional) – Database user password.

  • cache (bool, optional) – If True (the default), cache Python objects after conversion from Java objects.

  • jvmargs (str, optional) – Java Virtual Machine arguments. See start_jvm().

  • dbprops (os.PathLike, optional) – With driver='oracle', the path to a database properties file containing driver, url, user, and password information.

JDBCBackend supports:

  • Databases in local files (HyperSQL) using driver='hsqldb' and the path argument.

  • Remote, Oracle databases using driver='oracle' and the url, username and password arguments.

  • Temporary, in-memory databases using driver='hsqldb' and the url argument. Use the url parameter with the format jdbc:hsqldb:mem:[NAME], where [NAME] is any string:

    mp = ixmp.Platform(
        backend="jdbc",
        driver="hsqldb",
        url="jdbc:hsqldb:mem:temporary platform",
    )
    

JDBCBackend caches values in memory to improve performance when repeatedly reading data from the same items with par(), equ(), or var().

Tip

If repeatedly accessing the same item with different filters:

  1. First, access the item by calling e.g. par() without any filters. This causes the full contents of the item to be loaded into cache.

  2. Then, access by making multiple par() calls with different filters arguments. The cache value is filtered and returned without further access to the database.

Tip

Modifying an item by adding or deleting elements invalidates its cache.

JDBCBackend has the following limitations:

JDBCBackend’s implementation allows the following kinds of file input and output:

read_file(path, item_type, **kwargs)

Read Platform, TimeSeries, or Scenario data from file.

write_file(path, item_type, **kwargs)

Write Platform, TimeSeries, or Scenario data to file.

classmethod handle_config(args, kwargs)[source]

Handle platform/backend config arguments.

args will overwrite any kwargs, and may be one of:

  • (“oracle”, url, user, password, [jvmargs]) for an Oracle database.

  • (“hsqldb”, path, [jvmargs]) for a file-backed HyperSQL database.

  • (“hsqldb”,) with “url” supplied via kwargs, e.g. “jdbc:hsqldb:mem://foo” for an in-memory database.

read_file(path, item_type: ItemType, **kwargs)[source]

Read Platform, TimeSeries, or Scenario data from file.

JDBCBackend supports reading from:

  • path='*.gdx', item_type=ItemType.MODEL. The keyword arguments check_solution, comment, equ_list, and var_list are required.

Parameters:
  • check_solution (bool) – If True, raise an exception if the GAMS solver did not reach optimality. (Only for MESSAGE-scheme Scenarios.)

  • comment (str) – Comment added to Scenario when importing the solution.

  • equ_list (list of str) – Equations to be imported.

  • var_list (list of str) – Variables to be imported.

  • filters (dict of dict of str) – Restrict items read.

write_file(path, item_type: ItemType, **kwargs)[source]

Write Platform, TimeSeries, or Scenario data to file.

JDBCBackend supports writing to:

  • path='*.gdx', item_type=ItemType.SET | ItemType.PAR.

  • path='*.csv', item_type=TS. The default keyword argument is required.

Parameters:

filters (dict of dict of str) –

Restrict items written. The following filters may be used:

  • model : str

  • scenario : str

  • variable : list of str

  • default : bool. If True, only data from TimeSeries versions with TimeSeries.set_as_default() are written.

ixmp.backend.jdbc.start_jvm(jvmargs=None)[source]

Start the Java Virtual Machine via JPype.

Parameters:

jvmargs (str or list of str, optional) –

Additional arguments for launching the JVM, passed to jpype.startJVM().

For instance, to set the maximum heap space to 4 GiB, give jvmargs=['-Xmx4G']. See the JVM documentation for a list of options.

Backend API

ixmp.backend.base.Backend()

Abstract base class for backends.

ixmp.backend.base.CachingBackend([cache_enabled])

Backend with additional features for caching data.

ixmp.backend.ItemType(value[, names, ...])

Type of data items in ixmp.TimeSeries and ixmp.Scenario.

ixmp.backend.FIELDS

Lists of field names for tuples returned by Backend API methods.

ixmp.backend.IAMC_IDX

Partial list of dimensions for the IAMC data structure, or “IAMC format”.

  • ixmp.Platform implements a user-friendly API for scientific programming. This means its methods can take many types of arguments, check, and transform them—in a way that provides modeler-users with easy, intuitive workflows.

  • In contrast, Backend has a very simple API that accepts arguments and returns values in basic Python data types and structures.

  • As a result:

    • Platform code is not affected by where and how data is stored; it merely handles user arguments and then makes, usually, a single Backend call.

    • Backend code does not need to perform argument checking; merely store and retrieve data reliably.

  • Additional Backends may inherit from Backend or CachingBackend.

class ixmp.backend.base.Backend[source]

Abstract base class for backends.

In the following, the bold-face words required, optional, etc. have specific meanings as described in IETF RFC 2119.

Backend is an abstract class; this means it must be subclassed. Most of its methods are decorated with abc.abstractmethod; this means they are required and must be overridden by subclasses.

Others, marked below with “OPTIONAL:”, are not so decorated. For these methods, the behaviour in the base Backend—often, nothing—is an acceptable default behaviour. Subclasses may extend or replace this behaviour as desired, so long as the methods still perform the actions described in the description.

Backends:

  • must only raise standard Python exceptions.

  • must implement the data model as described, or raise NotImplementedError for not implemented parts of the data model.

Methods related to ixmp.Platform:

add_model_name

Add (register) new model name.

add_scenario_name

Add (register) new scenario name.

close_db

OPTIONAL: Close database connection(s).

get_auth

OPTIONAL: Return user authorization for models.

get_doc

Read documentation from database

get_log_level

OPTIONAL: Get logging level for the backend and other code.

get_meta

Retrieve all metadata attached to a specific target.

get_model_names

List existing model names.

get_nodes

Iterate over all nodes stored on the Platform.

get_scenarios

Iterate over TimeSeries stored on the Platform.

get_scenario_names

List existing scenario names.

get_units

Return all registered symbols for units of measurement.

handle_config

OPTIONAL: Handle platform/backend config arguments.

open_db

OPTIONAL: (Re-)open database connection(s).

read_file

OPTIONAL: Read Platform, TimeSeries, or Scenario data from file.

remove_meta

Remove metadata attached to a target.

set_doc

Save documentation to database

set_log_level

OPTIONAL: Set logging level for the backend and other code.

set_meta

Set metadata on a target.

set_node

Add a node name to the Platform.

set_unit

Add a unit of measurement to the Platform.

write_file

OPTIONAL: Write Platform, TimeSeries, or Scenario data to file.

Methods related to ixmp.TimeSeries:

  • Each method has an argument ts, a reference to the TimeSeries object being manipulated.

  • ‘Geodata’ is otherwise identical to regular timeseries data, except value are str rather than float.

check_out

Check out ts for modification.

commit

Commit changes to ts.

delete

Remove time series data.

delete_geo

Remove 'geodata' values.

discard_changes

Discard changes to ts since the last check_out().

get

Retrieve the existing TimeSeries (or Scenario) ts.

get_data

Retrieve time series data.

get_geo

Retrieve time-series 'geodata'.

init

Create a new TimeSeries (or Scenario) ts.

is_default

Return True if ts is the default version for its (model, scenario).

last_update

Return the date of the last modification of the ts, if any.

preload

OPTIONAL: Load ts data into memory.

run_id

Return the run ID of the ts.

set_data

Store data.

set_as_default

Set the current TimeSeries.version as the default.

set_geo

Store time series geodata.

Methods related to ixmp.Scenario:

  • Each method has an argument s, a reference to the Scenario object being manipulated.

clone

Clone s.

delete_item

Remove an item name of type.

get_meta

Retrieve all metadata attached to a specific target.

has_solution

Return True if Scenario s has been solved.

init_item

Initialize an item name of type.

item_delete_elements

Remove elements of item name.

item_get_elements

Return elements of item name.

item_set_elements

Add keys or values to item name.

item_index

Return the index sets or names of item name.

list_items

Return a list of names of items of type.

remove_meta

Remove metadata attached to a target.

set_meta

Set metadata on a target.

Methods related to message_ix.Scenario:

  • Each method has an argument ms, a reference to the Scenario object being manipulated.

Warning

These methods may be moved to ixmp in a future release.

cat_get_elements

Get elements of a category mapping.

cat_list

Return list of categories in mapping name.

cat_set_elements

Add elements to category mapping.

abstract add_model_name(name: str) None[source]

Add (register) new model name.

Parameters:

name (str) – New model name

abstract add_scenario_name(name: str) None[source]

Add (register) new scenario name.

Parameters:

name (str) – New scenario name

abstract cat_get_elements(ms: Scenario, name: str, cat: str) List[str][source]

Get elements of a category mapping.

Parameters:
  • name (str) – Name of the category mapping set.

  • cat (str) – Name of the category within name.

Returns:

All set elements mapped to cat in name.

Return type:

list of str

abstract cat_list(ms: Scenario, name: str) List[str][source]

Return list of categories in mapping name.

Parameters:

name (str) – Name of the category mapping set.

Returns:

All categories in name.

Return type:

list of str

abstract cat_set_elements(ms: Scenario, name: str, cat: str, keys: str | Sequence[str], is_unique: bool) None[source]

Add elements to category mapping.

Parameters:
  • name (str) – Name of the category mapping set.

  • cat (str) – Name of the category within name.

  • keys (collections.abc.Iterable of str or list of str) – Keys to add to cat.

  • is_unique (bool) –

    If True:

    • keys must contain only one key.

    • The Backend must remove any existing member of cat, so that it has only one element.

abstract check_out(ts: TimeSeries, timeseries_only: bool) None[source]

Check out ts for modification.

Parameters:

timeseries_only (bool) –

???

abstract clear_solution(s: Scenario, from_year=None)[source]

Remove data associated with a model solution.

Todo

Document.

abstract clone(s: Scenario, platform_dest: Platform, model: str, scenario: str, annotation: str, keep_solution: bool, first_model_year: int | None = None) Scenario[source]

Clone s.

Parameters:
  • platform_dest (Platform) – Target backend. May be the same as s.platform.

  • model (str) – New model name.

  • scenario (str) – New scenario name.

  • annotation (str) – Description for the creation of the new scenario.

  • keep_solution (bool) – If True, model solution data is also cloned. If False, it is discarded.

  • first_model_year (int or None) – If int, must be greater than the first model year of s.

Returns:

The cloned Scenario. If s is an instance of a subclass of ixmp.Scenario, the returned object must be of the same subclass.

Return type:

Scenario

close_db() None[source]

OPTIONAL: Close database connection(s).

Close any database connection(s), if open.

abstract commit(ts: TimeSeries, comment: str) None[source]

Commit changes to ts.

ts_init may modify the version attribute of ts.

Parameters:

comment (str) – Description of the changes being committed.

del_ts(ts: TimeSeries) None[source]

OPTIONAL: Free memory associated with the TimeSeries ts.

The default implementation has no effect.

abstract delete(ts: TimeSeries, region: str, variable: str, subannual: str, years: Iterable[int], unit: str) None[source]

Remove time series data.

Parameters:
  • region (str) – Region name.

  • variable (str) – Variable name.

  • years (Iterable of int) – Years.

  • unit (str) – Unit symbol.

  • subannual (str) – Name of time slice.

abstract delete_geo(ts: TimeSeries, region: str, variable: str, subannual: str, years: Iterable[int], unit: str) None[source]

Remove ‘geodata’ values.

Parameters:
  • region (str) – Region name.

  • variable (str) – Variable name.

  • subannual (str) – Name of time slice.

  • years (Iterable of int) – Years.

  • unit (str) – Unit symbol.

abstract delete_item(s: Scenario, type: str, name: str) None[source]

Remove an item name of type.

Parameters:
  • type ('set' or 'par' or 'equ') –

  • name (str) – Name of the item to delete.

abstract discard_changes(ts: TimeSeries) None[source]

Discard changes to ts since the last check_out().

abstract get(ts: TimeSeries) None[source]

Retrieve the existing TimeSeries (or Scenario) ts.

The TimeSeries is identified based on the unique combination of the attributes of ts:

If version is None, the Backend must return the version marked as default, and must set the attribute value.

If ts is a Scenario, get() must set the scheme attribute with the value previously passed to init().

Raises:

ValueError – If model or scenario does not exist on the Platform.

get_auth(user: str, models: Sequence[str], kind: str) Dict[str, bool][source]

OPTIONAL: Return user authorization for models.

If the Backend implements access control, this method must indicate whether user has permission kind for each of models.

kind may be ‘read’/’view’, ‘write’/’modify’, or other values; get_auth() should raise exceptions on invalid values.

Parameters:
  • user (str) – User name or identifier.

  • models (list of str) – Model names.

  • kind (str) – Type of permission being requested

Returns:

Mapping of model name (str)bool; True if the user is authorized for the model.

Return type:

dict

abstract get_data(ts: TimeSeries, region: Sequence[str], variable: Sequence[str], unit: Sequence[str], year: Sequence[str]) Iterable[Tuple[str, str, str, int, float]][source]

Retrieve time series data.

Parameters:
  • region (list of str) – Region names to filter results.

  • variable (list of str) – Variable names to filter results.

  • unit (list of str) – Unit symbols to filter results.

  • year (list of str) – Years to filter results.

Yields:

tuple – The members of each tuple are:

ID

Type

Description

region

str

Region name

variable

str

Variable name

unit

str

Unit symbol

year

int

Year

value

float

Data value

abstract get_doc(domain: str, name: str | None = None) str | Dict[source]

Read documentation from database

Parameters:
  • domain (str) – Documentation domain, e.g. model, scenario etc

  • name (str, optional) – Name of domain entity (e.g. model name).

Returns:

String representing fragment of documentation if name is passed as parameter or dictionary containing mapping between name of domain object (e.g. model name) and string representing fragment when name parameter is omitted.

Return type:

str or dict

abstract get_geo(ts: TimeSeries) Iterable[Tuple[str, str, int, str, str, str, bool]][source]

Retrieve time-series ‘geodata’.

Yields:

tuple – The members of each tuple are:

ID

Type

Description

region

str

Region name

variable

str

Variable name

year

int

Year

value

str

Value

unit

str

Unit symbol

subannual

str

Name of time slice

meta

bool

True if the data is marked as metadata

get_log_level() str[source]

OPTIONAL: Get logging level for the backend and other code.

The default implementation returns the effective level of the “ixmp.backend.base” logger; usually the same as “ixmp” or “ixmp.backend” (if set).

Returns:

Name of a Python logging level.

Return type:

str

See also

set_log_level

abstract get_meta(model: str | None, scenario: str | None, version: int | None, strict: bool) Dict[str, Any][source]

Retrieve all metadata attached to a specific target.

Depending on which of model, scenario, version are None, metadata attached to one of the four kinds of metadata targets (see Metadata) is returned.

If strict is False, then get_meta() must also return metadata attached to less specific or “higher level” targets:

  • For (model, scenario, version), these are (model, scenario); (model,); and (scenario).

  • For (model, scenario), these are (model,) and (scenario,).

  • For (model,) or (scenario,), there are no less specific targets.

Parameters:
  • model (str, optional) – Model name of metadata target.

  • scenario (str, optional) – Scenario name of metadata target.

  • version (int, optional) – TimeSeries.version of metadata target.

  • strict (bool) – Only retrieve metadata from the specified target.

Returns:

Mapping from metadata names/identifiers (str) to values (Any).

Return type:

dict

Raises:

ValueError – on unsupported (model, scenario, version) combinations.

abstract get_model_names() Iterable[str][source]

List existing model names.

Returns:

List of the retrieved model names.

Return type:

list of str

abstract get_nodes() Iterable[Tuple[str, str | None, str, str]][source]

Iterate over all nodes stored on the Platform.

Yields:

tuple – The members of each tuple are:

ID

Type

Description

region

str

Node name or synonym for node

mapped_to

str or None

Node name

parent

str

Parent node name

hierarchy

str

Node hierarchy ID

See also

set_node

abstract get_scenario_names() Iterable[str][source]

List existing scenario names.

Returns:

List of the retrieved scenario names.

Return type:

list of str

abstract get_scenarios(default: bool, model: str | None, scenario: str | None) Iterable[Tuple[str, str, str, bool, bool, str, str, str, str, str, str, str, int]][source]

Iterate over TimeSeries stored on the Platform.

Scenarios, as subclasses of TimeSeries, are also included.

Parameters:
  • default (bool) – True to include only TimeSeries versions marked as default.

  • model (str or None) – Model name to filter results.

  • scenario (str or None) – Scenario name to filter results.

Yields:

tuple – The members of each tuple are:

ID

Type

Description

model

str

Model name

scenario

str

Scenario name

scheme

str

Scheme name

is_default

bool

True if version is the default

is_locked

bool

True if read-only

cre_user

str

Name of user who created the TimeSeries

cre_date

str

Creation datetime

upd_user

str

Name of user who last modified the TimeSeries

upd_date

str

Modification datetime

lock_user

str

Name of user who locked the TimeSeries

lock_date

str

Lock datetime

annotation

str

Description of the TimeSeries

version

int

Version

abstract get_timeslices() Iterable[Tuple[str, str, float]][source]

Iterate over subannual timeslices defined on the Platform instance.

Yields:

tuple – The members of each tuple are:

ID

Type

Description

name

str

Time slice name

category

str

Time slice category

duration

float

Time slice duration (fraction of year)

See also

set_timeslice

abstract get_units() List[str][source]

Return all registered symbols for units of measurement.

Return type:

list of str

See also

set_unit

classmethod handle_config(args: Sequence, kwargs: MutableMapping) Dict[str, Any][source]

OPTIONAL: Handle platform/backend config arguments.

Returns a dict to be stored in the configuration file. This dict must be valid as keyword arguments to the __init__() method of a Backend subclass.

The default implementation expects both args and kwargs to be empty.

abstract has_solution(s: Scenario) bool[source]

Return True if Scenario s has been solved.

If True, model solution data is available from the Backend.

abstract init(ts: TimeSeries, annotation: str) None[source]

Create a new TimeSeries (or Scenario) ts.

init may modify the version attribute of ts.

If ts is a Scenario; the Backend must store the Scenario.scheme attribute.

Parameters:

annotation (str) – If ts is newly-created, the Backend must store this annotation with the TimeSeries.

abstract init_item(s: Scenario, type: str, name: str, idx_sets: Sequence[str], idx_names: Sequence[str] | None) None[source]

Initialize an item name of type.

Parameters:
  • type ('set' or 'par' or 'equ' or 'var') –

  • name (str) – Name for the new item.

  • idx_sets (collections.abc.Sequence of str) – If empty, a 0-dimensional/scalar item is initialized. Otherwise, a 1+-dimensional item is initialized.

  • idx_names (collections.abc.Sequence of str or None) – Optional names for the dimensions. If not supplied, the names of the idx_sets (if any) are used. If supplied, idx_names and idx_sets must be the same length.

Raises:

ValueError – if any of the idx_sets is not an existing set in the Scenario; if idx_names and idx_sets are not the same length.

abstract is_default(ts: TimeSeries) bool[source]

Return True if ts is the default version for its (model, scenario).

See also

get, set_as_default

abstract item_delete_elements(s: Scenario, type: str, name: str, keys) None[source]

Remove elements of item name.

Parameters:
  • type ('par' or 'set') –

  • name (str) –

  • keys (collections.abc.Iterable of collections.abc.Iterable of str) – If name is indexed by other set(s), then the number of elements of each key in keys, and their contents, must match the index set(s). If name is a basic set, then each key must be a list containing a single str, which must exist in the set.

abstract item_get_elements(s: Scenario, type: Literal['equ', 'par', 'set', 'var'], name: str, filters: Dict[str, List[Any]] | None = None) Dict[str, Any] | Series | DataFrame[source]

Return elements of item name.

Parameters:
  • type (str) – Type of the item.

  • name (str) – Name of the item.

  • filters (dict, optional) –

    If provided, a mapping from dimension names (class:str) to allowed values along that dimension (list).

    item_get_elements must silently accept values that are not members of the set indexing a dimension. Elements which are not str must be handled as equivalent to their string representation; that is, item_get_elements must return the same data for filters={'foo': [42]} and filters={'foo': ['42']}.

Returns:

  • pandas.Series – When type is ‘set’ and name an index set (not indexed by other sets).

  • dict – When type is ‘equ’, ‘par’, or ‘var’ and name is scalar (zero- dimensional). The value has the keys ‘value’ and ‘unit’ (for ‘par’) or ‘lvl’ and ‘mrg’ (for ‘equ’ or ‘var’).

  • pandas.DataFrame – For mapping sets, or all 1+-dimensional values. The dataframe has one column per index name with dimension values; plus the columns ‘value’ and ‘unit’ (for ‘par’) or ‘lvl’ and ‘mrg’ (for ‘equ’ or ‘var’).

Raises:

KeyError – If name does not exist in s.

abstract item_index(s: Scenario, name: str, sets_or_names: str) List[str][source]

Return the index sets or names of item name.

Parameters:

sets_or_names ('sets' or 'names') –

Return type:

list of str

abstract item_set_elements(s: Scenario, type: str, name: str, elements: Iterable[Tuple[Any, float | None, str | None, str | None]]) None[source]

Add keys or values to item name.

Parameters:
  • type ('par' or 'set') –

  • name (str) – Name of the items.

  • elements (collections.abc.Iterable of tuple) –

    The members of each tuple are:

    ID

    Type

    Description

    key

    str or list of str or None

    Set elements or value indices

    value

    float or None

    Parameter value

    unit

    str or None

    Unit symbol

    comment

    str or None

    Description of the change

    If name is indexed by other set(s), then the number of elements of each key, and their contents, must match the index set(s). When type is ‘set’, value and unit must be None.

Raises:
  • ValueError – If elements contain invalid values, e.g. key values not in the index set(s).

  • Exception – If the Backend encounters any error adding the elements.

abstract last_update(ts: TimeSeries) str | None[source]

Return the date of the last modification of the ts, if any.

abstract list_items(s: Scenario, type: str) List[str][source]

Return a list of names of items of type.

Parameters:

type ('set' or 'par' or 'equ') –

open_db() None[source]

OPTIONAL: (Re-)open database connection(s).

A backend may connect to a database server. This method opens the database connection if it is closed.

preload(ts: TimeSeries) None[source]

OPTIONAL: Load ts data into memory.

read_file(path: PathLike, item_type: ItemType, **kwargs) None[source]

OPTIONAL: Read Platform, TimeSeries, or Scenario data from file.

A backend may implement read_file for one or more combinations of the path and item_type methods. For all other combinations, it must raise NotImplementedError.

The default implementation supports:

  • path ending in ‘.xlsx’, item_type is ItemType.MODEL: read a single Scenario given by kwargs['filters']['scenario'] from file, using s_read_excel().

Parameters:
  • path (os.PathLike) –

    File for input. The filename suffix determines the input format:

    Suffix

    Format

    .csv

    Comma-separated values

    .gdx

    GAMS data exchange

    .xlsx

    Microsoft Office Open XML spreadsheet

  • item_type (ItemType) – Type(s) of items to read.

Raises:
  • ValueError – If ts is not None and ‘scenario’ is a key in filters.

  • NotImplementedError – If input of the specified items from the file format is not supported.

See also

write_file

abstract remove_meta(names: list, model: str | None, scenario: str | None, version: int | None) None[source]

Remove metadata attached to a target.

Parameters:
  • names (list of str) – Metadata names/identifiers to remove.

  • model (str or None) – Model name of metadata target.

  • scenario (str or None) – Scenario name of metadata target.

  • version (int or None) – TimeSeries.version of metadata target.

Raises:

ValueError – on unsupported (model, scenario, version) combinations.

See also

get_meta

abstract run_id(ts: TimeSeries) int[source]

Return the run ID of the ts.

abstract set_as_default(ts: TimeSeries) None[source]

Set the current TimeSeries.version as the default.

See also

get, is_default

abstract set_data(ts: TimeSeries, region: str, variable: str, data: Dict[int, float], unit: str, subannual: str, meta: bool) None[source]

Store data.

Parameters:
  • region (str) – Region name.

  • variable (str) – Variable name.

  • subannual (str) – Name of time slice.

  • unit (str) – Unit symbol.

  • data – Mapping from year (int) to value (float).

  • meta (bool) – True to mark data as metadata.

abstract set_doc(domain: str, docs) None[source]

Save documentation to database

Parameters:
  • domain (str) – Documentation domain, e.g. model, scenario, etc.

  • docs (dict or collections.abc.Iterable of tuple) – Dictionary or tuple array containing mapping between name of domain object (e.g. model name) and string representing fragment of documentation.

abstract set_geo(ts: TimeSeries, region: str, variable: str, subannual: str, year: int, value: str, unit: str, meta: bool) None[source]

Store time series geodata.

Parameters:
  • region (str) – Region name.

  • variable (str) – Variable name.

  • subannual (str) – Name of time slice.

  • year (int) – Year.

  • value (str) – Data value.

  • unit (str) – Unit symbol.

  • meta (bool) – True to mark data as metadata.

set_log_level(level: int) None[source]

OPTIONAL: Set logging level for the backend and other code.

The default implementation has no effect.

Parameters:

level (int) – A Python logging level.

See also

get_log_level

abstract set_meta(meta: dict, model: str | None, scenario: str | None, version: int | None) None[source]

Set metadata on a target.

Parameters:
  • meta (dict) – Mapping from metadata names/identifiers to values.

  • model (str or None) – Model name of metadata target.

  • scenario (str or None) – Scenario name of metadata target.

  • version (int or None) – TimeSeries.version of metadata target.

Raises:

ValueError – on unsupported (model, scenario, version) combinations.

See also

get_meta

abstract set_node(name: str, parent: str | None = None, hierarchy: str | None = None, synonym: str | None = None) None[source]

Add a node name to the Platform.

This method must have one of two effects, depending on the arguments:

  • With parent and hierarchy: name is added as a child of parent in the named hierarchy.

  • With synonym: synonym is added as an alias for name.

Parameters:
  • name (str) – Node name.

  • parent (str, optional) – Parent node name.

  • hierarchy (str, optional) – Node hierarchy ID.

  • synonym (str, optional) – Synonym for node.

See also

get_nodes

abstract set_timeslice(name: str, category: str, duration: float) None[source]

Add a subannual time slice to the Platform.

Parameters:
  • name (str) – Node name.

  • category (str) – Time slice category.

  • duration (float) – Time slice duration (a fraction of a year).

See also

get_timeslices

abstract set_unit(name: str, comment: str) None[source]

Add a unit of measurement to the Platform.

Parameters:
  • name (str) – Symbol of the unit.

  • comment (str) – Description of the change or of the unit.

See also

get_units

write_file(path: PathLike, item_type: ItemType, **kwargs) None[source]

OPTIONAL: Write Platform, TimeSeries, or Scenario data to file.

A backend may implement write_file for one or more combinations of the path and item_type methods. For all other combinations, it must raise NotImplementedError.

The default implementation supports:

  • path ending in ‘.xlsx’, item_type is either MODEL or SET | PAR: write a single Scenario given by kwargs['filters']['scenario'] to file using s_write_excel().

Parameters:
  • path (os.PathLike) – File for output. The filename suffix determines the output format.

  • item_type (ItemType) – Type(s) of items to write.

Raises:
  • ValueError – If ts is not None and ‘scenario’ is a key in filters.

  • NotImplementedError – If output of the specified items to the file format is not supported.

See also

read_file

class ixmp.backend.base.CachingBackend(cache_enabled=True)[source]

Backend with additional features for caching data.

CachingBackend stores cache values for multiple TimeSeries/Scenario objects, and for multiple values of a filters argument.

Subclasses must call cache(), cache_get(), and cache_invalidate() as appropriate to manage the cache; CachingBackend does not enforce any such logic.

_cache: Dict[Tuple, object] = {}[source]

Cache of values. Keys are given by _cache_key(); values depend on the subclass’ usage of the cache.

_cache_hit: Dict[Tuple, int] = {}[source]

Count of number of times a value was retrieved from cache successfully using cache_get().

classmethod _cache_key(ts: TimeSeries, ix_type: str | None, name: str | None, filters: Dict[str, Hashable] | None = None) Tuple[Hashable, ...][source]

Return a hashable cache key.

ixmp filters (a dict of list) are converted to a unique id that is hashable.

Returns:

A hashable key with 4 elements for ts, ix_type, name, and filters.

Return type:

tuple

cache(ts: TimeSeries, ix_type: str, name: str, filters: Dict, value: Any) bool[source]

Store value in cache.

Returns:

True if the key was already in the cache and its value was overwritten.

Return type:

bool

cache_enabled = True[source]

True if caching is enabled.

cache_get(ts: TimeSeries, ix_type: str, name: str, filters: Dict) Any | None[source]

Retrieve value from cache.

The value in _cache is copied to avoid cached values being modified by user code. _cache_hit is incremented.

Raises:

KeyError – If the key for ts, ix_type, name and filters is not in the cache.

cache_invalidate(ts: TimeSeries, ix_type: str | None = None, name: str | None = None, filters: Dict | None = None) None[source]

Invalidate cached values.

With all arguments given, single key/value is removed from the cache. Otherwise, multiple keys/values are removed:

  • ts only: all cached values associated with the TimeSeries or Scenario object.

  • ts, ix_type, and name: all cached values associated with the item, whether filtered or unfiltered.

del_ts(ts: TimeSeries)[source]

Invalidate cache entries associated with ts.

Backend API.

ixmp.backend.FIELDS = {'get_nodes': ('region', 'mapped_to', 'parent', 'hierarchy'), 'get_scenarios': ('model', 'scenario', 'scheme', 'is_default', 'is_locked', 'cre_user', 'cre_date', 'upd_user', 'upd_date', 'lock_user', 'lock_date', 'annotation', 'version'), 'get_timeslices': ('name', 'category', 'duration'), 'ts_get': ('region', 'variable', 'unit', 'subannual', 'year', 'value'), 'ts_get_geo': ('region', 'variable', 'subannual', 'year', 'value', 'unit', 'meta'), 'write_file': ('MODEL', 'SCENARIO', 'VERSION', 'VARIABLE', 'UNIT', 'REGION', 'META', 'SUBANNUAL', 'YEAR', 'VALUE')}[source]

Lists of field names for tuples returned by Backend API methods.

The key “write_file” refers to the columns appearing in the CSV output from export_timeseries_data() when using JDBCBackend.

Todo

Make this consistent with other dimension orders and with IAMC_IDX.

ixmp.backend.IAMC_IDX: List[str | int] = ['model', 'scenario', 'region', 'variable', 'unit'][source]

Partial list of dimensions for the IAMC data structure, or “IAMC format”. This omits “year” and “subannual” which appear in some variants of the structure, but not in others.

class ixmp.backend.ItemType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]

Type of data items in ixmp.TimeSeries and ixmp.Scenario.

TS = 1[source]
T = 1[source]

Time series data variable.

SET = 2[source]
S = 2[source]

Set.

PAR = 4[source]
P = 4[source]

Parameter.

VAR = 8[source]
V = 8[source]

Model variable.

EQU = 16[source]
E = 16[source]

Equation.

MODEL = 30[source]
M = 30[source]

All kinds of model-related data, i.e. SET, PAR, VAR and EQU.

SOLUTION = 24[source]

Model solution data, i.e. VAR and EQU.

ALL = 31[source]
A = 31[source]

All data, i.e. MODEL and TS.

Common input/output routines for backends

ixmp.backend.io.EXCEL_MAX_ROWS = 1048576[source]

Maximum number of rows supported by the Excel file format. See to_excel() and Scenario/model data.

ixmp.backend.io.maybe_init_item(scenario, ix_type, name, new_idx, path)[source]

Call init_set(), init_par(), etc. if possible.

Logs an intelligible warning and then raises ValueError in two cases:

  • the new_idx is ambiguous, e.g. containing index names that cannot be used to infer index sets, or

  • an existing item has index names that are different from new_idx.

ixmp.backend.io.s_read_excel(be, s, path, add_units=False, init_items=False, commit_steps=False)[source]

Read data from a Microsoft Excel file at path into s.

ixmp.backend.io.s_write_excel(be, s, path, item_type, filters=None, max_row=None)[source]

Write s to a Microsoft Excel file at path.

ixmp.backend.io.ts_read_file(ts, path, firstyear=None, lastyear=None)[source]

Read data from a CSV or Microsoft Excel file at path into ts.