Storage back ends (ixmp.backend)

ixmp includes ixmp.backend.jdbc.JDBCBackend, which can store data in many types of relational database management systems (RDBMS) that have Java DataBase Connector (JDBC) interfaces—hence its name.

ixmp is extensible to support other methods of storing data: in non-JDBC RDBMS, non-relational databases, local files, memory, or other ways. Developers wishing to add such capabilities may subclass ixmp.backend.base.Backend and implement its methods.

Provided backends

ixmp.backend.BACKENDS: Dict[str, Type] = {'jdbc': <class 'ixmp.backend.jdbc.JDBCBackend'>}

Mapping from names to available backends. To register additional backends, add entries to this dictionary.

class ixmp.backend.jdbc.JDBCBackend(jvmargs=None, **kwargs)

Backend using JPype/JDBC to connect to Oracle and HyperSQL databases.

Parameters
  • driver ('oracle' or 'hsqldb') – JDBC driver to use.

  • path (path-like, optional) – Path to the HyperSQL database.

  • url (str, optional) – Partial or complete JDBC URL for the Oracle or HyperSQL database, e.g. database-server.example.com:PORT:SCHEMA. See Configuration.

  • user (str, optional) – Database user name.

  • password (str, optional) – Database user password.

  • cache (bool, optional) – If True (the default), cache Python objects after conversion from Java objects.

  • jvmargs (str, optional) – Java Virtual Machine arguments. See start_jvm().

  • dbprops (path-like, optional) – With driver='oracle', the path to a database properties file containing driver, url, user, and password information.

JDBCBackend supports:

  • Databases in local files (HyperSQL) using driver='hsqldb' and the path argument.

  • Remote, Oracle databases using driver='oracle' and the url, username and password arguments.

  • Temporary, in-memory databases using driver='hsqldb' and the url argument. Use the url parameter with the format jdbc:hsqldb:mem:[NAME], where [NAME] is any string:

    mp = ixmp.Platform(
        backend="jdbc",
        driver="hsqldb",
        url="jdbc:hsqldb:mem:temporary platform",
    )
    

JDBCBackend caches values in memory to improve performance when repeatedly reading data from the same items with par(), equ(), or var().

Tip

If repeatedly accessing the same item with different filters:

  1. First, access the item by calling e.g. par() without any filters. This causes the full contents of the item to be loaded into cache.

  2. Then, access by making multiple par() calls with different filters arguments. The cache value is filtered and returned without further access to the database.

Tip

Modifying an item by adding or deleting elements invalidates its cache.

JDBCBackend has the following limitations:

  • The comment argument to Platform.add_unit() is limited to 64 characters.

JDBCBackend’s implementation allows the following kinds of file input and output:

read_file(path, item_type, **kwargs)

Read Platform, TimeSeries, or Scenario data from file.

write_file(path, item_type, **kwargs)

Write Platform, TimeSeries, or Scenario data to file.

read_file(path, item_type: ixmp.backend.ItemType, **kwargs)

Read Platform, TimeSeries, or Scenario data from file.

JDBCBackend supports reading from:

  • path='*.gdx', item_type=ItemType.MODEL. The keyword arguments check_solution, comment, equ_list, and var_list are required.

Parameters
  • check_solution (bool) – If True, raise an exception if the GAMS solver did not reach optimality. (Only for MESSAGE-scheme Scenarios.)

  • comment (str) – Comment added to Scenario when importing the solution.

  • equ_list (list of str) – Equations to be imported.

  • var_list (list of str) – Variables to be imported.

  • filters (dict of dict of str) – Restrict items read.

write_file(path, item_type: ixmp.backend.ItemType, **kwargs)

Write Platform, TimeSeries, or Scenario data to file.

JDBCBackend supports writing to:

  • path='*.gdx', item_type=ItemType.SET | ItemType.PAR.

  • path='*.csv', item_type=TS. The default keyword argument is required.

Parameters

filters (dict of dict of str) –

Restrict items written. The following filters may be used:

  • model : str

  • scenario : str

  • variable : list of str

  • default : bool. If True, only data from TimeSeries versions with set_default() are written.

jdbc.start_jvm()

Start the Java Virtual Machine via JPype.

Parameters

jvmargs (str or list of str, optional) –

Additional arguments for launching the JVM, passed to jpype.startJVM().

For instance, to set the maximum heap space to 4 GiB, give jvmargs=['-Xmx4G']. See the JVM documentation for a list of options.

Backend API

ixmp.backend.base.Backend()

Abstract base class for backends.

ixmp.backend.base.CachingBackend([cache_enabled])

Backend with additional features for caching data.

ixmp.backend.ItemType(value)

Type of data items in TimeSeries and Scenario.

ixmp.backend.FIELDS

Lists of field names for tuples returned by Backend API methods.

  • ixmp.Platform implements a user-friendly API for scientific programming. This means its methods can take many types of arguments, check, and transform them—in a way that provides modeler-users with easy, intuitive workflows.

  • In contrast, Backend has a very simple API that accepts arguments and returns values in basic Python data types and structures.

  • As a result:

    • Platform code is not affected by where and how data is stored; it merely handles user arguments and then makes, usually, a single Backend call.

    • Backend code does not need to perform argument checking; merely store and retrieve data reliably.

  • Additional Backends may inherit from Backend or CachingBackend.

class ixmp.backend.base.Backend

Abstract base class for backends.

In the following, the bold-face words required, optional, etc. have specific meanings as described in IETF RFC 2119.

Backend is an abstract class; this means it must be subclassed. Most of its methods are decorated with abc.abstractmethod(); this means they are required and must be overridden by subclasses.

Others, marked below with “OPTIONAL:”, are not so decorated. For these methods, the behaviour in the base Backend—often, nothing—is an acceptable default behaviour. Subclasses may extend or replace this behaviour as desired, so long as the methods still perform the actions described in the description.

Backends:

  • must only raise standard Python exceptions.

  • must implement the data model as described, or raise NotImplementedError for not implemented parts of the data model.

Methods related to ixmp.Platform:

add_model_name

Add (register) new model name.

add_scenario_name

Add (register) new scenario name.

close_db

OPTIONAL: Close database connection(s).

get_auth

OPTIONAL: Return user authorization for models.

get_doc

Read documentation from database

get_log_level

OPTIONAL: Get logging level for the backend and other code.

get_meta

Retrieve all metadata attached to a specific target.

get_model_names

List existing model names.

get_nodes

Iterate over all nodes stored on the Platform.

get_scenarios

Iterate over TimeSeries stored on the Platform.

get_scenario_names

List existing scenario names.

get_units

Return all registered symbols for units of measurement.

handle_config

OPTIONAL: Handle platform/backend config arguments.

open_db

OPTIONAL: (Re-)open database connection(s).

read_file

OPTIONAL: Read Platform, TimeSeries, or Scenario data from file.

remove_meta

Remove metadata attached to a target.

set_doc

Save documentation to database

set_log_level

OPTIONAL: Set logging level for the backend and other code.

set_meta

Set metadata on a target.

set_node

Add a node name to the Platform.

set_unit

Add a unit of measurement to the Platform.

write_file

OPTIONAL: Write Platform, TimeSeries, or Scenario data to file.

Methods related to ixmp.TimeSeries:

  • Each method has an argument ts, a reference to the TimeSeries object being manipulated.

  • ‘Geodata’ is otherwise identical to regular timeseries data, except value are str rather than float.

check_out

Check out ts for modification.

commit

Commit changes to ts.

delete

Remove time series data.

delete_geo

Remove 'geodata' values.

discard_changes

Discard changes to ts since the last ts_check_out().

get

Retrieve the existing TimeSeries (or Scenario) ts.

get_data

Retrieve time series data.

get_geo

Retrieve time-series 'geodata'.

init

Create a new TimeSeries (or Scenario) ts.

is_default

Return True if ts is the default version for its (model, scenario).

last_update

Return the date of the last modification of the ts, if any.

preload

OPTIONAL: Load ts data into memory.

run_id

Return the run ID of the ts.

set_data

Store data.

set_as_default

Set the current TimeSeries.version as the default.

set_geo

Store time series geodata.

Methods related to ixmp.Scenario:

  • Each method has an argument s, a reference to the Scenario object being manipulated.

clone

Clone s.

delete_item

Remove an item name of type.

get_meta

Retrieve all metadata attached to a specific target.

has_solution

Return True if Scenario s has been solved.

init_item

Initialize an item name of type.

item_delete_elements

Remove elements of item name.

item_get_elements

Return elements of item name.

item_set_elements

Add keys or values to item name.

item_index

Return the index sets or names of item name.

list_items

Return a list of names of items of type.

remove_meta

Remove metadata attached to a target.

set_meta

Set metadata on a target.

Methods related to message_ix.Scenario:

  • Each method has an argument ms, a reference to the Scenario object being manipulated.

Warning

These methods may be moved to ixmp in a future release.

cat_get_elements

Get elements of a category mapping.

cat_list

Return list of categories in mapping name.

cat_set_elements

Add elements to category mapping.

abstract add_model_name(name: str) None

Add (register) new model name.

Parameters

name (str) – New model name

abstract add_scenario_name(name: str) None

Add (register) new scenario name.

Parameters

name (str) – New scenario name

abstract cat_get_elements(ms: ixmp.core.scenario.Scenario, name: str, cat: str) List[str]

Get elements of a category mapping.

Parameters
  • name (str) – Name of the category mapping set.

  • cat (str) – Name of the category within name.

Returns

All set elements mapped to cat in name.

Return type

list of str

abstract cat_list(ms: ixmp.core.scenario.Scenario, name: str) List[str]

Return list of categories in mapping name.

Parameters

name (str) – Name of the category mapping set.

Returns

All categories in name.

Return type

list of str

abstract cat_set_elements(ms: ixmp.core.scenario.Scenario, name: str, cat: str, keys: Union[str, Sequence[str]], is_unique: bool) None

Add elements to category mapping.

Parameters
  • name (str) – Name of the category mapping set.

  • cat (str) – Name of the category within name.

  • keys (iterable of str or list of str) – Keys to add to cat.

  • is_unique (bool) –

    If True:

    • keys must contain only one key.

    • The Backend must remove any existing member of cat, so that it has only one element.

abstract check_out(ts: ixmp.core.timeseries.TimeSeries, timeseries_only: bool) None

Check out ts for modification.

Parameters

timeseries_only (bool) –

???

abstract clear_solution(s: ixmp.core.scenario.Scenario, from_year=None)

Remove data associated with a model solution.

Todo

Document.

abstract clone(s: ixmp.core.scenario.Scenario, platform_dest: ixmp.core.platform.Platform, model: str, scenario: str, annotation: str, keep_solution: bool, first_model_year: Optional[int] = None) ixmp.core.scenario.Scenario

Clone s.

Parameters
  • platform_dest (Platform) – Target backend. May be the same as s.platform.

  • model (str) – New model name.

  • scenario (str) – New scenario name.

  • annotation (str) – Description for the creation of the new scenario.

  • keep_solution (bool) – If True, model solution data is also cloned. If False, it is discarded.

  • first_model_year (int or None) – If int, must be greater than the first model year of s.

Returns

The cloned Scenario.

Return type

Same class as s

close_db() None

OPTIONAL: Close database connection(s).

Close any database connection(s), if open.

abstract commit(ts: ixmp.core.timeseries.TimeSeries, comment: str) None

Commit changes to ts.

ts_init may modify the version attribute of ts.

Parameters

comment (str) – Description of the changes being committed.

del_ts(ts: ixmp.core.timeseries.TimeSeries) None

OPTIONAL: Free memory associated with the TimeSeries ts.

The default implementation has no effect.

abstract delete(ts: ixmp.core.timeseries.TimeSeries, region: str, variable: str, subannual: str, years: Iterable[int], unit: str) None

Remove time series data.

Parameters
  • region (str) – Region name.

  • variable (str) – Variable name.

  • years (Iterable of int) – Years.

  • unit (str) – Unit symbol.

  • subannual (str) – Name of time slice.

abstract delete_geo(ts: ixmp.core.timeseries.TimeSeries, region: str, variable: str, subannual: str, years: Iterable[int], unit: str) None

Remove ‘geodata’ values.

Parameters
  • region (str) – Region name.

  • variable (str) – Variable name.

  • subannual (str) – Name of time slice.

  • years (Iterable of int) – Years.

  • unit (str) – Unit symbol.

abstract delete_item(s: ixmp.core.scenario.Scenario, type: str, name: str) None

Remove an item name of type.

Parameters
  • type ('set' or 'par' or 'equ') –

  • name (str) – Name of the item to delete.

abstract discard_changes(ts: ixmp.core.timeseries.TimeSeries) None

Discard changes to ts since the last ts_check_out().

abstract get(ts: ixmp.core.timeseries.TimeSeries) None

Retrieve the existing TimeSeries (or Scenario) ts.

The TimeSeries is identified based on the unique combination of the attributes of ts:

If version is None, the Backend must return the version marked as default, and must set the attribute value.

If ts is a Scenario, get() must set the scheme attribute with the value previously passed to init().

Raises

ValueError – If model or scenario does not exist on the Platform.

get_auth(user: str, models: Sequence[str], kind: str) Dict[str, bool]

OPTIONAL: Return user authorization for models.

If the Backend implements access control, this method must indicate whether user has permission kind for each of models.

kind may be ‘read’/’view’, ‘write’/’modify’, or other values; get_auth() should raise exceptions on invalid values.

Parameters
  • user (str) – User name or identifier.

  • models (list of str) – Model names.

  • kind (str) – Type of permission being requested

Returns

Mapping of model name (str)bool; True if the user is authorized for the model.

Return type

dict

abstract get_data(ts: ixmp.core.timeseries.TimeSeries, region: Sequence[str], variable: Sequence[str], unit: Sequence[str], year: Sequence[str]) Iterable[Tuple[str, str, str, int, float]]

Retrieve time series data.

Parameters
  • region (list of str) – Region names to filter results.

  • variable (list of str) – Variable names to filter results.

  • unit (list of str) – Unit symbols to filter results.

  • year (list of str) – Years to filter results.

Yields

tuple – The members of each tuple are:

ID

Type

Description

region

str

Region name

variable

str

Variable name

unit

str

Unit symbol

year

int

Year

value

float

Data value

abstract get_doc(domain: str, name: Optional[str] = None) Union[str, Dict]

Read documentation from database

Parameters
  • domain (str) – Documentation domain, e.g. model, scenario etc

  • name (str, optional) – Name of domain entity (e.g. model name).

Returns

String representing fragment of documentation if name is passed as parameter or dictionary containing mapping between name of domain object (e.g. model name) and string representing fragment when name parameter is omitted.

Return type

str or dict

abstract get_geo(ts: ixmp.core.timeseries.TimeSeries) Iterable[Tuple[str, str, int, str, str, str, bool]]

Retrieve time-series ‘geodata’.

Yields

tuple – The members of each tuple are:

ID

Type

Description

region

str

Region name

variable

str

Variable name

year

int

Year

value

str

Value

unit

str

Unit symbol

subannual

str

Name of time slice

meta

bool

True if the data is marked as metadata

get_log_level() str

OPTIONAL: Get logging level for the backend and other code.

The default implementation has no effect.

Returns

Name of a Python logging level.

Return type

str

See also

set_log_level

abstract get_meta(model: Optional[str], scenario: Optional[str], version: Optional[int], strict: bool) Dict[str, Any]

Retrieve all metadata attached to a specific target.

Depending on which of model, scenario, version are None, metadata attached to one of the four kinds of metadata targets (see Metadata) is returned.

If strict is False, then get_meta() must also return metadata attached to less specific or “higher level” targets:

  • For (model, scenario, version), these are (model, scenario); (model,); and (scenario).

  • For (model, scenario), these are (model,) and (scenario,).

  • For (model,) or (scenario,), there are no less specific targets.

Parameters
  • model (str or None) – Model name of metadata target.

  • scenario (str or None) – Scenario name of metadata target.

  • version (int or None) – TimeSeries.version of metadata target.

  • strict (bool) – Only retrieve metadata from the specified target.

Returns

Mapping from metadata names/identifiers to values.

Return type

dict (str -> any)

Raises

ValueError – on unsupported (model, scenario, version) combinations.

abstract get_model_names() Iterable[str]

List existing model names.

Returns

List of the retrieved model names.

Return type

list of str

abstract get_nodes() Iterable[Tuple[str, Optional[str], str, str]]

Iterate over all nodes stored on the Platform.

Yields

tuple – The members of each tuple are:

ID

Type

Description

region

str

Node name or synonym for node

mapped_to

str or None

Node name

parent

str

Parent node name

hierarchy

str

Node hierarchy ID

See also

set_node

abstract get_scenario_names() Iterable[str]

List existing scenario names.

Returns

List of the retrieved scenario names.

Return type

list of str

abstract get_scenarios(default: bool, model: Optional[str], scenario: Optional[str]) Iterable[Tuple[str, str, str, bool, bool, str, str, str, str, str, str, str, int]]

Iterate over TimeSeries stored on the Platform.

Scenarios, as subclasses of TimeSeries, are also included.

Parameters
  • default (bool) – True to include only TimeSeries versions marked as default.

  • model (str or None) – Model name to filter results.

  • scenario (str or None) – Scenario name to filter results.

Yields

tuple – The members of each tuple are:

ID

Type

Description

model

str

Model name

scenario

str

Scenario name

scheme

str

Scheme name

is_default

bool

True if version is the default

is_locked

bool

True if read-only

cre_user

str

Name of user who created the TimeSeries

cre_date

str

Creation datetime

upd_user

str

Name of user who last modified the TimeSeries

upd_date

str

Modification datetime

lock_user

str

Name of user who locked the TimeSeries

lock_date

str

Lock datetime

annotation

str

Description of the TimeSeries

version

int

Version

abstract get_timeslices() Iterable[Tuple[str, str, float]]

Iterate over subannual timeslices defined on the Platform instance.

Yields

tuple – The members of each tuple are:

ID

Type

Description

name

str

Time slice name

category

str

Time slice category

duration

float

Time slice duration (fraction of year)

See also

set_timeslice

abstract get_units() List[str]

Return all registered symbols for units of measurement.

Returns

Return type

list of str

See also

set_unit

classmethod handle_config(args: Sequence, kwargs: MutableMapping) Dict[str, Any]

OPTIONAL: Handle platform/backend config arguments.

Returns a dict to be stored in the configuration file. This dict must be valid as keyword arguments to the __init__() of a backend subclass.

The default implementation expects both args and kwargs to be empty.

See also

Config.add_platform

abstract has_solution(s: ixmp.core.scenario.Scenario) bool

Return True if Scenario s has been solved.

If True, model solution data is available from the Backend.

abstract init(ts: ixmp.core.timeseries.TimeSeries, annotation: str) None

Create a new TimeSeries (or Scenario) ts.

init may modify the version attribute of ts.

If ts is a Scenario; the Backend must store the Scenario.scheme attribute.

Parameters

annotation (str) – If ts is newly-created, the Backend must store this annotation with the TimeSeries.

abstract init_item(s: ixmp.core.scenario.Scenario, type: str, name: str, idx_sets: Sequence[str], idx_names: Optional[Sequence[str]]) None

Initialize an item name of type.

Parameters
  • type ('set' or 'par' or 'equ' or 'var') –

  • name (str) – Name for the new item.

  • idx_sets (sequence of str) – If empty, a 0-dimensional/scalar item is initialized. Otherwise, a 1+-dimensional item is initialized.

  • idx_names (sequence of str or None) – Optional names for the dimensions. If not supplied, the names of the idx_sets (if any) are used. If supplied, idx_names and idx_sets must be the same length.

Raises

ValueError – if any of the idx_sets is not an existing set in the Scenario; if idx_names and idx_sets are not the same length.

abstract is_default(ts: ixmp.core.timeseries.TimeSeries) bool

Return True if ts is the default version for its (model, scenario).

See also

get, set_as_default

abstract item_delete_elements(s: ixmp.core.scenario.Scenario, type: str, name: str, keys) None

Remove elements of item name.

Parameters
  • type ('par' or 'set') –

  • name (str) –

  • keys (iterable of iterable of str) – If name is indexed by other set(s), then the number of elements of each key in keys, and their contents, must match the index set(s). If name is a basic set, then each key must be a list containing a single str, which must exist in the set.

See also

s_init_item, s_item_set_elements

abstract item_get_elements(s: ixmp.core.scenario.Scenario, type: str, name: str, filters: Optional[Dict[str, List[Any]]] = None) Union[Dict[str, Any], pandas.core.series.Series, pandas.core.frame.DataFrame]

Return elements of item name.

Parameters
  • type ('equ' or 'par' or 'set' or 'var') –

  • name (str) – Name of the item.

  • filters (dict (str -> list), optional) –

    If provided, a mapping from dimension names to allowed values along that dimension.

    item_get_elements must silently accept values that are not members of the set indexing a dimension. Elements which are not str must be handled as equivalent to their string representation; i.e. item_get_elements must return the same data for filters={'foo': [42]} and filters={'foo': ['42']}.

Returns

  • pandas.Series – When type is ‘set’ and name an index set (not indexed by other sets).

  • dict – When type is ‘equ’, ‘par’, or ‘var’ and name is scalar (zero- dimensional). The value has the keys ‘value’ and ‘unit’ (for ‘par’) or ‘lvl’ and ‘mrg’ (for ‘equ’ or ‘var’).

  • pandas.DataFrame – For mapping sets, or all 1+-dimensional values. The dataframe has one column per index name with dimension values; plus the columns ‘value’ and ‘unit’ (for ‘par’) or ‘lvl’ and ‘mrg’ (for ‘equ’ or ‘var’).

Raises

KeyError – If name does not exist in s.

abstract item_index(s: ixmp.core.scenario.Scenario, name: str, sets_or_names: str) List[str]

Return the index sets or names of item name.

Parameters

sets_or_names ('sets' or 'names') –

Returns

Return type

list of str

abstract item_set_elements(s: ixmp.core.scenario.Scenario, type: str, name: str, elements: Iterable[Tuple[Any, Optional[float], Optional[str], Optional[str]]]) None

Add keys or values to item name.

Parameters
  • type ('par' or 'set') –

  • name (str) – Name of the items.

  • elements (iterable of 4-tuple) –

    The members of each tuple are:

    ID

    Type

    Description

    key

    str or list of str or None

    Set elements or value indices

    value

    float or None

    Parameter value

    unit

    str or None

    Unit symbol

    comment

    str or None

    Description of the change

    If name is indexed by other set(s), then the number of elements of each key, and their contents, must match the index set(s). When type is ‘set’, value and unit must be None.

Raises
  • ValueError – If elements contain invalid values, e.g. key values not in the index set(s).

  • Exception – If the Backend encounters any error adding the elements.

See also

s_init_item, s_item_delete_elements

abstract last_update(ts: ixmp.core.timeseries.TimeSeries) Optional[str]

Return the date of the last modification of the ts, if any.

abstract list_items(s: ixmp.core.scenario.Scenario, type: str) List[str]

Return a list of names of items of type.

Parameters

type ('set' or 'par' or 'equ') –

open_db() None

OPTIONAL: (Re-)open database connection(s).

A backend may connect to a database server. This method opens the database connection if it is closed.

preload(ts: ixmp.core.timeseries.TimeSeries) None

OPTIONAL: Load ts data into memory.

read_file(path: os.PathLike, item_type: ixmp.backend.ItemType, **kwargs) None

OPTIONAL: Read Platform, TimeSeries, or Scenario data from file.

A backend may implement read_file for one or more combinations of the path and item_type methods. For all other combinations, it must raise NotImplementedError.

The default implementation supports:

  • path ending in ‘.xlsx’, item_type is ItemType.MODEL: read a single Scenario given by kwargs['filters']['scenario'] from file, using s_read_excel().

Parameters
  • path (os.PathLike) –

    File for input. The filename suffix determines the input format:

    Suffix

    Format

    .csv

    Comma-separated values

    .gdx

    GAMS data exchange

    .xlsx

    Microsoft Office Open XML spreadsheet

  • item_type (ItemType) – Type(s) of items to read.

Raises
  • ValueError – If ts is not None and ‘scenario’ is a key in filters.

  • NotImplementedError – If input of the specified items from the file format is not supported.

See also

write_file

abstract remove_meta(names: list, model: Optional[str], scenario: Optional[str], version: Optional[int]) None

Remove metadata attached to a target.

Parameters
  • names (list of str) – Metadata names/identifiers to remove.

  • model (str or None) – Model name of metadata target.

  • scenario (str or None) – Scenario name of metadata target.

  • version (int or None) – TimeSeries.version of metadata target.

Raises

ValueError – on unsupported (model, scenario, version) combinations.

See also

get_meta

abstract run_id(ts: ixmp.core.timeseries.TimeSeries) int

Return the run ID of the ts.

abstract set_as_default(ts: ixmp.core.timeseries.TimeSeries) None

Set the current TimeSeries.version as the default.

See also

get, is_default

abstract set_data(ts: ixmp.core.timeseries.TimeSeries, region: str, variable: str, data: Dict[int, float], unit: str, subannual: str, meta: bool) None

Store data.

Parameters
  • region (str) – Region name.

  • variable (str) – Variable name.

  • subannual (str) – Name of time slice.

  • unit (str) – Unit symbol.

  • data (dict (int -> float)) – Mapping from year to value.

  • meta (bool) – True to mark data as metadata.

abstract set_doc(domain: str, docs) None

Save documentation to database

Parameters
  • domain (str) – Documentation domain, e.g. model, scenario, etc.

  • docs (dict or array of tuples) – Dictionary or tuple array containing mapping between name of domain object (e.g. model name) and string representing fragment of documentation.

abstract set_geo(ts: ixmp.core.timeseries.TimeSeries, region: str, variable: str, subannual: str, year: int, value: str, unit: str, meta: bool) None

Store time series geodata.

Parameters
  • region (str) – Region name.

  • variable (str) – Variable name.

  • subannual (str) – Name of time slice.

  • year (int) – Year.

  • value (str) – Data value.

  • unit (str) – Unit symbol.

  • meta (bool) – True to mark data as metadata.

set_log_level(level: int) None

OPTIONAL: Set logging level for the backend and other code.

The default implementation has no effect.

Parameters

level (int or Python logging level) –

See also

get_log_level

abstract set_meta(meta: dict, model: Optional[str], scenario: Optional[str], version: Optional[int]) None

Set metadata on a target.

Parameters
  • meta (dict) – Mapping from metadata names/identifiers to values.

  • model (str or None) – Model name of metadata target.

  • scenario (str or None) – Scenario name of metadata target.

  • version (int or None) – TimeSeries.version of metadata target.

Raises

ValueError – on unsupported (model, scenario, version) combinations.

See also

get_meta

abstract set_node(name: str, parent: Optional[str] = None, hierarchy: Optional[str] = None, synonym: Optional[str] = None) None

Add a node name to the Platform.

This method must have one of two effects, depending on the arguments:

  • With parent and hierarchy: name is added as a child of parent in the named hierarchy.

  • With synonym: synonym is added as an alias for name.

Parameters
  • name (str) – Node name.

  • parent (str, optional) – Parent node name.

  • hierarchy (str, optional) – Node hierarchy ID.

  • synonym (str, optional) – Synonym for node.

See also

get_nodes

abstract set_timeslice(name: str, category: str, duration: float) None

Add a subannual time slice to the Platform.

Parameters
  • name (str) – Node name.

  • category (str) – Time slice category.

  • duration (float) – Time slice duration (a fraction of a year).

See also

get_timeslices

abstract set_unit(name: str, comment: str) None

Add a unit of measurement to the Platform.

Parameters
  • name (str) – Symbol of the unit.

  • comment (str) – Description of the change or of the unit.

See also

get_units

write_file(path: os.PathLike, item_type: ixmp.backend.ItemType, **kwargs) None

OPTIONAL: Write Platform, TimeSeries, or Scenario data to file.

A backend may implement write_file for one or more combinations of the path and item_type methods. For all other combinations, it must raise NotImplementedError.

The default implementation supports:

  • path ending in ‘.xlsx’, item_type is either MODEL or SET | PAR: write a single Scenario given by kwargs['filters']['scenario'] to file using s_write_excel().

Parameters
  • path (os.PathLike) – File for output. The filename suffix determines the output format.

  • item_type (ItemType) – Type(s) of items to write.

Raises
  • ValueError – If ts is not None and ‘scenario’ is a key in filters.

  • NotImplementedError – If output of the specified items to the file format is not supported.

See also

read_file

class ixmp.backend.base.CachingBackend(cache_enabled=True)

Backend with additional features for caching data.

CachingBackend stores cache values for multiple TimeSeries/Scenario objects, and for multiple values of a filters argument.

Subclasses must call cache(), cache_get(), and cache_invalidate() as appropriate to manage the cache; CachingBackend does not enforce any such logic.

_cache: Dict[Tuple, object] = {}

Cache of values. Keys are given by _cache_key(); values depend on the subclass’ usage of the cache.

_cache_hit: Dict[Tuple, int] = {}

Count of number of times a value was retrieved from cache successfully using cache_get().

classmethod _cache_key(ts: ixmp.core.timeseries.TimeSeries, ix_type: Optional[str], name: Optional[str], filters: Optional[Dict[str, Hashable]] = None) Tuple[Hashable, ...]

Return a hashable cache key.

ixmp filters (a dict of list) are converted to a unique id that is hashable.

Returns

A hashable key with 4 elements for ts, ix_type, name, and filters.

Return type

tuple

cache(ts: ixmp.core.timeseries.TimeSeries, ix_type: str, name: str, filters: Dict, value: Any) bool

Store value in cache.

Returns

True if the key was already in the cache and its value was overwritten.

Return type

bool

cache_enabled = True

True if caching is enabled.

cache_get(ts: ixmp.core.timeseries.TimeSeries, ix_type: str, name: str, filters: Dict) Optional[Any]

Retrieve value from cache.

The value in _cache is copied to avoid cached values being modified by user code. _cache_hit is incremented.

Raises

KeyError – If the key for ts, ix_type, name and filters is not in the cache.

cache_invalidate(ts: ixmp.core.timeseries.TimeSeries, ix_type: Optional[str] = None, name: Optional[str] = None, filters: Optional[Dict] = None) None

Invalidate cached values.

With all arguments given, single key/value is removed from the cache. Otherwise, multiple keys/values are removed:

  • ts only: all cached values associated with the TimeSeries or Scenario object.

  • ts, ix_type, and name: all cached values associated with the item, whether filtered or unfiltered.

del_ts(ts: ixmp.core.timeseries.TimeSeries)

Invalidate cache entries associated with ts.

class ixmp.backend.ItemType(value)

Type of data items in TimeSeries and Scenario.

TS = 1
T = 1

Time series data variable.

SET = 2
S = 2

Set.

PAR = 4
P = 4

Parameter.

VAR = 8
V = 8

Model variable.

EQU = 16
E = 16

Equation.

MODEL = 30
M = 30

All kinds of model-related data, i.e. SET, PAR, VAR and EQU.

SOLUTION = 24

Model solution data, i.e. VAR and EQU.

ALL = 31
A = 31

All data, i.e. MODEL and TS.

ixmp.backend.FIELDS = {'get_nodes': ('region', 'mapped_to', 'parent', 'hierarchy'), 'get_scenarios': ('model', 'scenario', 'scheme', 'is_default', 'is_locked', 'cre_user', 'cre_date', 'upd_user', 'upd_date', 'lock_user', 'lock_date', 'annotation', 'version'), 'get_timeslices': ('name', 'category', 'duration'), 'ts_get': ('region', 'variable', 'unit', 'subannual', 'year', 'value'), 'ts_get_geo': ('region', 'variable', 'subannual', 'year', 'value', 'unit', 'meta'), 'write_file': ('MODEL', 'SCENARIO', 'VERSION', 'VARIABLE', 'UNIT', 'REGION', 'META', 'SUBANNUAL', 'YEAR', 'VALUE')}

Lists of field names for tuples returned by Backend API methods.

The key “write_file” refers to the columns appearing in the CSV output from export_timeseries_data() when using JDBCBackend.

Todo

Make this consistent with other dimension orders and with IAMC_IDX.