Reporting / postprocessing

ixmp.report provides features for computing derived values from the contents of a ixmp.Scenario, after it has been solved using a model and the solution data has been stored. It is built on the genno package, which has its own, separate documentation. This page provides only API documentation.

• For an introduction and basic concepts, see Concepts and usage in the genno documentation.

• For automatic reporting of message_ix.Scenario, see Postprocessing and reporting in the MESSAGEix documentation.

Top-level classes and functions

The following top-level objects from genno may also be imported from ixmp.report.

ComputationError(exc)	Wrapper to print intelligible exception information for Computer.get().
Key(name_or_value[, dims, tag, _fast])	A hashable key for a quantity that includes its dimensionality.
KeyExistsError	Raised by Computer.add() when the target key exists.
MissingKeyError	Raised by Computer.add() when a required input key is missing.
Quantity(*args, **kwargs)	A sparse data structure that behaves like xarray.DataArray.
configure([path])	Configure genno globally.

ixmp.report additionally defines:

Reporter(*args, **kwargs)

Class for describing and executing computations.

class ixmp.report.Reporter(*args, **kwargs)

Class for describing and executing computations.

A Reporter extends a genno.Computer to postprocess data from one or more ixmp.Scenario objects.

Using the from_scenario(), a Reporter is automatically populated with:

• Keys that retrieve the data for every ixmp item (parameter, variable, equation, or scalar) available in the Scenario.

finalize(scenario)	Prepare the Reporter to act on scenario.
from_scenario(scenario, **kwargs)	Create a Reporter by introspecting scenario.
set_filters(**filters)	Apply filters ex ante (before computations occur).

The Reporter class inherits from Computer the following methods:

add(data, *args, **kwargs)	General-purpose method to add computations.
add_queue(queue[, max_tries, fail])	Add tasks from a list or queue.
add_single(key, *computation[, strict, index])	Add a single computation at key.
apply(generator, *keys, **kwargs)	Add computations by applying generator to keys.
check_keys(*keys[, predicate, action])	Check that keys are in the Computer.
configure([path, fail, config])	Configure the Computer.
describe([key, quiet])	Return a string describing the computations that produce key.
full_key(name_or_key)	Return the full-dimensionality key for name_or_key.
get([key])	Execute and return the result of the computation key.
infer_keys(key_or_keys[, dims])	Infer complete key_or_keys.
keys()	Return the keys of graph.
visualize(filename[, key, optimize_graph])	Generate an image describing the Computer structure.
write(key, path, **kwargs)	Compute key and write the result directly to path.

The following methods are deprecated; equivalent or better functionality is available through Reporter.add. See the genno documentation for each method for suggested changes/migrations.

add_file(*args, **kwargs)	Deprecated.
add_product(*args, **kwargs)	Deprecated.
aggregate(qty, tag, dims_or_groups[, ...])	Deprecated.
convert_pyam(*args, **kwargs)	Deprecated.
disaggregate(qty, new_dim[, method, args])	Deprecated.

finalize(scenario: Scenario) → None

Prepare the Reporter to act on scenario.

The TimeSeries (thus also Scenario or message_ix.Scenario) object scenario is stored with the key 'scenario'. All subsequent processing will act on data from this Scenario.

classmethod from_scenario(scenario: Scenario, **kwargs) → Reporter

Create a Reporter by introspecting scenario.

Parameters:

• scenario (Scenario) – Scenario to introspect in creating the Reporter.

• kwargs – Passed to genno.Computer.configure().

Returns:

A Reporter instance containing:

• A ‘scenario’ key referring to the scenario object.

• Each parameter, equation, and variable in the scenario.

• All possible aggregations across different sets of dimensions.

• Each set in the scenario.

Return type:

Reporter

set_filters(**filters) → None

Apply filters ex ante (before computations occur).

See the description of filters() under Configuration.

Configuration

ixmp.report adds handlers for two configuration sections, and modifies the behaviour of one from genno

ixmp.report.filters(c: Computer, filters: dict)

Handle the entire filters: config section.

Reporter-specific configuration.

Affects data loaded from a Scenario using data_for_quantity(), which filters the data before any other computation takes place. Filters are stored at Reporter.graph["config"]["filters"].

If no arguments are provided, all filters are cleared. Otherwise, filters is a mapping of str → (list of str or None. Keys are dimension IDs. Values are either lists of allowable labels along the respective dimension or None to clear any existing filters for that dimension.

This configuration can be applied through Reporter.set_filters(); Reporter.configure, or in a configuration file:

filters:
  # Exclude a label "x2" on the "x" dimension, etc.
  x: [x1, x3, x4]
  technology: [coal_ppl, wind_ppl]
  # Clear existing filters for the "commodity" dimension
  commodity: null

report.rename_dims(info: dict)

Handle the entire rename_dims: config section.

Reporter-specific configuration.

Affects data loaded from a Scenario using data_for_quantity(). Native dimension names are mapped; in the example below, the dimension “i” is present in the Reporter as “i_renamed” on all quantities/keys in which it appears.

rename_dims:
  i: i_renamed

report.units(info: dict)

Handle the entire units: config section.

The only difference from genno.config.units() is that this handler keeps the configuration values stored in Reporter.graph["config"]. This is so that data_for_quantity() can make use of ["units"]["apply"]

report.configure(**config)

Configure genno globally.

Modifies global variables that affect the behaviour of all Computers and operators. Configuration keys loaded from file are superseded by keyword arguments. Messages are logged at level logging.INFO if config contains unhandled sections.

Parameters:

• path (pathlib.Path, optional) – Path to a configuration file in JSON or YAML format.

• **config – Configuration keys/sections and values.

This is the same as genno.configure().

Operators

ixmp.report defines these operators:

data_for_quantity(ix_type, name, column, ...)	Retrieve data from scenario.
from_url(url[, cls])	Return a ixmp.TimeSeries or subclass instance, given its url.
get_ts(ts[, filters, iamc, subannual])	Retrieve timeseries data from ts.
map_as_qty(set_df, full_set)	Convert set_df to a Quantity.
update_scenario(scenario, *quantities[, params])	Update scenario with computed data from reporting quantities.
store_ts(scenario, *data[, strict])	Store time series data on scenario.
remove_ts(ts[, data, after])	Remove all time series data from ts.

Basic operators are defined by genno.operator and its compatibility modules; see there for details:

Plot()	Class for plotting using plotnine.
add	Sum across multiple quantities.
aggregate(quantity, groups, keep)	Aggregate quantity by groups.
apply_units(qty, units)	Apply units to qty.
as_pyam	Return a pyam.IamDataFrame containing the data from quantity.
broadcast_map(quantity, map[, rename, strict])	Broadcast quantity using a map.
combine(*quantities[, select, weights])	Sum distinct quantities by weights.
concat()	Concatenate Quantity objs.
disaggregate_shares(quantity, shares)	Deprecated: Disaggregate quantity by shares.
div	Compute the ratio numerator / denominator.
group_sum(qty, group, sum)	Group by dimension group, then sum across dimension sum.
interpolate(qty[, coords, method, ...])	Interpolate qty.
load_file	Read the file at path and return its contents as a Quantity.
mul	Compute the product of any number of quantities.
pow(a, b)	Compute a raised to the power of b.
product	Alias of mul(), for backwards compatibility.
relabel(qty[, labels])	Replace specific labels along dimensions of qty.
rename_dims(qty[, new_name_or_name_dict])	Rename the dimensions of qty.
ratio	Alias of div(), for backwards compatibility.
select(qty, indexers, *[, inverse, drop])	Select from qty based on indexers.
sum	Sum quantity over dimensions, with optional weights.
write_report(-> None  -> None)	Write a quantity to a file.

ixmp.report.operator.data_for_quantity(ix_type: Literal['equ', 'par', 'var'], name: str, column: Literal['mrg', 'lvl', 'value'], scenario: Scenario, config: Mapping[str, Mapping]) → Quantity

Retrieve data from scenario.

Parameters:

• ix_type – Type of the ixmp object.

• name (str) – Name of the ixmp object.

• column – Data to retrieve. ‘mrg’ and ‘lvl’ are valid only for ix_type='equ',and ‘level’ otherwise.

• scenario (ixmp.Scenario) – Scenario containing data to be retrieved.

• config – Configuration. The key ‘filters’ may contain a mapping from dimensions to iterables of allowed values along each dimension. The key ‘units’/’apply’ may contain units to apply to the quantity; any such units overwrite existing units, without conversion.

Returns:

Data for name.

Return type:

Quantity

ixmp.report.operator.from_url(url: str, cls=<class 'ixmp.core.timeseries.TimeSeries'>) → TimeSeries

Return a ixmp.TimeSeries or subclass instance, given its url.

Parameters:

cls (type, optional) – Subclass to instantiate and return; for instance, Scenario.

ixmp.report.operator.get_ts(ts: TimeSeries, filters: dict | None = None, iamc: bool = False, subannual: bool | str = 'auto') → DataFrame

Retrieve timeseries data from ts.

Corresponds to TimeSeries.timeseries().

Parameters:

filters – Names and values for the region, variable, unit, and year keyword arguments to timeseries().

ixmp.report.operator.map_as_qty(set_df: DataFrame, full_set)

Convert set_df to a Quantity.

For the MESSAGE sets named cat_* (see Category types and mappings) ixmp.Scenario.set() returns a DataFrame with two columns: the category set (S1) elements and the category member set (S2, also required as the argument full_set) elements.

map_as_qty converts such a DataFrame (set_df) into a Quantity with two dimensions. At the coordinates (s₁, s₂), the value is 1 if s₂ is mapped from s₁; otherwise 0.

A category named ‘all’, containing all elements of full_set, is added automatically.

See also

broadcast_map

ixmp.report.operator.remove_ts(ts: TimeSeries, data: DataFrame | None = None, after: int | None = None) → None

Remove all time series data from ts.

Note that data stored with add_timeseries() using meta=True as a keyword argument cannot be removed using TimeSeries.remove_timeseries(), and thus also not with this operator.

Parameters:

• data (pandas.DataFrame, optional) – Specific data to be removed. If not given, all time series data is removed.

• after (int, optional) – If given, only data with year labels equal to or greater than after are removed.

ixmp.report.operator.store_ts(scenario, *data, strict: bool = False) → None

Store time series data on scenario.

The data is stored using add_timeseries(); scenario is checked out and then committed.

Parameters:

• scenario – Scenario on which to store data.

• data (pandas.DataFrame or pyam.IamDataFrame) – 1 or more objects containing data to store. If pandas.DataFrame, the data are passed through to_iamc_layout().

• strict (bool) – If True (default False), raise an exception if any of data are not successfully added. Otherwise, log on level ERROR and continue.

ixmp.report.operator.update_scenario(scenario, *quantities, params=[])

Update scenario with computed data from reporting quantities.

Parameters:

• scenario (Scenario) –

• quantities (genno.Quantity or pandas.DataFrame) – If DataFrame, must be valid input to Scenario.add_par().

• params (list of str, optional) – For every element of quantities that is a pd.DataFrame, the element of params at the same index gives the name of the parameter to update.

Utilities

ixmp.report.common.RENAME_DIMS: Dict[str, str] = {}

Dimensions to rename when extracting raw data from Scenario objects. Mapping from Scenario dimension name -> preferred dimension name.

User code should avoid directly manipulating RENAME_DIMS. Instead, call configure():

# Rename dimension "long_dimension_name" to "ldn"
configure(rename_dims={"long_dimension_name": "ldn"})

As well, importing the variable into the global namespace of another module creates a copy of the dictionary that may become out of sync with other changes. Thus, instead of:

from ixmp.report import RENAME_DIMS

def my_operator(...):
    # Code that references RENAME_DIMS

Do this:

def my_operator(...):
    from ixmp.report import common

    # Code that references common.RENAME_DIMS

ixmp.report.util.dims_for_qty(data)

Return the list of dimensions for data.

If data is a pandas.DataFrame, its columns are processed; otherwise it must be a list.

RENAME_DIMS is used to rename dimensions.

ixmp.report.util.keys_for_quantity(ix_type, name, scenario)

Return keys for name in scenario.