Reporting¶
Warning
ixmp.reporting
is experimental in ixmp 0.2, and only supports
Python 3. The API and functionality may change without advance notice or a
deprecation period in subsequent releases.
-
reporting.
configure
(**config)¶ Configure reporting globally.
Valid configuration keys include:
- units:
- define: a string, passed to
pint.UnitRegistry.define()
. - replace: a mapping from str to str, used to replace units before they are parsed by pints
- define: a string, passed to
Warns: UserWarning – If config contains unrecognized keys. - units:
-
class
ixmp.reporting.
Reporter
(**kwargs)¶ Class for generating reports on
ixmp.Scenario
objects.A Reporter is used to postprocess data from from one or more
ixmp.Scenario
objects. Theget()
method can be used to:Retrieve individual quantities. A quantity has zero or more dimensions and optional units. Quantities include the ‘parameters’, ‘variables’, ‘equations’, and ‘scalars’ available in an
ixmp.Scenario
.Generate an entire report composed of multiple quantities. A report may:
- Read in non-model or exogenous data,
- Trigger output to files(s) or a database, or
- Execute user-defined methods.
Every report and quantity (including the results of intermediate steps) is identified by a
utils.Key
; all the keys in a Reporter can be listed withkeys()
.Reporter uses a graph data structure to keep track of computations, the atomic steps in postprocessing: for example, a single calculation that multiplies two quantities to create a third. The graph allows
get()
to perform only the requested computations. Advanced users may manipulate the graph directly; but common reporting tasks can be handled by using Reporter methods:add
(key, computation[, strict])Add computation to the Reporter under key. add_file
(path[, key])Add exogenous quantities from path. aggregate
(qty, tag, dims_or_groups[, …])Add a computation that aggregates qty. apply
(generator, *keys)Add computations from generator applied to key. configure
([path])Configure the Reporter. describe
([key])Return a string describing the computations that produce key. disaggregate
(qty, new_dim[, method, args])Add a computation that disaggregates var using method. finalize
(scenario)Prepare the Reporter to act on scenario. full_key
(name)Return the full-dimensionality key for name. get
([key])Execute and return the result of the computation key. read_config
(path)Configure the Reporter with information from a YAML file at path. visualize
(filename, **kwargs)Generate an image describing the reporting structure. write
(key, path)Write the report key to the file path. -
add
(key, computation, strict=False)¶ Add computation to the Reporter under key.
Parameters: - key (hashable) – A string, Key, or other value identifying the output of task.
- computation (object) –
One of:
- any existing key in the Reporter.
- any other literal value or constant.
- a task, i.e. a tuple with a callable followed by one or more computations.
- A list containing one or more of #1, #2, and/or #3.
- strict (bool, optional) – If True (default), key must not already exist in the Reporter.
Raises: KeyError
– If key is already in the Reporter.add()
may be used to:Provide an alias from one key to another:
>>> r.add('aliased name', 'original name')
Define an arbitrarily complex computation in a Python function that operates directly on the
ixmp.Scenario
:>>> def my_report(scenario): >>> # many lines of code >>> return 'foo' >>> r.add('my report', (my_report, 'scenario')) >>> r.finalize(scenario) >>> r.get('my report') foo
Note
Use care when adding literal
str
values (2); these may conflict with keys that identify the results of other computations.
-
add_file
(path, key=None)¶ Add exogenous quantities from path.
A file at a path like ‘/path/to/foo.ext’ is added at the key
'file:foo.ext'
.
-
add_product
(name, quantities, sums=True)¶ Add a computation that takes the product of quantities.
Parameters: Returns: The full key of the new quantity.
Return type:
-
aggregate
(qty, tag, dims_or_groups, weights=None, keep=True)¶ Add a computation that aggregates qty.
Parameters: - qty (
Key
or str) – Key of the quantity to be disaggregated. - tag (str) – Additional string to add to the end the key for the aggregated quantity.
- dims_or_groups (str or iterable of str or dict) – Name(s) of the dimension(s) to sum over, or nested dict.
- weights (xr.DataArray) – Weights for weighted aggregation.
Returns: The key of the newly-added node.
Return type: Key
- qty (
-
apply
(generator, *keys)¶ Add computations from generator applied to key.
Parameters: - generator (callable) – Function to apply to keys. Must yield a sequence (0 or more) of
(key, computation), which are added to the
graph
. - keys (hashable) – The starting key(s)
- generator (callable) – Function to apply to keys. Must yield a sequence (0 or more) of
(key, computation), which are added to the
-
configure
(path=None, **config)¶ Configure the Reporter.
Accepts a path to a configuration file and/or keyword arguments. Configuration keys loaded from file are replaced by keyword arguments.
Valid configuration keys include:
- default: the default reporting key; sets
default_key
. - filters: a
dict
, passed toset_filters()
. - files: a
dict
mapping keys to file paths. - alias: a
dict
mapping aliases to original keys.
Warns: UserWarning – If config contains unrecognized keys. - default: the default reporting key; sets
-
default_key
= None¶ The default reporting key.
-
describe
(key=None)¶ Return a string describing the computations that produce key.
If key is not provided, all keys in the Reporter are described.
-
disaggregate
(qty, new_dim, method='shares', args=[])¶ Add a computation that disaggregates var using method.
Parameters: - var (hashable) – Key of the variable to be disaggregated.
- new_dim (str) – Name of the new dimension of the disaggregated variable.
- method (callable or str) – Disaggregation method. If a callable, then it is applied to var with any extra args. If then a method named ‘disaggregate_{method}’ is used.
- args (list, optional) – Additional arguments to the method. The first element should be the key for a quantity giving shares for disaggregation.
Returns: The key of the newly-added node.
Return type:
-
finalize
(scenario)¶ Prepare the Reporter to act on scenario.
The
Scenario
object scenario is associated with the key'scenario'
. All subsequent processing will act on data from this scenario.
-
classmethod
from_scenario
(scenario, **kwargs)¶ Create a Reporter by introspecting scenario.
Parameters: - scenario (ixmp.Scenario) – Scenario to introspect in creating the Reporter.
- kwargs (optional) – Passed to
Scenario.configure()
.
Returns: A Reporter instance containing:
- A ‘scenario’ key referring to the scenario object.
- Each parameter, equation, and variable in the scenario.
- All possible aggregations across different sets of dimensions.
- Each set in the scenario.
Return type:
-
full_key
(name)¶ Return the full-dimensionality key for name.
An ixmp variable ‘foo’ indexed by a, c, n, q, and x is available in the Reporter at
'foo:a-c-n-q-x'
.full_key('foo')
retrieves thisKey
.
-
get
(key=None)¶ Execute and return the result of the computation key.
Only key and its dependencies are computed.
Parameters: key (str, optional) – If not provided, default_key
is used.Raises: ValueError
– If key anddefault_key
are bothNone
.
-
read_config
(path)¶ Configure the Reporter with information from a YAML file at path.
See
configure()
.
-
set_filters
(**filters)¶ Apply filters ex ante (before computations occur).
filters has the same form as the argument of the same name to
ixmp.Scenario.par()
and analogous methods. A value ofNone
will clear the filter for the named dimension.
-
visualize
(filename, **kwargs)¶ Generate an image describing the reporting structure.
This is a shorthand for
dask.visualize()
. Requires graphviz.
-
write
(key, path)¶ Write the report key to the file path.
Computations¶
Elementary computations for reporting.
Calculations:
aggregate (quantity, groups, keep) |
Aggregate quantity by groups. |
disaggregate_shares (quantity, shares) |
Disaggregate quantity by shares. |
product (*quantities[, drop]) |
Return the product of any number of quantities. |
ratio (numerator, denominator[, drop]) |
Return the ratio numerator / denominator. |
sum (quantity[, weights, dimensions]) |
Sum quantity over dimensions, with optional weights. |
Input and output:
load_file (path) |
Read the file at path and return its contents. |
write_report (quantity, path) |
Write the report identified by key to the file at path. |
Conversion:
make_dataframe (*quantities) |
Concatenate quantities into a single pd.DataFrame. |
-
ixmp.reporting.computations.
aggregate
(quantity, groups, keep)¶ Aggregate quantity by groups.
Disaggregate quantity by shares.
-
ixmp.reporting.computations.
make_dataframe
(*quantities)¶ Concatenate quantities into a single pd.DataFrame.
-
ixmp.reporting.computations.
load_file
(path)¶ Read the file at path and return its contents.
Some file formats are automatically converted into objects for direct use in reporting code:
- csv: converted to
xarray.DataArray
. CSV files must have a ‘value’ column; all others are treated as indices.
- csv: converted to
-
ixmp.reporting.computations.
sum
(quantity, weights=None, dimensions=None)¶ Sum quantity over dimensions, with optional weights.
-
ixmp.reporting.computations.
write_report
(quantity, path)¶ Write the report identified by key to the file at path.
Utilities¶
-
class
ixmp.reporting.utils.
Key
(name, dims=[], tag=None)¶ A hashable key for a quantity that includes its dimensionality.
Quantities in a
Scenario
can be indexed by one or more dimensions. For example, a parameter with three dimensions can be initialized with:>>> scenario.init_par('foo', ['a', 'b', 'c'], ['apple', 'bird', 'car'])
Computations for this scenario might use the quantity
foo
in different ways:- in its full resolution, i.e. indexed by a, b, and c;
- aggregated (e.g. summed) over any one dimension, e.g. aggregated over c and thus indexed by a and b;
- aggregated over any two dimensions; etc.
A Key for (1) will hash, display, and evaluate as equal to
'foo:a-b-c'
. A Key for (2) corresponds to'foo:a-b'
, and so forth.Keys may be generated concisely by defining a convenience method:
>>> def foo(dims): >>> return Key('foo', dims.split('')) >>> foo('a b') foo:a-b
-
classmethod
from_str_or_key
(value, drop=None, append=None, tag=None)¶ Return a new Key from value.
-
iter_sums
()¶ Yield (key, task) for all possible partial sums of the Key.
-
classmethod
product
(new_name, *keys)¶ Return a new Key that has the union of dimensions on keys.
Dimensions are ordered by their first appearance.
-
class
ixmp.reporting.utils.
AttrSeries
(*args, **kwargs)¶ pandas.Series
subclass imitatingxarray.DataArray
.Future versions of
ixmp.reporting
will usexarray.DataArray
asQuantity
; however, becausexarray
currently lacks sparse matrix support, ixmp quantities may be too large for memory.The AttrSeries class provides similar methods and behaviour to
xarray.DataArray
, such as an attrs dictionary for metadata, so thatixmp.reporting.computations
methods can use xarray-like syntax.
-
ixmp.reporting.utils.
Quantity
¶ alias of
ixmp.reporting.utils.AttrSeries
-
ixmp.reporting.utils.
clean_units
(input_string)¶ Tolerate messy strings for units.
Handles two specific cases found in MESSAGEix test cases:
- Dimensions enclosed in ‘[]’ have these characters stripped.
- The ‘%’ symbol cannot be supported by pint, because it is a Python operator; it is translated to ‘percent’.
-
ixmp.reporting.utils.
collect_units
(*args)¶ Return an list of ‘_unit’ attributes for args.
-
ixmp.reporting.utils.
data_for_quantity
(ix_type, name, column, scenario, filters=None)¶ Retrieve data from scenario.
Parameters: - ix_type ('equ' or 'par' or 'var') – Type of the ixmp object.
- name (str) – Name of the ixmp object.
- column ('mrg' or 'lvl' or 'value') – Data to retrieve. ‘mrg’ and ‘lvl’ are valid only for
ix_type='equ'
, and ‘level’ otherwise. - scenario (ixmp.Scenario) – Scenario containing data to be retrieved.
- filters (dict, optional) – Mapping from dimensions to iterables of allowed values along each dimension.
Returns: Data for name.
Return type:
-
ixmp.reporting.utils.
keys_for_quantity
(ix_type, name, scenario)¶ Iterate over keys for name in scenario.