Postprocessing and reporting
The MESSAGEix framework provides zero-configuration reporting of models built on the framework.
The word “reporting” refers to calculations and other post-processing performed after a Scenario
has been solved by the associated optimization model: first the model solution is obtained, and then things are “reported” based on that solution.
message_ix.report
is developed on the basis of ixmp.report
and genno
.
It provides a basis for other code and packages—such as message_ix_models
—that perform reporting calculations tailored to specific model structures.
Each layer of this “stack” builds on the features in the level below:
Package |
Role |
Core features |
Reporting features |
---|---|---|---|
|
MESSAGEix-GLOBIOM models |
Specific model structure ( |
Calculations for specific technologies |
|
Energy model framework |
Common sets/parameters ( |
Derived quantities ( |
|
Optimization models & data |
|
|
|
Structured calculations |
— |
These features are accessible through Reporter
, which can produce multiple reports from one or more Scenarios.
A report and the quantities that enter it is identified by a key, and may…
perform arbitrarily complex calculations while intelligently handling units;
read and make use of data that is ‘exogenous’ to (not included in) a Scenario;
produce output as Python or R objects (in code), or write to files or databases;
calculate only a requested subset of quantities; and
much, much more!
Contents:
Concepts
See Concepts and usage in the genno documentation for an introduction to concepts including quantity, key, computation, task, graph, and operator.
In message_ix.report
:
The
Reporter
class is an extended version of thegenno.Computer
class.ixmp
parameters, scalars, equations, and time-series data all become quantities for the purpose of reporting.For example, the MESSAGEix parameter
resource_cost
, defined with the dimensions (node n, commodity c, grade g, year y) is identified by the keyresource_cost:n-c-g-y
. When summed across the grade/g dimension, it has dimensions n, c, y and is identified by the keyresource_cost:n-c-y
.Reporter.from_scenario()
automatically sets up keys and tasks (such asresource_cost:n-c-g-y
) that simply retrieve raw/unprocessed data from aScenario
and return it as agenno.Quantity
.Operators are defined as functions in modules including:
message_ix.report.operator
,ixmp.report.operator
, andgenno.operator
. These are documented below.
Usage
A MESSAGEix reporting workflow has the following steps:
Use
Reporter.from_scenario()
to prepare a Reporter object with many calculations automatically prepared.(optionally) Use the built-in features of
Reporter
to describe additional calculations.Use
get()
1 or more times to execute tasks, including all the calculations on which they depend:
from ixmp import Platform
from message_ix import Scenario, Reporter
mp = Platform()
scen = Scenario(scen)
rep = Reporter.from_scenario(scen)
rep.get("all")
Note that keys and tasks are described in steps (2–3), but they are not executed until get()
is called—or the results of one task are required by another.
This design allows the Reporter to skip unneeded (and potentially slow) computations and deliver good performance.
The Reporter’s graph
may contain thousands of tasks for retrieving model quantities and calculating derived quantities, but a particular call to get()
may only execute a few of these.
Customization
A Reporter prepared with from_scenario()
always contains a key
scenario
, referring to the Scenario to be reported.
The method Reporter.add()
can be used to add arbitrary Python code that operates directly on the Scenario object:
def my_custom_report(scenario):
"""Function with custom code that manipulates the `scenario`."""
print("Model name:", scenario.model)
# Add a task at the key "custom". The task executes my_custom_report().
# The key "scenario" means that the Scenario object is retrieved and
# passed as an argument to the function.
rep.add("custom", (my_custom_report, "scenario"))
rep.get("custom")
In this example, the function my_custom_report()
could run to thousands of lines; read to and write from multiple files; invoke other programs or Python scripts; etc.
In order to take advantage of the performance-optimizing features of the Reporter, such calculations can instead be composed from atomic (i.e. small, indivisible) operators or functions.
See the genno
documentation for more.
API reference
Top-level classes and functions
message_ix.report
provides:
|
MESSAGEix Reporter. |
The following objects from genno
may also be imported from message_ix.report
.
Their documentation is repeated below for convenience.
|
Wrapper to print intelligible exception information for |
|
A hashable key for a quantity that includes its dimensionality. |
Raised by |
|
Raised by |
|
alias of |
|
|
Configure |
ixmp.Reporter.from_scenario
automatically adds keys based on the contents of the Scenario
argument;
that is, every ixmp
set, parameter, variable, and equation available in the Scenario.
message_ix.Reporter.from_scenario
extends this to add additional keys for derived quantities specific to the MESSAGEix model framework.
These include:
Tip
Use full_key()
to retrieve the full-dimensionality Key
for any of these quantities.
out
=output
×ACT
; that is, the product ofoutput
(output efficiency) andACT
(activity)out_hist
=output
×ref_activity
(historical reference activity)in
=input
×ACT
in_hist
=input
×ref_activity
emi
=emission_factor
×ACT
emi_hist
=emission_factor
×ref_activity
inv
=inv_cost
×CAP_NEW
inv_hist
=inv_cost
×ref_new_capacity
fom
=fix_cost
×CAP
; the name refers to “Fixed Operation and Maintenance costs”fom_hist
=fix_cost
×ref_capacity
vom
=var_cost
×ACT
; “Variable Operation and Maintenance costs”vom_hist
=var_cost
×ref_activity
tom
=fom
+vom
; “Total Operation and Maintenance costs”land_out
=land_output
×LAND
land_use_qty
=land_use
×LAND
land_emi
=land_emission
×LAND
addon conversion
, the model parameteraddon_conversion
(note space versus underscore), except broadcast across individual add-on technologies (ta) rather than add-on types (type_addon).addon up
, which isaddon_up
similarly broadcast.addon ACT
=addon conversion
×ACT
addon in
=input
×addon ACT
addon out
=output
×addon ACT
addon potential
=addon up
×addon ACT
, the maximum potential activity by add-on technology.price emission
, the model variablePRICE_EMISSION
broadcast across emission species (e) and technologies (t) rather than types (type_emission, type_tec).
Other added keys include:
message_ix
adds the standard short symbols for MESSAGEix dimensions (sets) based onmodels.DIMS
. Each of these is also available in a Reporter: for examplerep.get("n")
returns a list with the elements of the MESSAGEix set named “node”;rep.get("t")
returns the elements of the set “technology”, and so on. These keys can be used as input to other computations.y0
= thefirstmodelyear
or \(y_0\) (int
).y::model
= only the periods in the year set (y
) that are equal to or greater thany0
.
Computations to convert internal
Quantity
data format to the IAMC data format, specifically aspyam.IamDataFrame
objects. These include:<name>::pyam
for most of the above derived quantities.CAP::pyam
(fromCAP
)CAP_NEW::pyam
(fromCAP_NEW
)
map_<name>
as “one-hot” or indicator quantities for the respective MESSAGEix mapping setscat_<name>
.Standard reports
message::system
,message::costs
, andmessage::emissions
perTASKS1
.The report
message::default
, collecting all of the above reports.
These automatic contents are prepared using:
Common reporting tasks. |
|
Quantities to automatically convert to IAMC format using |
|
Automatic reports that |
- class message_ix.report.Reporter(*args, **kwargs)[source]
Bases:
Reporter
MESSAGEix Reporter.
add
(data, *args, **kwargs)General-purpose method to add computations.
add_queue
(queue[, max_tries, fail])Add tasks from a list or queue.
add_single
(key, *computation[, strict, index])Add a single computation at key.
apply
(generator, *keys, **kwargs)Add computations by applying generator to keys.
check_keys
(*keys[, predicate, action])Check that keys are in the Computer.
configure
([path, fail, config])Configure the Computer.
describe
([key, quiet])Return a string describing the computations that produce key.
eval
(expr)Evaluate expr to add tasks and keys.
finalize
(scenario)Prepare the Reporter to act on scenario.
from_scenario
(scenario, **kwargs)Create a Reporter by introspecting scenario.
full_key
(name_or_key)Return the full-dimensionality key for name_or_key.
get
([key])Execute and return the result of the computation key.
infer_keys
(key_or_keys[, dims])Infer complete key_or_keys.
keys
()Return the keys of
graph
.set_filters
(**filters)Apply filters ex ante (before computations occur).
visualize
(filename[, key, optimize_graph])Generate an image describing the Computer structure.
write
(key, path, **kwargs)Compute key and write the result directly to path.
add_file
(*args, **kwargs)Deprecated.
add_product
(*args, **kwargs)Deprecated.
aggregate
(qty, tag, dims_or_groups[, ...])Deprecated.
convert_pyam
(*args, **kwargs)Deprecated.
disaggregate
(qty, new_dim[, method, args])Deprecated.
- add(data, *args, **kwargs) Key | str | tuple[Key | str, ...] [source]
General-purpose method to add computations.
add()
can be called in several ways; its behaviour depends on data; see below. It chains to methods such asadd_single()
,add_queue()
, and/orapply()
; each can also be called directly.See also
- add_aggregate(qty: Key | str, tag: str, dims_or_groups: Mapping | str | Sequence[str], weights: DataArray | None = None, keep: bool = True, sums: bool = False, fail: str | int | None = None)[source]
Deprecated.
Add a computation that aggregates qty.
Deprecated since version 1.18.0: Instead, for a mapping/
dict
dims_or_groups, use:c.add(qty, "aggregate", groups=dims_or_groups, keep=keep, ...)
Or, for
str
or sequence ofstr
dims_or_groups, use:c.add(None, "sum", qty, dimensions=dims_or_groups, ...)
- Parameters:
tag (
str
) – Additional string to add to the end the key for the aggregated quantity.dims_or_groups (
str
orcollections.abc.Iterable
ofstr
ordict
) – Name(s) of the dimension(s) to sum over, or nested dict.weights (
xarray.DataArray
, optional) – Weights for weighted aggregation.keep (
bool
, optional) – Passed tooperator.aggregate
.fail (
str
orint
, optional) – Passed toadd_queue()
viaadd()
.
- Returns:
The key of the newly-added node.
- Return type:
- add_file(*args, **kwargs)[source]
Deprecated.
Deprecated since version 1.18.0: Instead use
add_load_file()
via:c.add(..., "load_file", ...)
- add_product(*args, **kwargs)[source]
Deprecated.
Deprecated since version 1.18.0: Instead use
add_binop()
via:c.add(..., "mul", ...)
- add_queue(queue: Iterable[tuple], max_tries: int = 1, fail: str | int | None = None) tuple[Key | str, ...] [source]
Add tasks from a list or queue.
- Parameters:
queue (
collections.abc.Iterable
oftuple
) – Each item is either a N-tuple
of positional arguments toadd()
, or a 2-tuple
of (tuple
of positional arguments,dict
of keyword arguments).max_tries (
int
, optional) – Retry adding elements up to this many times.fail (
"raise"
orstr
orlogging
level, optional) – Action to take when a computation from queue cannot be added after max_tries: “raise” an exception, or log messages on the indicated level and continue.
- add_single(key: Key | str, *computation, strict=False, index=False) Key | str [source]
Add a single computation at key.
- Parameters:
key (
str
orKey
orhashable
) – A string, Key, or other value identifying the output of computation.strict (
bool
, optional) – If True, key must not already exist in the Computer, and any keys referred to by computation must exist.index (
bool
, optional) – If True, key is added to the index as a full-resolution key, so it can be later retrieved withfull_key()
.
- Raises:
KeyExistsError – If strict is
True
and either (a) key already exists; or (b) sums isTrue
and the key for one of the partial sums of key already exists.MissingKeyError – If strict is
True
and any key referred to by computation does not exist.
- add_tasks(fail_action: int | str = 'raise') None [source]
Add the pre-defined MESSAGEix reporting tasks to the Reporter.
- Parameters:
fail_action (
"raise"
orint
) –logging
level or level name, passed to the fail argument ofReporter.add_queue()
.
- aggregate(qty: Key | str, tag: str, dims_or_groups: Mapping | str | Sequence[str], weights: DataArray | None = None, keep: bool = True, sums: bool = False, fail: str | int | None = None)[source]
Deprecated.
Add a computation that aggregates qty.
Deprecated since version 1.18.0: Instead, for a mapping/
dict
dims_or_groups, use:c.add(qty, "aggregate", groups=dims_or_groups, keep=keep, ...)
Or, for
str
or sequence ofstr
dims_or_groups, use:c.add(None, "sum", qty, dimensions=dims_or_groups, ...)
- Parameters:
tag (
str
) – Additional string to add to the end the key for the aggregated quantity.dims_or_groups (
str
orcollections.abc.Iterable
ofstr
ordict
) – Name(s) of the dimension(s) to sum over, or nested dict.weights (
xarray.DataArray
, optional) – Weights for weighted aggregation.keep (
bool
, optional) – Passed tooperator.aggregate
.fail (
str
orint
, optional) – Passed toadd_queue()
viaadd()
.
- Returns:
The key of the newly-added node.
- Return type:
- apply(generator: Callable, *keys, **kwargs) Key | str | tuple[Key | str, ...] [source]
Add computations by applying generator to keys.
- Parameters:
generator (callable) –
Function to apply to keys. This function may take a first positional argument annotated with
Computer
or a subtype; if so, then it is provided with a reference to self.The function may:
yield
or return an iterable of (key, computation). These are used to directly update thegraph
, and thenapply()
returns the added keys.If it is provided with a reference to the Computer, call
add()
or any other method to update the graph. In this case, it shouldreturn
aKey
or sequence of keys, indicating what was added; these are in turn returned byapply()
.
keys (
Hashable
) – The starting key(s). These are provided as positional arguments to generator.kwargs – Keyword arguments to generator.
- check_keys(*keys: str | Key, predicate=None, action='raise') list[Key | str] [source]
Check that keys are in the Computer.
- Parameters:
- Returns:
One item for each item
k
in keys:k
itself, unchanged, if predicate is given andpredicate(k)
returnsTrue
.Graph.unsorted_key()
, that is,k
but with its dimensions in a specific order that already appears ingraph
.Graph.full_key()
, that is, an existing key with the namek
with its full dimensionality.None
otherwise.
- Return type:
- Raises:
MissingKeyError – If action is “raise” and 1 or more of keys do not appear (either in different dimension order, or full dimensionality) in the
graph
.
- configure(path: Path | str | None = None, fail: str | int = 'raise', config: Mapping[str, Any] | None = None, **config_kw)[source]
Configure the Computer.
Accepts a path to a configuration file and/or keyword arguments. Configuration keys loaded from file are superseded by keyword arguments. Messages are logged at level
logging.INFO
if config contains unhandled sections.See config for a list of all configuration sections and keys, and details of the configuration file format.
- Parameters:
path (
pathlib.Path
, optional) – Path to a configuration file in JSON or YAML format.fail (
"raise"
orstr
orlogging
level, optional) – Passed toadd_queue()
. If not “raise”, then log messages are generated for config handlers that fail. The Computer may be only partially configured.config – Configuration keys/sections and values, as a mapping. Use this if any of the keys/sections are not valid Python names, for instance if they contain “-” or “ “.
**config_kw – Configuration keys/sections and values, as keyword arguments.
- convert_pyam(*args, **kwargs)[source]
Deprecated.
Deprecated since version 1.18.0: Instead use
add_as_pyam()
via:c.require_compat("pyam") c.add(..., "as_pyam", ...)
- default_key: 'genno.core.key.KeyLike' | None = None[source]
The default key to
get()
with no argument.
- describe(key=None, quiet=True)[source]
Return a string describing the computations that produce key.
If key is not provided, all keys in the Computer are described.
Unless quiet, the string is also printed to the console.
- Returns:
Description of computations.
- Return type:
- disaggregate(qty, new_dim, method='shares', args=[])[source]
Deprecated.
Deprecated since version 1.18.0: Instead, for method = “disaggregate_shares”, use:
c = Computer() c.add(qty.append(new_dim), "mul", qty, ..., strict=True)
Or for a
callable()
method, use:c.add(qty.append(new_dim), method, qty, ..., strict=True)
- eval(expr: str) tuple[Key, ...] [source]
Evaluate expr to add tasks and keys.
Parse a statement or block of statements using
ast
from the Python standard library. expr may include:Constants.
References to existing keys in the Computer by their name; these are expanded using
full_key()
.Multiple statements on separate lines or separated by “;”.
Python arithmetic operators including
+
,-
,*
,/
,**
; these are mapped to the correspondingoperator
.Function calls, also mapped to the corresponding
operator
viaget_operator()
. These may include simple positional (constants or key references) or keyword (constants only) arguments.
- Parameters:
expr (
str
) – Expression to be evaluated.- Returns:
One key for the left-hand side of each expression.
- Return type:
- Raises:
NotImplementedError – For complex expressions not supported; if any of the statements is anything other than a simple assignment.
NameError – If a function call references a non-existent computation.
- finalize(scenario: Scenario) None [source]
Prepare the Reporter to act on scenario.
The
TimeSeries
(thus alsoScenario
ormessage_ix.Scenario
) object scenario is stored with the key'scenario'
. All subsequent processing will act on data from this Scenario.
- classmethod from_scenario(scenario, **kwargs) Reporter [source]
Create a Reporter by introspecting scenario.
Warnings are logged if scenario does not have a solution. In this case, any keys/computations based on model output (ixmp variables and equations) may return an empty Quantity, fail, or behave unpredictably. Keys/computations based only on model input (ixmp sets and parameters) should function normally.
- Returns:
A reporter for scenario.
- Return type:
- full_key(name_or_key: Key | str) Key | str [source]
Return the full-dimensionality key for name_or_key.
An quantity ‘foo’ with dimensions (a, c, n, q, x) is available in the Computer as
'foo:a-c-n-q-x'
. ThisKey
can be retrieved with:c.full_key("foo") c.full_key("foo:c") # etc.
- Raises:
KeyError – if name_or_key is not in the graph.
- get(key=None)[source]
Execute and return the result of the computation key.
Only key and its dependencies are computed.
- Parameters:
key (
str
, optional) – If not provided,default_key
is used.- Raises:
ValueError – If key and
default_key
are bothNone
.
- get_comp(name) Callable | None [source]
Return a function,
Operator
, or callable for use in a task.get_operator()
checks each of themodules
for a callable with the given name. Modules at the end of the list take precedence over those earlier in the list.
- get_operator(name) Callable | None [source]
Return a function,
Operator
, or callable for use in a task.get_operator()
checks each of themodules
for a callable with the given name. Modules at the end of the list take precedence over those earlier in the list.
- graph: genno.core.graph.Graph = {'config': {}}[source]
- infer_keys(key_or_keys: Key | str | Iterable[Key | str], dims: Iterable[str] = [])[source]
Infer complete key_or_keys.
Each return value is one of:
a
Key
with eitherdimensions dims, if any are given, otherwise
its full dimensionality (cf.
full_key()
)
str
, the same as input, if the key is not defined in the Computer.
- modules: MutableSequence[types.ModuleType] = [][source]
List of modules containing operators.
By default, this includes the
genno
built-in operators ingenno.operator
.require_compat()
appends additional modules, for instancegenno.compat.plotnine
, to this list. User code may also add modules to this list directly.
- require_compat(pkg: str | ModuleType)[source]
Register a module for
get_operator()
.The specified module is appended to
modules
.- Parameters:
pkg (
str
ormodule
) –One of:
the name of a package (for instance “plotnine”), corresponding to a submodule of
genno.compat
(genno.compat.plotnine
).genno.compat.{pkg}.operator
is added.the name of any importable module, for instance “foo.bar”.
a module object that has already been imported.
- Raises:
ModuleNotFoundError – If the required packages are missing.
Examples
Operators packaged with genno for compatibility:
>>> c = Computer() >>> c.require_compat("pyam")
Operators in another module, using the module name:
>>> c.require_compat("ixmp.reporting.computations")
or using imported module object directly:
>>> import ixmp.reporting.computations as mod >>> c.require_compat(mod)
- set_filters(**filters) None [source]
Apply filters ex ante (before computations occur).
See the description of
filters()
under Configuration.
- property unit_registry[source]
The
pint.UnitRegistry
used by the Computer.
- visualize(filename, key=None, optimize_graph=False, **kwargs)[source]
Generate an image describing the Computer structure.
This is similar to
dask.visualize()
; seecompat.graphviz.visualize()
. Requires graphviz.
- message_ix.report.TASKS0: Tuple[Tuple, ...] = (('map_addon', 'map_as_qty', 'cat_addon', 't'), ('map_emission', 'map_as_qty', 'cat_emission', 'e'), ('map_tec', 'map_as_qty', 'cat_tec', 't'), ('map_year', 'map_as_qty', 'cat_year', 'y'), ('out', 'mul', 'output', 'ACT'), ('in', 'mul', 'input', 'ACT'), ('rel', 'mul', 'relation_activity', 'ACT'), ('emi', 'mul', 'emission_factor', 'ACT'), ('inv', 'mul', 'inv_cost', 'CAP_NEW'), ('fom', 'mul', 'fix_cost', 'CAP'), ('vom', 'mul', 'var_cost', 'ACT'), ('land_out', 'mul', 'land_output', 'LAND'), ('land_use_qty', 'mul', 'land_use', 'LAND'), ('land_emi', 'mul', 'land_emission', 'LAND'), ('y::model', 'model_periods', 'y', 'cat_year'), ('y0', operator.itemgetter(0), 'y::model'), ('tom', 'add', 'fom:nl-t-yv-ya', 'vom:nl-t-yv-ya'), (('addon conversion:nl-t-yv-ya-m-h-ta', <function broadcast_map>, 'addon_conversion:n-t-yv-ya-m-h-type_addon', 'map_addon'), {'rename': {'n': 'nl'}}), ('addon ACT', 'mul', 'addon conversion', 'ACT'), ('addon in', 'mul', 'input', 'addon ACT'), ('addon out', 'mul', 'output', 'addon ACT'), (('addon up:nl-t-ya-m-h-ta', <function broadcast_map>, 'addon_up:n-t-ya-m-h-type_addon', 'map_addon'), {'rename': {'n': 'nl'}}), ('addon potential', 'mul', 'addon up', 'addon ACT'), ('price emission:n-e-t-y', <function broadcast_map>, (<function broadcast_map>, 'PRICE_EMISSION:n-type_emission-type_tec-y', 'map_emission'), 'map_tec'))[source]
Common reporting tasks. These include:
MESSAGE mapping sets, converted to reporting quantities via
map_as_qty()
.For instance, the mapping set
cat_addon
is available at the reporting keymap_addon
, which produces agenno.Quantity
with the two dimensionstype_addon
andta
(short form oftechnology_addon
). This Quantity contains the value 1 at every valid (type_addon, ta) location, and 0 elsewhere.Simple products of 2 or mode quantities.
Other derived quantities.
- message_ix.report.PYAM_CONVERT: List[Tuple[str, CollapseMessageColsKw]] = [('out:nl-t-ya-m-nd-c-l', {'kind': 'ene', 'var': 'out'}), ('in:nl-t-ya-m-no-c-l', {'kind': 'ene', 'var': 'in'}), ('CAP:nl-t-ya', {'var': 'capacity'}), ('CAP_NEW:nl-t-yv', {'var': 'new capacity'}), ('inv:nl-t-yv', {'var': 'inv cost'}), ('fom:nl-t-ya', {'var': 'fom cost'}), ('vom:nl-t-ya', {'var': 'vom cost'}), ('tom:nl-t-ya', {'var': 'total om cost'}), ('emi:nl-t-ya-m-e', {'kind': 'emi', 'var': 'emis'})][source]
Quantities to automatically convert to IAMC format using
as_pyam()
.
- message_ix.report.TASKS1 = (('message::system', 'concat', 'out::pyam', 'in::pyam', 'CAP::pyam', 'CAP_NEW::pyam'), ('message::costs', 'concat', 'inv::pyam', 'fom::pyam', 'vom::pyam', 'tom::pyam'), ('message::emissions', 'concat', 'emi::pyam'), ('message::default', 'concat', 'message::system', 'message::costs', 'message::emissions'))[source]
Automatic reports that
concat()
quantities converted to IAMC format.
- exception message_ix.report.ComputationError(exc)[source]
Wrapper to print intelligible exception information for
Computer.get()
.In order to aid in debugging, this helper:
Omits the parts of the stack trace that are internal to
Dask
, andGives the key in the
Computer.graph
and the computation/task that caused the exception.
- class message_ix.report.Key(name_or_value: str | Key | AnyQuantity, dims: Iterable[str] = [], tag: str | None = None, _fast: bool = False)[source]
A hashable key for a quantity that includes its dimensionality.
- classmethod bare_name(value) str | None [source]
If value is a bare name (no dims or tags), return it; else
None
.
- classmethod from_str_or_key(value: str | Key | AnyQuantity, drop: Iterable[str] | bool = [], append: Iterable[str] = [], tag: str | None = None) Key [source]
Return a new Key from value.
Changed in version 1.18.0: Calling
from_str_or_key()
with a single argument is no longer necessary; simply give the same value as an argument toKey
.The class method is retained for convenience when calling with multiple arguments. However, the following are equivalent and may be more readable:
k1 = Key("foo:a-b-c:t1", drop="b", append="d", tag="t2") k2 = Key("foo:a-b-c:t1").drop("b").append("d)"
- Parameters:
drop (
list
ofstr
orTrue
, optional) – Existing dimensions of value to drop. Seedrop()
.append (
list
ofstr
, optional) – New dimensions to append to the returned Key. Seeappend()
.tag (
str
, optional) – Tag for returned Key. If value has a tag, the two are joined using a ‘+’ character. Seeadd_tag()
.
- Return type:
- iter_sums() Generator[tuple[Key, Callable, Key], None, None] [source]
Generate (key, task) for all possible partial sums of the Key.
- classmethod product(new_name: str, *keys, tag: str | None = None) Key [source]
Return a new Key that has the union of dimensions on keys.
Dimensions are ordered by their first appearance:
First, the dimensions of the first of the keys.
Next, any additional dimensions in the second of the keys that were not already added in step 1.
etc.
- Parameters:
new_name (
str
) – Name for the new Key. The names of keys are discarded.
- remove_tag(*tags: str) Key [source]
Return a key with any of tags dropped.
- Raises:
ValueError – If none of tags are in
tags
.
- exception message_ix.report.KeyExistsError[source]
Raised by
Computer.add()
when the target key exists.
- exception message_ix.report.MissingKeyError[source]
Raised by
Computer.add()
when a required input key is missing.
- message_ix.report.Quantity[source]
alias of
AttrSeries
- message_ix.report.configure(path: Path | str | None = None, **config)[source]
Configure
genno
globally.Modifies global variables that affect the behaviour of all Computers and operators. Configuration keys loaded from file are superseded by keyword arguments. Messages are logged at level
logging.INFO
if config contains unhandled sections.- Parameters:
path (
pathlib.Path
, optional) – Path to a configuration file in JSON or YAML format.**config – Configuration keys/sections and values.
Operators
message_ix.report
provides a small number of operators.
Two of these (plot_cumulative()
and stacked_bar()
) are currently only used in the tutorials to produce simple plots; for more flexible plotting, genno.compat.plotnine
is recommended instead.
Convert qty to an |
|
|
Return the elements of y beyond the firstmodelyear of cat_year. |
|
Plot a supply curve. |
|
Plot qty as a stacked bar chart. |
Other operators are provided by ixmp.report
:
|
Retrieve data from scenario. |
|
Return a |
|
Retrieve timeseries data from ts. |
|
Convert set_df to a |
|
Remove all time series data from ts. |
|
Store time series data on scenario. |
|
Update scenario with computed data from reporting quantities. |
…and by genno.operator
and its compatibility modules.
See the package documentation for details.
|
Class for plotting using |
Sum across multiple quantities. |
|
|
Aggregate quantity by groups. |
|
Apply units to qty. |
Return a |
|
|
Broadcast quantity using a map. |
|
Sum distinct quantities by weights. |
|
Concatenate Quantity objs. |
Compute the ratio numerator / denominator. |
|
|
Return a Quantity with dropped variables (coordinates). |
|
Group by dimension group, then sum across dimension sum. |
|
Compute an index of qty against certain of its values. |
|
Interpolate qty. |
Read the file at path and return its contents as a |
|
Compute the product of any number of quantities. |
|
|
Compute a raised to the power of b. |
|
Replace specific labels along dimensions of qty. |
|
Returns a new Quantity with renamed dimensions or a new name. |
|
Like |
|
Select from qty based on indexers. |
Subtract b from a. |
|
Sum quantity over dimensions, with optional weights. |
|
|
Write a quantity to a file. |
|
Deprecated: Disaggregate quantity by shares. |
Alias of |
|
Alias of |
- message_ix.report.operator.as_message_df(qty: AnyQuantity, name: str, dims: Mapping, common: Mapping, wrap: Literal[True] = True) Dict[str, DataFrame] [source]
- message_ix.report.operator.as_message_df(qty: AnyQuantity, name: str, dims: Mapping, common: Mapping, wrap: Literal[False]) DataFrame
Convert qty to an
add_par()
-ready data frame usingmake_df()
.The resulting data frame has:
A “value” column populated with the values of qty.
A “unit” column with the string representation of the units of qty.
Other dimensions/key columns filled with labels of qty according to dims.
Other dimensions/key columns filled with uniform values from common.
- Parameters:
qty (
genno.Quantity
)name (
str
) – Name of the MESSAGEix parameter to prepare.dims (
collections.abc.Mapping
) – Each key corresponds to a dimension of the target parameter name, for instance “node_loc”; the label corresponds to a dimension of qty, for instance “nl”.common (
collections.abc.Mapping
) – Each key corresponds to a dimension of the target parameter; values are used literally, as if passed tomake_df()
.wrap (
bool
, optional) – See below.
- Returns:
dict
– if wrap isTrue
(the default): length 1, mapping from name topandas.DataFrame
containing the converted data.pandas.DataFrame
– if wrap isFalse
.
- message_ix.report.operator.model_periods(y: List[int], cat_year: DataFrame) List[int] [source]
Return the elements of y beyond the firstmodelyear of cat_year.
- message_ix.report.operator.plot_cumulative(x: AnyQuantity, y: AnyQuantity, labels: Tuple[str, str, str])[source]
Plot a supply curve.
x and y must share the first two dimensions.
The first dimension must contain unique values.
One rectangle is plotted for each unique value in the second dimension.
- Parameters:
x (
genno.Quantity
) – e.g.<resource_volume:n-g>
.y (
genno.Quantity
) – e.g.<resource_cost:n-g-y>
. Themean()
is taken across the third dimension.
- message_ix.report.operator.stacked_bar(qty: AnyQuantity, dims: Tuple[str, ...] = ('nl', 't', 'ya'), units: str = '', title: str = '', cf: float = 1.0, stacked: bool = True)[source]
Plot qty as a stacked bar chart.
- Parameters:
qty (
genno.Quantity
) – Data to plot.3 or more dimensions for, respectively:
1 dimension: The node/region.
1 or more dimensions: to stack in bars of different colour.
1 dimension: The ordinate (x-axis); typically the year.
units (
str
) – Units to display on the plot.title (
str
) – Title fragment; the plot title is “{node} {title}”.cf (
float
, optional) – Conversion factor to apply to data.
Utilities
- message_ix.report.pyam.collapse_message_cols(df: pandas.DataFrame, var: str | None = None, kind: str | None = None, var_cols=[]) pandas.DataFrame [source]
genno.compat.pyam
collapse=… callback for MESSAGEix quantities.Wraps
collapse()
with arguments particular to MESSAGEix.- Parameters:
var (
str
) – Name for ‘variable’ column.kind (
None
or'ene'
or'emi'
, optional) –Determines which other columns are combined into the ‘region’ and ‘variable’ columns:
’ene’: ‘variable’ is
'<var>|<level>|<commodity>|<technology>|<mode>'
and ‘region’ is'<region>|<node_dest>'
(if var=’out’) or'<region>|<node_origin>'
(if ‘var=’in’).’emi’: ‘variable’ is
'<var>|<emission>|<technology>|<mode>'
.Otherwise: ‘variable’ is
'<var>|<technology>'
.