Reporting (`report`)

On this page:

Elsewhere:

global.yaml, the Default reporting configuration.
Documentation for genno (genno: efficient, transparent calculation on N-D data), ixmp.report, and message_ix.report.
Reporting for specific model variants:
- water.reporting
- transport/output of model.transport
‘Legacy’ reporting.

Introduction 

See the discussion in the MESSAGEix docs about the stack. In short, for instance:

message_ix must not contain reporting code that references technology="coal_ppl", because not every model built on the MESSAGE framework will have a technology with this name.
Any model in the MESSAGEix-GLOBIOM family —built with message_ix_models and/or message_data— should, with few exceptions, have a technology="coal_ppl", since this appears in the common list of Technologies (technology.yaml). Reporting specific to this technology ID, as it is represented in this model family, should be in message_ix_models or user code.

The basic design pattern of message_ix_models.report is:

prepare_reporter() populates a new Reporter for a given Scenario with many tasks to report all quantities of interest in a MESSAGEix-GLOBIOM–family model.
This function relies on callbacks defined in multiple submodules to add keys and tasks for general or tailored reporting calculations and actions. Additional modules should define callback functions and register them with register() when they are to be used. For example:
1. The module message_ix_models.report.plot defines plot.callback() that adds standard plots to the Reporter.
2. The module message_ix_models.model.transport.report defines callback() that adds tasks specific to the MESSAGEix-Transport model variant.
3. The module message_ix_models.project.navigate.report defines callback()
  that add tasks specific to the ‘NAVIGATE’ research project.
The callback (1) is always registered, because these plots are always applicable and can be expected to function correctly for all models in the family. In contrast, (2) and (3) should only be registered and run for the specific model variants for which they are developed/intended.

Modules with tailored reporting configuration may also be indicated on the command line by using the -m/--modules option: mix-models report -m model.transport.
A file global.yaml (in YAML format) contains a description of some of the reporting computations needed for a MESSAGE-GLOBIOM model. prepare_reporter() uses the configuration handlers built into genno (and some extensions specific to message_ix_models) to handle the different sections of the file.

Features 

By combining these genno, ixmp, message_ix, and message_ix_models features, the following functionality is provided.

Note

If any of this does not appear to work as advertised, file a bug!

Scope of `genno`-based vs. `legacy` reporting

prepare_reporter() currently handles only a subset of the IAMC-structured data produced by report.legacy. The module globals NOT_IMPLEMENTED_MEASURE and NOT_IMPLEMENTED_IAMC express the measures and IAMC ‘variables’ that are either not implemented or not exactly matching the legacy reporting output. The test test_report.test_compare() compares outputs from the two forms of reporting using iamc.compare(), ensuring that data match for all entries except these exclusions.

See GitHub issues and pull requests with the ‘report’ label for ongoing work to expand the scope of prepare_reporter().

Units 

Are read automatically for ixmp parameters.
Pass through calculations/are derived automatically.
Are recognized based on the definitions of non-SI units from IAMconsortium/units.
Are discarded when inconsistent.
Can be overridden for entire parameters:
```
units:
  apply:
    inv_cost: USD
```

Can be set explicitly when converting data to IAMC format:

iamc:
# 'value' will be in kJ; 'units' will be the string 'kJ'
- variable: Variable Name
  base: example_var:a-b-c
  units: kJ

API reference 

General-purpose code:

`Config`(from_file, _legacy, cli_output, key, ...)	Settings for `message_ix_models.report`.
`defaults`(rep, context)	Prepare default contents and configuration of rep for MESSAGEix-GLOBIOM.
`prepare_reporter`(context[, scenario, reporter])	Return a `Reporter` and key prepared to report a `Scenario`.
`register`(name_or_callback)	Deprecated alias for `report.Config.register()`.
`report`(context, args, *kwargs)	Report (post-process) solution data in a `Scenario`.

The following submodules prepare reporting of specific measures or quantities:

extraction

Report resource extraction.

class message_ix_models.report.Config(from_file: dataclasses.InitVar[pathlib._local.Path | str | None] = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/iiasa-energy-program-message-ix-models/envs/latest/lib/python3.13/site-packages/message_ix_models/data/report/global.yaml'), _legacy: dataclasses.InitVar[bool | None] = False, cli_output: ~pathlib.Path | None = None, key: KeyLike | None = None, modules: list[str] = <factory>, use_scenario_path: bool = True, *, callback: list[~collections.abc.Callable[[~message_ix_models.report.config.ComputerT, Context], None]] = <factory>, genno_config: dict = <factory>, output_dir: ~pathlib.Path = <factory>, legacy: dict = <factory>)[source]

Settings for message_ix_models.report.

When initializing a new instance, the from_file and _legacy parameters are respected.

callback: list[Callable[[ComputerT, Context], None]]

List of callback functions for preparing the Reporter.

Each registered function is called by prepare_reporter() in order to add or modify the task graph. Specific model variants and projects can register a callback to extend the reporting graph. The default list is:

A callback function must take two arguments: the Computer/Reporter, and a Context:

from message_ix.report import Reporter
from message_ix_models import Context

def cb(rep: Reporter, ctx: Context) -> None:
    # Modify `rep` by calling its methods ...
    pass

# Register this callback on an existing Context/report.Config instance
context.report.register(cb)

Keys 

Keys for setting up reporting tasks.

Pre-set genno.Key instances and genno.Keys in this module can be referenced across modules, instead of hand-typing the same strings in multiple places.

message_ix_models.report.key.GDP = <GDP:n-y>: Gross domestic product.

message_ix_models.report.key.PRICE_COMMODITY = <PRICE_COMMODITY:n-c-l-y-h>: Commodity price.

Note

genno ≤ 1.27.1 is sensitive to the dimension order; more recent versions are not.

message_ix_models.report.key.all_iamc = <all::iamc>: All IAMC-structured data.

message_ix_models.report.key.coords = <1 keys: n_glb>

Identifiers for coordinates, including:

.n_glb: the output of node_glb().

message_ix_models.report.key.groups = <1 keys: c>

Identifiers for grouping/aggregation mappings, including:

.c: the output of get_commodity_groups().

Plots 

Plots for MESSAGEix-GLOBIOM reporting.

The current set functions on time series data stored on the scenario by message_ix_models.report or message_data legacy reporting.

class message_ix_models.report.plot.EmissionsCO2[source]

CO₂ Emissions.

basename = 'emission-CO2': File name base for saving the plot.

generate(data: DataFrame, scenario: Scenario)[source]

Generate and return the plot.

A subclass of Plot must implement this method.

Parameters:

args (Sequence of pandas.DataFrame or other) –

One argument is given corresponding to each of the inputs.

Because plotnine operates on pandas data structures, save() automatically converts any Quantity inputs to pandas.DataFrame before they are passed to generate().

inputs: Sequence[str] = ['Emissions|CO2::iamc', 'scenario']: Keys referring to Quantities or other inputs accepted by generate().

static: list['PlotAddable'] = [<plotnine.themes.theme.theme object>, {'color': 'region', 'x': 'year', 'y': 'value'}, <plotnine.geoms.geom_line.geom_line object>, <plotnine.geoms.geom_point.geom_point object>, labs(x='Period', y='', alpha=None, color='Region', colour=None, fill=None, linetype=None, shape=None, size=None, stroke=None, title=None, subtitle=None, caption=None, tag=None)]

list of plotnine objects that are not dynamic.

Type:: ‘Static’ geoms

class message_ix_models.report.plot.FinalEnergy0[source]

Final Energy.

basename = 'fe0': File name base for saving the plot.

inputs: Sequence[str] = ['Final Energy::iamc', 'scenario']: Keys referring to Quantities or other inputs accepted by generate().

class message_ix_models.report.plot.FinalEnergy1[source]

Final Energy.

basename = 'fe1': File name base for saving the plot.

generate(data: DataFrame, scenario: Scenario)[source]

Generate and return the plot.

A subclass of Plot must implement this method.

Parameters:

args (Sequence of pandas.DataFrame or other) –

One argument is given corresponding to each of the inputs.

Because plotnine operates on pandas data structures, save() automatically converts any Quantity inputs to pandas.DataFrame before they are passed to generate().

inputs: Sequence[str] = ['fe1-0::iamc', 'scenario']: Keys referring to Quantities or other inputs accepted by generate().

inputs_regex: list[re.Pattern] = [re.compile('Final Energy\\|(Electricity|Gases|Geothermal|Heat|Hydrogen|Liquids|Solar|Solids)')]: List of regular expressions corresponding to inputs. These are passed as the expr argument to filter_ts() to filter the entire set of time series data.

static: list['PlotAddable'] = [<plotnine.themes.theme.theme object>, {'fill': 'variable', 'x': 'year', 'y': 'value'}, <plotnine.geoms.geom_bar.geom_bar object>, labs(x='Period', y='', alpha=None, color=None, colour=None, fill='Commodity', linetype=None, shape=None, size=None, stroke=None, title=None, subtitle=None, caption=None, tag=None)]

list of plotnine objects that are not dynamic.

Type:: ‘Static’ geoms

message_ix_models.report.plot.PLOTS = (<class 'message_ix_models.report.plot.EmissionsCO2'>, <class 'message_ix_models.report.plot.FinalEnergy0'>, <class 'message_ix_models.report.plot.FinalEnergy1'>, <class 'message_ix_models.report.plot.PrimaryEnergy0'>, <class 'message_ix_models.report.plot.PrimaryEnergy1'>): All plot classes.

class message_ix_models.report.plot.Plot[source]

Base class for plots based on reported time-series data.

Subclasses should be used like:

class MyPlot(Plot):
    ...

c.add("plot myplot", MyPlot, "scenario")

…that is, giving “scenario” or another key that points to a Scenario object with stored time series data. See the examples in this file.

classmethod add_tasks(c: Computer, key: KeyLike, *inputs, strict: bool = False) → KeyLike[source]

Add a task to c to generate and save the Plot.

Analogous to Operator.add_tasks().

ggtitle(value=None) → ggtitle[source]: Return plotnine.ggtitle including the current date & time.

groupby_plot(data: DataFrame, *args)[source]

Combination of groupby and ggplot().

Groups by args and yields a series of plotnine.ggplot objects, one per group, with static geoms and ggtitle() appended to each.

inputs: Sequence[str] = []: Keys referring to Quantities or other inputs accepted by generate().

inputs_regex: list[Pattern] = []: List of regular expressions corresponding to inputs. These are passed as the expr argument to filter_ts() to filter the entire set of time series data.

static: list[PlotAddable] = [<plotnine.themes.theme.theme object>]

list of plotnine objects that are not dynamic.

Type:: ‘Static’ geoms

title = None: Fixed plot title string. If not given, the first line of the class docstring is used.

unit = None: Units expression for plot title.

url: str | None = None: Scenario URL for plot title.

class message_ix_models.report.plot.PrimaryEnergy0[source]

Primary Energy.

basename = 'pe0': File name base for saving the plot.

inputs: Sequence[str] = ['Primary Energy::iamc', 'scenario']: Keys referring to Quantities or other inputs accepted by generate().

class message_ix_models.report.plot.PrimaryEnergy1[source]

Primary Energy.

basename = 'pe1': File name base for saving the plot.

inputs: Sequence[str] = ['pe1-0::iamc', 'scenario']: Keys referring to Quantities or other inputs accepted by generate().

inputs_regex: list[re.Pattern] = [re.compile('Primary Energy\\|((?!Fossil|Non-Biomass Renewables|Secondary Energy Trade)[^\\|]*)')]: List of regular expressions corresponding to inputs. These are passed as the expr argument to filter_ts() to filter the entire set of time series data.

message_ix_models.report.plot.callback(c: Computer, context: Context) → None[source]

Add all PLOTS to c.

Also add a key “plot all” to triggers the generation of all plots.

Operators 

Atomic reporting operations for MESSAGEix-GLOBIOM.

message_ix_models.report.operator provides the following:

`broadcast_wildcard`(qty, *coords[, dim])	Broadcast over coordinates coords along respective dimension(s) dim.
`call`(callable, args, *kwargs)	Invoke a callable on other arguments.
`codelist_to_groups`(codes[, dim])	Convert codes into a mapping from parent items to their children.
`compound_growth`(qty, dim)	Compute compound growth along dim of qty.
`filter_ts`(df, expr, *[, column])	Filter time series data in df.
`from_url`(url[, cls])	Return a `ixmp.TimeSeries` or subclass instance, given its url.
`get_commodity_groups`()	Return groups of commodities for aggregation.
`get_ts`(scenario[, filters, iamc, subannual])	Retrieve timeseries data from scenario.
`gwp_factors`()	Use `iam_units` to generate a Quantity of GWP factors.
`make_output_path`(config, name)	Return a path under the "output_dir" Path from the reporter configuration.
`model_periods`(y, cat_year)	Return the elements of y beyond the firstmodelyear of cat_year.
`node_glb`(nodes)	Return `{"n": ["R##_GLB"]}` derived from existing nodes.
`nodes_world_agg`(config, *[, dim, name])	Mapping to aggregate e.g. nl="World" from values for child nodes of "World".
`remove_ts`(scenario[, config, after, dump])	Remove all time series data from scenario.
`select_allow_empty`(qty, indexers, *[, ...])	`genno.operator.select()` allowing for missing data.
`select_expand`(qty, dim_cb)	Select and expand dimensions using callbacks for existing coordinates.
`share_curtailment`(curt, *parts)	Apply a share of curt to the first of parts.
`zeros_like`(qty, *[, drop])	Return a quantity with the same coords as qty, filled with zeros.

The following functions, defined elsewhere, are exposed through operator and so can also be referenced by name:

`message_ix_models.util.add_par_data`(...[, ...])	Add data to scenario.
`message_ix_models.util.merge_data`(base, *others)	Merge dictionaries of DataFrames together into base.
`message_ix_models.util.nodes_ex_world`(nodes)	Exclude "World" and anything containing "GLB" from nodes.

Other operators or genno-compatible functions are provided by:

Upstream packages:
Other submodules:
- model.emissions: get_emission_factors().

Any of these can be made available for a Computer instance using require_compat(), for instance:

# Indicate that a certain module contains functions to
# be referenced by name
c.require_compat("message_ix_models.model.emissions")

# Add computations to the graph by referencing functions
c.add("ef:c", "get_emission_factors", units="t C / kWa")

message_ix_models.report.operator.broadcast_wildcard(qty: TQuantity, *coords: Sequence[SupportsLessThan], dim: Hashable | Sequence[Hashable] = 'n') → TQuantity[source]

Broadcast over coordinates coords along respective dimension(s) dim.

dim may identify a single dimension or a sequence of dimensions; coords must be given for each dimension.

For each respective items from dim and coords, any missing coordinates along the dimension are populated using the values of qty keyed with the ‘wildcard’ label “*”.

message_ix_models.report.operator.call(callable, *args, **kwargs)[source]: Invoke a callable on other arguments.

message_ix_models.report.operator.codelist_to_groups(codes: list[Code], dim: str = 'n') → Mapping[str, Mapping[str, list[str]]][source]

Convert codes into a mapping from parent items to their children.

The returned value is suitable for use with genno.operator.aggregate().

If this is a list of nodes per get_codes(), then the mapping is from regions to the ISO 3166-1 alpha-3 codes of the countries within each region. The code for the region itself is also included in the values to be aggregated, so that already- aggregated data will pass through.

message_ix_models.report.operator.compound_growth(qty: TQuantity, dim: str) → TQuantity[source]: Compute compound growth along dim of qty.

message_ix_models.report.operator.filter_ts(df: DataFrame, expr: Pattern, *, column='variable') → DataFrame[source]

Filter time series data in df.

Keep only rows in df where expr is a full match ( fullmatch()) for the entry in column.
Retain only the first match group (”…(…)…”) from expr as the column entry.

message_ix_models.report.operator.from_url(url: str, cls=<class 'ixmp.core.timeseries.TimeSeries'>) → TimeSeries[source]

Return a ixmp.TimeSeries or subclass instance, given its url.

Todo

Move upstream, to ixmp.report.

Parameters:: cls (type, optional) – Subclass to instantiate and return; for instance, Scenario.

message_ix_models.report.operator.get_commodity_groups() → dict[Literal['c'], dict[str, list[str]]][source]

Return groups of commodities for aggregation.

The structure is retrieved from Commodities (commodity.yaml) using leaf_ids(). The group IDs include only those commodities from the code list with children. The group members include only ‘leaf’ codes via leaf_ids(); any intermediate codes that have children are expanded to the list of those children.

Returns:: with one top-level key, ‘c’, and at the second level mapping from group IDs to their components. This is suitable for use as the groups argument to genno.operator.aggregate().
Return type:: dict

message_ix_models.report.operator.get_ts(scenario: Scenario, filters: dict | None = None, iamc: bool = False, subannual: bool | str = 'auto')[source]

Retrieve timeseries data from scenario.

Corresponds to ixmp.Scenario.timeseries().

Todo

Move upstream, e.g. to ixmp alongside store_ts().

message_ix_models.report.operator.gwp_factors() → AnyQuantity[source]

Use iam_units to generate a Quantity of GWP factors.

The quantity is dimensionless, e.g. for converting [mass] to [mass], andhas dimensions:

‘gwp metric’: the name of a GWP metric, e.g. ‘SAR’, ‘AR4’, ‘AR5’. All metrics are
on a 100-year basis.
‘e’: emissions species, as in MESSAGE. The entry ‘HFC’ is added as an alias for the species ‘HFC134a’ from iam_units.
‘e equivalent’: GWP-equivalent species, always ‘CO2’.

message_ix_models.report.operator.make_output_path(config: Mapping, name: str | Path) → Path[source]: Return a path under the “output_dir” Path from the reporter configuration.

message_ix_models.report.operator.merge_data(*others: ParameterData) → ParameterData[source]

Alternate form of message_ix_models.util.merge_data().

This form returns a new, distinct dict and does not mutate any of its arguments.

message_ix_models.report.operator.model_periods(y: list[int], cat_year: DataFrame) → list[int][source]: Return the elements of y beyond the firstmodelyear of cat_year.

Todo

Move upstream, to message_ix.

message_ix_models.report.operator.node_glb(nodes: list[str]) → dict[str, list[str]][source]: Return {"n": ["R##_GLB"]} derived from existing nodes.

message_ix_models.report.operator.nodes_ex_world(nodes: Sequence[str | Code]) → list[str | Code][source]

Exclude “World” and anything containing “GLB” from nodes.

May also be used as a genno (reporting) operator.

message_ix_models.report.operator.nodes_world_agg(config: dict, *, dim: Hashable = 'nl', name: str | None = '{}_GLB') → Mapping[Hashable, Mapping[Hashable, list[str]]][source]

Mapping to aggregate e.g. nl=”World” from values for child nodes of “World”.

This mapping should be used with genno.operator.aggregate(), giving the argument keep=False. It includes 1:1 mapping from each region name to itself.

message_ix_models.report.operator.quantity_from_iamc(qty: AnyQuantity, variable: str) → AnyQuantity[source]

Extract data for a single measure from qty with (at least) dimensions v, u.

Deprecated since version 2025-02-17: Use genno.compat.pyam.operator.quantity_from_iamc() instead.

Parameters:: variable (str) – Regular expression to match the v dimension of qty.

message_ix_models.report.operator.remove_ts(scenario: Scenario, config: dict | None = None, after: int | None = None, dump: bool = False) → None[source]

Remove all time series data from scenario.

Note that data stored with add_timeseries() using meta=True as a keyword argument cannot be removed using TimeSeries.remove_timeseries(), and thus also not with this operator.

Todo

Move upstream, to ixmp alongside store_ts().

message_ix_models.report.operator.select_allow_empty(qty: TQuantity, indexers: Mapping[Hashable, Iterable[Hashable]], *, inverse: bool = False, drop: bool = False) → TQuantity[source]

genno.operator.select() allowing for missing data.

If any of the indexers are missing from qty, return an empty Quantity with the same dimensions as qty.

Todo

Move upstream to genno.

message_ix_models.report.operator.select_expand(qty: TQuantity, dim_cb: dict[str, Callable[[Hashable], dict[Hashable, Hashable]]]) → TQuantity[source]

Select and expand dimensions using callbacks for existing coordinates.

This combines behaviours of genno.operator.select(), genno.operator.relabel(), and genno.operator.rename(). Specifically, for each (dim, function) in dim_cb:

The function is applied to each coordinate/label along dim of qty.
If the return value is False-y (for example, empty dict), all data indexed by that label is discarded.
If the return value is a non-empty dict, data are preserved. The dim label is replaced by 1+ new dimension(s) and label(s) from the keys and values, respectively, of the return value.

Parameters:: dim_cb – Mapping from dimensions of qty to callback functions. Each function should receive a single coordinate label, and return a dict.

message_ix_models.report.operator.share_curtailment(curt, *parts)[source]

Apply a share of curt to the first of parts.

If this is being used, it usually will indicate the need to split curt into multiple technologies; one for each of parts.

message_ix_models.report.operator.zeros_like(qty: TQuantity, *, drop: Collection[str] = []) → TQuantity[source]: Return a quantity with the same coords as qty, filled with zeros.

Utilities 

`IAMCConversion`(base, var_parts, unit, sums, ...)	Description of a conversion to IAMC data structure.
`add_replacements`(dim, codes)	Update `REPLACE_DIMS` for dimension dim with values from codes.
`collapse`(df[, var])	Callback for the collapse argument to `convert_pyam()`.
`collapse_gwp_info`(df, var)	`collapse()` helper for emissions data with GWP dimensions.
`copy_ts`(rep, other, filters)	Prepare rep to copy time series data from other to scenario.

class message_ix_models.report.util.IAMCConversion(base: ~genno.core.key.Key, var_parts: list[str], unit: str, sums: list[str] = <factory>, GLB_zeros: bool = False)[source]

Description of a conversion to IAMC data structure.

Instance fields contain information needed to prepare the conversion. add_tasks() adds tasks to a Computer to perform it.

GLB_zeros: bool = False: If True, ensure data is present for R##_GLB.

add_tasks(c: Computer) → None[source]

Add tasks to convert base to IAMC structure.

The tasks include, in order:

If GLB_zeroes is True:
- Create a quantity with the same shape as base, filled with all zeros (zeros_like()) and a single coord like R##_GLB for the \(n\) dimension (node_glb()).
- Add this to base.
These steps ensure that values for R##_GLB will appear in the IAMC-structured result.
Convert to the given units (convert_units()). The base quantity must have dimensionally compatible units.

Steps (3) to (6) are repeated for (at least) an empty string ("") and for any expressions like "x-y-z" in sums.

Subtract the given dimension(s) (if any) from the dimensions of base. For example, if base is <foo:x-y-z> and sums includes "x-z", this gives a reference to <foo:y>, which is the base quantity summed over the \((x, z)\) dimensions.
Reduce the var_parts in the same way. For example, if var_parts is ["Variable prefix", "z", "x", "y", "Foo"], the above sum reduces this to ["Variable prefix", "y", "Foo"].
Call genno.compat.pyam.iamc() to add further tasks to convert the quantity from (3) to IAMC structure. callback() in this module is used to help format the individual dimension labels and collapsed ‘variable’ labels.

This step results in keys like base 0::iamc, base 1::iamc, etc. added to rep.
Append the key from (5) to the task at report.key.all_iamc. This ensures that the converted data is concatenated with all other IAMC-structured data.

base: Key: Key for data to be converted.

sums: list[str]: Dimension(s) to sum over.

unit: str: Exact unit string for output.

var_parts: list[str]: Parts of the variable expression. This is passed as the var argument to collapse().

message_ix_models.report.util.REPLACE_DIMS: dict[str, dict[str, str]] = {'c': {'Agri_Ch4': 'GLOBIOM|Emissions|CH4 Emissions Total'}, 'l': {'Final Energy': 'Final Energy|Residential'}, 't': {}}

Replacements used in collapse(). These are applied using pandas.DataFrame.replace() with regex=True; see the documentation of that method.

Applied to whole strings along each dimension.
These columns have str.title() applied before these replacements.

Compatibility with `report.legacy`

Compatibility code that emulates legacy reporting.

report.compat prepares a Reporter to perform the same calculations as report.legacy, except using genno.

Warning

This code is under development and incomplete. It is not yet a full or exact replacement for report.legacy. Use with caution.

Main API:

`TECH_FILTERS`	Filters for determining subsets of technologies.
`callback`(rep, context)	Partially duplicate the behaviour of `default_tables.retr_CO2emi()`.
`prepare_techs`(c, technologies)	Prepare sets of technologies in c.
`get_techs`(c, prefix[, kinds])	Return a list of technologies.

Utility functions:

`inp`(c, technologies, *[, name, filters, ...])
`eff`(c, technologies[, filters_in, filters_out])	Throughput efficiency (input / output) for technologies.
`emi`(c, technologies, *[, name, filters, ...])
`out`(c, technologies, *[, name, filters, ...])

message_ix_models.report.compat.TECH_FILTERS = {'gas all': "c_in == 'gas' and l_in in 'secondary final' and '_ccs' not in id", 'gas extra': 'False', 'rc gas': "sector == 'residential/commercial' and c_in == 'gas'", 'trp coal': "sector == 'transport' and c_in == 'coal'", 'trp foil': "sector == 'transport' and c_in == 'fueloil'", 'trp gas': "sector == 'transport' and c_in == 'gas'", 'trp loil': "sector == 'transport' and c_in == 'lightoil'", 'trp meth': "sector == 'transport' and c_in == 'methanol'"}

Filters for determining subsets of technologies.

Each value is a Python expression eval()’d in an environment containing variables derived from the annotations on Codes for each technology. If the expression evaluates to True, then the code belongs to the set identified by the key.

Command-line interface 

$ mix-models report --help

Usage: mix-models report [OPTIONS] [KEY]

  Postprocess results.

  KEY defaults to the comprehensive report 'message::default', but may also be
  the name of a specific model quantity, e.g. 'output'.

  --config can give either the absolute path to a reporting configuration
  file, or the stem (i.e. name without .yaml extension) of a file in
  data/report.

  With --from-file, read multiple Scenario identifiers from FILE, and report
  each one. In this usage, --output-path may only be a directory.

Options:
  --dry-run             Only show what would be done.
  --config TEXT         Path or stem for reporting config file.  [default:
                        global]
  -L, --legacy          Invoke legacy reporting.
  -m, --module MODULES  Add extra reporting for MODULES.
  -o, --output PATH     Write output to file instead of console.
  --from-file FILE      Report multiple Scenarios listed in FILE.
  --help                Show this message and exit.

Testing 

Simulated solution data for testing report.

message_ix_models.report.sim.add_simulated_solution(rep: Reporter, info: ScenarioInfo, data: dict | None = None, path: Path | None = None)[source]

Add a simulated model solution to rep.

Parameters:

data (dict or pandas.DataFrame, optional) – If given, a mapping from MESSAGE item (set, parameter, or variable) names to inputs that are passed to simulate_qty().
path (Path, optional) – If given, a path to a directory containing one or more files with names like ACT.csv.gz. These files are taken as containing “simulated” model solution data for the MESSAGE variable with the same name. See data_from_file().

message_ix_models.report.sim.data_from_file(path: Path, *, name: str, dims: Sequence[str]) → AnyQuantity[source]

Read simulated solution data for item name from path.

For variables and equations (name in upper case), the file must have columns corresponding to dims followed by “Val”, “Marginal”, “Upper”, and “Scale”. The “Val” column is returned.

For parameters, the file must have columns corresponding to dims followed by “value” and “unit”. The “value” column is returned.

message_ix_models.report.sim.simulate_qty(name: str, dims: list[str], item_data: dict | DataFrame) → AnyQuantity[source]

Return simulated data for item name.

Parameters:

dims – Dimensions of the resulting quantity.
item_data – Optional data for the quantity.

message_ix_models.report.sim.to_simulate()[source]: Return items to be included in a simulated solution.

Continuous reporting 

As part of the Test suite (message_ix_models.tests), reporting is run on the same events (pushes and daily schedule) on publicly-available model snapshots. One goal of these tests inter alia is to ensure that adjustments and improvements to the reporting code do not disturb manually-verified model outputs.

As part of the (private) message_data test suite, multiple workflows run on regular schedules; some of these include a combination of message_ix_models-based and ‘legacy’ reporting. These workflows:

Operate on specific scenarios within IIASA databases.
Create files in CSV, Excel, and/or PDF formats that are that are preserved and made available as ‘build artifacts’ via the GitHub Actions web interface and API.

Reporting (report)

Scope of genno-based vs. legacy reporting

Reporting (`report`)

Scope of `genno`-based vs. `legacy` reporting