File formats and input/output
In addition to the data management features provided by Storage back ends (ixmp.backend), ixmp is able to write and read
Scenario data to and from files.
This page describes those options and formats.
Time series data
Time series data can be:
import_timeseries(), or the CLI command
ixmp import timeseries FILEfor a single TimeSeries object.
export_timeseries_data()for multiple TimeSeries objects at once.
Both CSV and Excel files in the IAMC time-series format are supported.
Scenario data can be read from/written to Microsoft Excel files using
to_excel(), and the CLI commands
ixmp import scenario FILE and
ixmp export FILE.
The files have the following structure:
One sheet named ‘ix_type_mapping’ with two columns:
‘item’: the name of an ixmp item.
‘ix_type’: the item’s type as a length-3 string: ‘set’, ‘par’, ‘var’, or ‘equ’.
One or more sheet per item. If the length of data is greater than the maximum number of rows per sheet supported by the Excel file format (
EXCEL_MAX_ROWS), the item is split across multiple sheets named, e.g., ‘foo’, ‘foo(2)’, ‘foo(3)’.
Sheets for one-dimensional indexed sets have one column, with a header cell that is the index set name.
Sheets for multi-dimensional indexed sets have multiple columns.
Sets with no elements are represented by empty sheets.
Parameters, variables, and equations:
Sheets have zero (for scalar items) or more columns with headers that are the index names (not necessarily sets; see below) for those dimensions.
Parameter sheets have ‘value’ and ‘unit’ columns.
Variable and equation sheets have ‘lvl’ and ‘mrg’ columns.
Items with no elements are not included in the file.
- Reading variables and equations
The ixmp API provides no way to set the data of variables and equations, because these are considered model solution data.
to_excel()will write files containing variable and equation data,
read_excel()can not add these to a Scenario, and only emits log messages indicating that they are ignored.
- Multiple dimensions indexed by the same set
read_excel()provides the init_items argument to create new sets and parameters when reading a file. However, the file format does not capture information needed to reconstruct the original data in all cases.
scenario.init_set('foo') scenario.add_set('foo', ['a', 'b', 'c']) scenario.init_par(name='bar', idx_sets=['foo']) scenario.init_par( name='baz', idx_sets=['foo', 'foo'], idx_names=['foo', 'another_dimension']) scenario.to_excel('file.xlsx')
file.xlsxwill contain sheets named ‘bar’ and ‘baz’. The sheet ‘bar’ will have column headers ‘foo’, ‘value’, and ‘unit’, which are adequate to reconstruct the parameter. However, the sheet ‘baz’ will have column headers ‘foo’ and ‘another_dimension’; this information does not allow ixmp to infer that ‘another_dimension’ is indexed by ‘foo’.
To work around this limitation, initialize ‘baz’ with the correct dimensions before reading its data:
new_scenario.init_par( name='baz', idx_sets=['foo', 'foo'], idx_names=['foo', 'another_dimension']) new_scenario.read_excel('file.xlsx', init_items=True)
- File formats other than
.xlsx(Office Open XML) file format is preferred for input and output.
ixmpuses openpyxl and
pandasin order to read and write this format. For other Excel file formats, including
.xlsb, see the Pandas documentation.