Panel data frame: private API

PortfolioOptimisers.panel_frame_columns — Function
panel_frame_columns(f::NumericPanelField, decode::Bool)
panel_frame_columns(f::CategoricalPanelField, decode::Bool)
panel_frame_columns(f::TensorPanelField, decode::Bool)

Return the table columns one Panel Field contributes to a panel_dataframe, in column order.

A column is the tuple (name, vals, omsk, pmsk). name is the column name, vals is the slab of values over the axes of the panel, omsk is the slab of the observed mask, or nothing when the Panel Field carries no observed mask, and pmsk is the slab of the placeholder mask, or nothing when the Panel Field holds no placeholder. Every slab is assets on a static panel and observations × assets on a time-varying one, so panel_array_view slices each of them by asset.

A table and a Feature Matrix give a categorical Panel Field a different number of columns. A Feature Matrix holds numbers, so it gives the Panel Field one indicator column per level. A table column can hold a string, so it gives the Panel Field one column that holds the level. A tensor Panel Field gets the same columns in both, one column per label-axis label under the "<field>=<label>" name that panel_field_labels gives it. A table has two axes, and the labels are a third axis, so each label becomes a column.

Algorithm

The method that Julia selects is the algorithm.

  1. NumericPanelField: one column with the name of the Panel Field, which holds its values.
  2. CategoricalPanelField: one column with the name of the Panel Field. It holds the level that each code names when decode is true, and the codes when decode is false.
  3. TensorPanelField: one column per label-axis label, named "<field>=<label>", which holds the slice of that label, transposed to the axes of the panel with panel_column_transpose.

Arguments

  • f: The Panel Field.
  • decode::Bool: Whether a categorical Panel Field gives its levels in place of its codes. A numeric and a tensor Panel Field carry no codes, so these methods do not read it.

Returns

  • cols::Vector{<:Tuple}: One (name, vals, omsk, pmsk) tuple per column, in column order. The vector takes the type of its tuples. The value slab of a numeric or a tensor Panel Field is the array of the Panel Field or a view of it, not a copy.

Related

source
PortfolioOptimisers.panel_frame_fields — Function
panel_frame_fields(pnl::AssetPanel, fields::Nothing)
panel_frame_fields(pnl::AssetPanel, fields)

Find the Panel Fields that a panel_dataframe call names, in the order of the call.

Algorithm

The method that Julia selects is the algorithm.

  1. fields is nothing: return the Panel Fields of the panel, in panel order.
  2. Otherwise: panel_field finds one Panel Field per entry of fields. Return them in the order of fields.

Arguments

  • pnl: The Asset Panel.
  • fields: The names of the Panel Fields to include, or nothing for all Panel Fields. An entry can be a String or a Symbol.

Validation

  • The panel holds a Panel Field with each name. Raises a KeyError. See panel_field.

Returns

  • fs: The Panel Fields, in column order. With fields = nothing this is each_panel_field of the panel: pnl.pf itself, not a copy, or () for a panel with no Panel Field.

Related

source
PortfolioOptimisers.panel_frame_assets — Function
panel_frame_assets(nx::VecStr, assets::Nothing) -> Vector{Int}
panel_frame_assets(nx::VecStr, assets::AbstractString) -> Vector{Int}
panel_frame_assets(nx::VecStr, assets) -> Vector{Int}

Convert the asset labels that a panel_dataframe call names into positions on the asset axis of the panel.

The panel holds no asset names. Its Panel Fields share an asset axis, and the PricesResult or the ReturnsResult that holds the panel names that axis. So the caller gives the names in nx, and this function finds the position of each label in nx. The functions that write the table slice every slab at these positions.

Algorithm

The method that Julia selects is the algorithm.

  1. assets is nothing: return every position, in axis order.
  2. assets is one string: return the position of that label, as a one-entry vector. The method finds and refuses the label as step 3 does for each entry, and the two copies of the rule agree.
  3. Otherwise: find the first position of each label in nx, in the order of assets, and check that no position repeats.

Arguments

  • nx: The asset names of the universe of the panel. See VecStr.
  • assets: The asset labels to include, or nothing for all assets. An entry can be a String or a Symbol.

Validation

  • nx holds every entry of assets. Raises a KeyError.
  • No entry of assets repeats. Raises an ArgumentError.

Returns

  • j::Vector{Int}: The selected positions on the asset axis, in column order.

Related

source
PortfolioOptimisers.panel_frame_column! — Function
panel_frame_column!(df::DataFrames.DataFrame, name::AbstractString, v::AbstractVector) -> nothing

Add one column to a table that panel_dataframe writes, and refuse a column name that the table already holds.

A Panel Field name, a label-axis label and an asset name are free text, so two columns can get the same name. For example, a Panel Field named "asset" gets the name of the asset key of the long layout, and a numeric Panel Field named "beta=size" gets the name of the "size" column of a tensor Panel Field "beta". df[!, name] = v replaces a column with the same name and raises no error, so the table loses a column. This function raises an error in its place.

The table gets a copy of v. A slab can be the array of a Panel Field or a view of it, so without the copy a change to a cell of the table changes the panel.

Algorithm

  1. Check that df holds no column named name.
  2. Add a copy of v to df as its last column, with the name name.

Arguments

  • df: The table that the caller writes. This function adds the column to it.
  • name: The column name.
  • v: The column values.

Validation

  • df holds no column named name. Raises an ArgumentError.

Returns

  • nothing. df holds the new column.

Related

source
PortfolioOptimisers.panel_frame_block! — Function
panel_frame_block!(df::DataFrames.DataFrame, A::AbstractVector, names::VecStr) -> nothing
panel_frame_block!(df::DataFrames.DataFrame, A::AbstractMatrix, names::VecStr) -> nothing

Write one slab of a Panel Field, or one universe mask, into a table as one column per asset.

The single-field shape and the wide layout write their columns through this function.

Algorithm

The method that Julia selects is the algorithm. A slab is a vector on a static panel and a matrix on a time-varying one.

  1. A vector slab has no observation axis. The value of each asset becomes a column with one row.
  2. A matrix slab has an observation axis. The column of each asset in the slab becomes a column of the table.

Each column goes through panel_frame_column!.

Arguments

  • df: The table that the caller writes. This function adds the columns to it.
  • A: The slab, sliced to the selected assets.
  • names: The column name of each selected asset, in the column order of the slab. See VecStr.

Validation

Returns

  • nothing. df holds the new columns.

Related

source
PortfolioOptimisers.panel_frame_mask! — Function
panel_frame_mask!(df::DataFrames.DataFrame, m::Nothing, name::AbstractString, j::VecInt, args...) -> nothing
panel_frame_mask!(df::DataFrames.DataFrame, m::AbstractArray{Bool}, name::AbstractString, j::VecInt) -> nothing
panel_frame_mask!(df::DataFrames.DataFrame, m::AbstractArray{Bool}, name::AbstractString, j::VecInt, nxj::VecStr, cols::Vector{String}) -> nothing

Add the slab of one mask of a Panel Field column to a panel_dataframe table, the observed mask or the placeholder mask. A Panel Field that carries no such mask adds nothing.

Algorithm

The method that Julia selects is the algorithm.

  1. m is nothing: add nothing.
  2. The long layout: add one column name, through panel_frame_column!, as panel_frame_long adds a value column.
  3. The wide layout: add one column "<name>@<asset>" per selected asset, through panel_frame_block!, as panel_frame_wide adds a value column.

Arguments

  • df: The table. This function adds the columns to it.
  • m: The slab of the mask, or nothing.
  • name: The column name, "<column>::observed" or "<column>::placeholder".
  • j: The selected positions on the asset axis. See VecInt.
  • nxj: The name of each selected asset, in the order of j. The wide layout alone reads it. See VecStr.
  • cols: A buffer of one column name per asset, which the wide layout writes.

Returns

  • nothing. df holds the new columns.

Related

source
PortfolioOptimisers.panel_frame_field — Function
panel_frame_field(pnl::AssetPanel, f::AbstractPanelField, j::VecInt, nxj::VecStr, ts, decode::Bool) -> DataFrames.DataFrame
panel_frame_field(pnl::AssetPanel, f::TensorPanelField, j::VecInt, nxj::VecStr, ts, decode::Bool)

Write one Panel Field as a table with one row per observation and one column per asset.

A panel_dataframe call that names one Panel Field returns this table. The values of a numeric or a categorical Panel Field are observations × assets, which is the shape of the table, so the table needs no asset key and no layout. It holds no universe mask and no observed mask. A static panel has no observation axis, so its table has one row.

This function refuses a TensorPanelField. Its values are assets × labels × observations, and a table with one column per asset has no place for the labels. The long and the wide layouts hold it, with one column per label.

Algorithm

The method that Julia selects refuses a tensor Panel Field. The other method runs these steps.

  1. Add the "observation" column with the labels ts, unless the panel is static.
  2. Add one column per selected asset with the values of f, through panel_frame_block!.

Arguments

  • pnl: The Asset Panel.
  • f: The Panel Field.
  • j: The selected positions on the asset axis. See VecInt.
  • nxj: The name of each selected asset, in the order of j. See VecStr.
  • ts: The observation labels. A static panel does not read it.
  • decode::Bool: Whether a categorical Panel Field gives its levels in place of its codes.

Validation

Returns

  • df::DataFrames.DataFrame: The table, with the "observation" column first when the panel is time-varying. The table holds copies, so a change to the table does not change the panel.

Related

source
PortfolioOptimisers.panel_frame_long — Function
panel_frame_long(pnl::AssetPanel, fs, j::VecInt, nxj::VecStr, ts, decode::Bool) -> DataFrames.DataFrame

Write the selected Panel Fields in the long layout, with one row per pair of observation and asset and one column per Panel Field column.

On a time-varying panel the table keeps only the rows where the active mask is true, so every row is an asset in the universe at that observation. The estimation mask is the "emsk" column, because it can be false on a row that the active mask keeps. A static panel has no mask, so its table has no "emsk" column and has one row per asset.

Algorithm

  1. Add the key columns. A time-varying panel gets "observation" and "asset", where row (t - 1) n + k holds observation t and asset k of the n selected assets. A static panel gets "asset" alone.
  2. For each Panel Field in order, and each of its columns from panel_frame_columns, add the slab in row-major order, so that each value is on the row of its observation and asset. When the Panel Field carries an observed mask, add the mask of that column next, as "<column>::observed", and when it carries a placeholder mask, add that mask after it, as "<column>::placeholder".
  3. On a time-varying panel, add the "emsk" column, and keep the rows where the active mask is true.

Each column goes through panel_frame_column!.

Arguments

  • pnl: The Asset Panel.
  • fs: The Panel Fields, in column order.
  • j: The selected positions on the asset axis. See VecInt.
  • nxj: The name of each selected asset, in the order of j. See VecStr.
  • ts: The observation labels. A static panel does not read it.
  • decode::Bool: Whether a categorical Panel Field gives its levels in place of its codes.

Validation

  • No two columns get the same name. A Panel Field named "observation", "asset" or "emsk" gets the name of a key or mask column. Raises an ArgumentError. See panel_frame_column!.

Returns

  • df::DataFrames.DataFrame: The long table. It holds copies, so a change to the table does not change the panel.

Related

source
PortfolioOptimisers.panel_frame_wide — Function
panel_frame_wide(pnl::AssetPanel, fs, j::VecInt, nxj::VecStr, ts, decode::Bool) -> DataFrames.DataFrame

Write the selected Panel Fields in the wide layout, with one row per observation and one column per pair of Panel Field column and asset.

A column name is the Panel Field column, an "@", and the asset, for example "mcap@AAPL", "beta=size@AAPL" and "mcap::observed@AAPL". The table keeps every cell of the selected assets, and on a time-varying panel it adds both universe masks as the "amsk@<asset>" and "emsk@<asset>" columns. So the wide layout drops no data, and the long layout drops the rows outside the universe.

To get the Panel Field column and the asset of a cell, read the long layout, which holds them in two key columns. Do not split a wide column name at "@", because a Panel Field name or an asset name can contain "@".

Algorithm

  1. Add the "observation" column, unless the panel is static. A static panel has one row.
  2. For each Panel Field in order, and each of its columns from panel_frame_columns, add one column per selected asset through panel_frame_block!. When the Panel Field carries an observed mask, add the mask of that column next, one column per asset, and then its placeholder mask in the same way when it holds a placeholder.
  3. On a time-varying panel, add the active mask and then the estimation mask, one column per asset each.

Arguments

  • pnl: The Asset Panel.
  • fs: The Panel Fields, in column order.
  • j: The selected positions on the asset axis. See VecInt.
  • nxj: The name of each selected asset, in the order of j. See VecStr.
  • ts: The observation labels. A static panel does not read it.
  • decode::Bool: Whether a categorical Panel Field gives its levels in place of its codes.

Validation

  • No two columns get the same name. A Panel Field named "amsk" or "emsk" gets the names of a mask column. Raises an ArgumentError. See panel_frame_column!.

Returns

  • df::DataFrames.DataFrame: The wide table. It holds copies, so a change to the table does not change the panel.

Related

source