GitHub

Asset panel summary

Report the missing cells of a panel

describe gives one row for each field of an AssetPanel, with the share of its cells that the builder filled, over all cells and over the active cells. With by, it gives one row for each field and each level of a categorical field. panel_info prints the dimensions of the panel, the coverage of its two universe masks, the same shares, and the levels of each categorical field.

DataAPI.describe — Method
DataFrames.describe(pnl::AssetPanel; by::Option{<:AbstractString} = nothing)

Report the share of the cells of each Panel Field of an AssetPanel that the builder filled.

A cell is missing when the observed mask of its Panel Field is false. The builder fills that cell by the fill policy of the field, and the observed mask keeps the record, so a filled cell stays missing in this report. A Panel Field with no observed mask cannot blank, and has no missing cell.

With by = nothing the table has one row for each Panel Field. With by the name of a CategoricalPanelField, the table has one row for each other Panel Field and each level of by. A cell belongs to a level when it is active, and when by is observed there and holds that level. A tensor Panel Field counts one cell for each label.

A share is a fraction in [0, 1], not a percentage. A table that reports a percentage gives the share times 100. A share over no cell is NaN: an empty set has no share.

Algorithm

  1. Read the active cells with panel_active_cells.
  2. by = nothing: count the cells of each Panel Field with panel_field_coverage, and divide each missing count by its cell count.
  3. Otherwise read the categorical Panel Field by with panel_field. For each other Panel Field and each level, count the cells of the level with panel_level_coverage, and divide.

Arguments

  • pnl: The Asset Panel.
  • by: The name of a categorical Panel Field to group the cells by, or nothing.

Validation

  • The panel holds a Panel Field named by. Raises a KeyError.
  • The Panel Field by is a CategoricalPanelField. Raises an ArgumentError.

Returns

  • df::DataFrames.DataFrame: With by = nothing, the columns field, kind, cells, missing (the share of the cells, a fraction in [0, 1]), active_cells, active_missing (the share of the active cells, a fraction in [0, 1]) and assets_missing (the count of assets whose active cells are all missing). With by, the columns field, level, cells (the count of cells of the level) and missing (their share, a fraction in [0, 1]). A share over no cell is NaN.

Examples

julia> using DataFramesjulia> pnl = AssetPanel(;                        pf = [NumericPanelField(; name = "mcap",                                                vals = [1.0 2.0; 3.0 4.0; 5.0 6.0],                                                omsk = Bool[0 1; 1 1; 1 0]),                              CategoricalPanelField(; name = "sector", levels = ["Tech", "Energy"],                                                    codes = [1 2; 1 2; 1 2])],                        amsk = Bool[1 1; 1 1; 1 0], emsk = Bool[1 1; 1 1; 1 0]);julia> select(describe(pnl), :field, :missing, :active_missing, :assets_missing)2×4 DataFrame Row │ field   missing   active_missing  assets_missing     │ String  Float64   Float64         Int64─────┼──────────────────────────────────────────────────   1 │ mcap    0.333333             0.2               0   2 │ sector  0.0                  0.0               0julia> describe(pnl; by = "sector")2×4 DataFrame Row │ field   level   cells  missing     │ String  String  Int64  Float64─────┼─────────────────────────────────   1 │ mcap    Tech        3  0.333333   2 │ mcap    Energy      2  0.0

Related

source
PortfolioOptimisers.panel_info — Function
panel_info([io::IO,] pnl::AssetPanel; ts::Option{<:AbstractVector} = nothing) -> nothing

Print a report of an AssetPanel: its dimensions, the coverage of its two universe masks, the missing share of each Panel Field, and the levels of each categorical Panel Field.

The panel holds no timestamps. Pass the timestamps of the ReturnsResult or the PricesResult that holds the panel as ts, and the report prints the first and the last. A missing cell is a cell that the builder filled, as DataFrames.describe(pnl::AssetPanel) states. The show of the panel is the tree of its fields, as for each Result, and this report does not replace it.

Algorithm

  1. Count the cells of each Panel Field with panel_field_coverage. Print the dimensions and the missing share of the cells with panel_info_header.
  2. Print the coverage of the active mask with panel_info_mask. Print the estimation mask the same way, or print that it equals the active mask.
  3. Print the counts of each Panel Field with panel_info_fields, the numbers of DataFrames.describe(pnl::AssetPanel) with the shares as percentages.
  4. Print the levels of each categorical Panel Field with panel_info_levels.

Arguments

  • io: The stream, stdout when it is not given.
  • pnl: The Asset Panel.
  • ts: The timestamps of the observations, or nothing. A static panel does not read it.

Validation

  • length(ts) is the count of observations of a time-varying panel. Raises a DimensionMismatch.

Returns

  • nothing.

Examples

julia> pnl = AssetPanel(;                        pf = [NumericPanelField(; name = "mcap",                                                vals = [1.0 2.0; 3.0 4.0; 5.0 6.0],                                                omsk = Bool[0 1; 1 1; 1 0]),                              CategoricalPanelField(; name = "sector", levels = ["Tech", "Energy"],                                                    codes = [1 2; 1 2; 1 2])],                        amsk = Bool[1 1; 1 1; 1 0], emsk = Bool[1 1; 1 1; 1 0]);julia> panel_info(pnl; ts = ["2024-01", "2024-02", "2024-03"])Asset Panel  observations : 3 (2024-01 to 2024-03)  assets       : 2  Panel Fields : 2  cells        : 6 (observations × assets)  missing      : 16.7% of the cells, 10.0% of the active cellsactive mask  in mask          : 5 / 6 cells (83.3%)  assets per obs.  : min 1, median 2.0, max 2  assets in mask   : 2 / 2  duration (obs.)  : min 2, median 2.5, max 3estimation mask: equal to the active maskPanel Fields  field   kind         cells  missing %  active cells  active missing %  assets missing  mcap    numeric          6       33.3             5              20.0               0  sector  categorical      6        0.0             5               0.0               0categorical Panel Fields  sector: 2 levels, by the smallest count of assets over the observations    < 10    : 2 levels (Tech, Energy)    10 - 19 : 0 levels    20 - 49 : 0 levels    >= 50   : 0 levels

Related

source

Align the active mask to the fields

panel_align_active moves the start of each asset to the first active observation where each named field is observed, and returns the new panel and the count of removed cells.

PortfolioOptimisers.panel_align_active — Function
panel_align_active(pnl::AssetPanel, fields) -> NamedTuple

Remove the leading active cells of each asset of an AssetPanel until each named Panel Field holds an observed value.

An asset enters the universe of a panel when its active mask is true, and a field of the asset can start later, for example a book value that is first reported months after the listing. The function moves the start of each asset to the first active observation where each named Panel Field is observed. Only the leading cells are removed: a missing cell after that observation stays active. An asset with no such observation leaves the universe. The estimation mask is intersected with the new active mask, so it stays a subset. An observation can be left with no active asset, and the panel admits it, as it admits a view with no asset.

After the alignment, the first active cell of each asset is observed in each named Panel Field. A second call removes nothing, and the order of fields does not change the result.

A tensor Panel Field is observed at a pair of observation and asset when each of its labels is observed there. A Panel Field with no observed mask cannot blank, and is observed at each cell. The Panel Fields do not change: a removed cell keeps its value and its observed mask.

Algorithm

  1. Find each named Panel Field with panel_field, and read where it is observed with panel_observed_cells.
  2. For each asset, find the first observation that is active and where each named Panel Field is observed. Remove the active cells before it, or each active cell when there is no such observation, and count them.
  3. Intersect the estimation mask with the new active mask, and make the AssetPanel of the Panel Fields and the two new masks.

Arguments

  • pnl: The Asset Panel.
  • fields: The name of one Panel Field, or a collection of names. An empty collection removes nothing.

Validation

  • pnl is time-varying. Raises an ArgumentError.
  • The panel holds each named Panel Field. Raises a KeyError.

Returns

  • res::NamedTuple: pnl, the aligned Asset Panel; n, the count of cells removed from the active mask.

Examples

julia> pnl = AssetPanel(;                        pf = [NumericPanelField(; name = "book",                                                vals = [1.0 2.0; 3.0 4.0; 5.0 6.0; 7.0 8.0],                                                omsk = Bool[0 1; 0 0; 1 1; 1 0])],                        amsk = trues(4, 2), emsk = trues(4, 2));julia> res = panel_align_active(pnl, "book");julia> res.n2julia> res.pnl.amsk4×2 BitMatrix: 0  1 0  1 1  1 1  1

Related

source