Asset Panel
The asset panel
An asset panel holds data about each asset next to its returns or prices, such as a market capitalisation, a sector classification or a table of factor exposures. ReturnsResult and PricesResult keep it in their pnl field. Each field of the panel holds its own values and its own observed mask, which marks the cells that held data. A distance that compares assets by these fields reads a feature matrix, and feature_matrix builds that matrix from the panel each time a distance needs it. No object stores the matrix.
A feature selector names the panel fields that the matrix stacks. Each entry of a selector names one of four things: a whole panel field, a field with the levels or labels to keep, a field with one level or label, or the observed mask of a field. feature_labels names each column of the matrix with the entry that selects that column alone. Pass the labels back as a selector, and you get the same matrix.
A panel is static or time-varying. A static panel has no observation axis and no universe masks. A time-varying panel adds an observation axis, and it carries the active mask and the estimation mask.
asset_panel builds a panel from raw fields that can hold blank cells. It fills every blank by the fill policy of its field, so every field of the panel is finite. A field that can hold blanks carries an observed mask, which is false at each cell that the fill wrote.
The library has no file format for a panel. panel_dataframe converts a panel to a DataFrames.DataFrame, which you can write with any tool you already use. Name one panel field, and you get that field in the layout it has. The :long layout gives one row per (observation, asset) pair where the asset is active. The :wide layout gives one row per observation and one column per (panel field column, asset) pair, and keeps every cell. A TensorPanelField gives one column per label of its label axis, named "<field>=<label>", as in a feature matrix.
Types
PortfolioOptimisers.AbstractPanelField — Type
abstract type AbstractPanelField <: AbstractResultSupertype of the Panel Fields an AssetPanel holds.
All concrete types that carry one Panel Field's name, its values and its observed mask should subtype AbstractPanelField.
A Panel Field owns its values. Nothing else holds them, and nothing needs an index to find them. An AssetPanel is a vector of Panel Fields, and panel_field finds one by its name. A Panel Field is static or time-varying, and all Panel Fields of one panel are the same of the two.
Interfaces
In order to implement a new concrete type that works seamlessly with the library, subtype AbstractPanelField and implement the following methods:
panel_field_axes
panel_field_axes(f::AbstractPanelField) -> Tuple{Vararg{Int}}:(N,)when the Panel Field is static, and(T, N)when it is time-varying.
panel_field_labels
panel_field_labels(f::AbstractPanelField) -> Vector{String}: The column names the Panel Field contributes to a derived Feature Matrix, in column order.
panel_field_stack!
panel_field_stack!(Z::AbstractArray, f::AbstractPanelField, cols::VecInt) -> nothing: Write the Panel Field's value columns intocolsofZ.
panel_field_view
panel_field_view(f::AbstractPanelField, i, j) -> AbstractPanelField: The Panel Field over the observationsiand the assetsj.
Related
PortfolioOptimisers.AssetPanel — Type
struct AssetPanel{__T_pf, __T_amsk, __T_emsk} <: AbstractResultThe Asset Panel: the Panel Fields of one universe, and the two point-in-time universe masks.
The two universe masks are the defining content of the panel, and the Panel Fields are optional payload. A panel with fields owns their values, so nothing else on a PricesResult or a ReturnsResult holds a feature matrix. feature_matrix derives the Feature Matrix that a distance measures from the panel, and nothing stores it. A panel with no field holds pf = nothing. It is the common case of the ingestion layer when the prices hold a gap. A caller that holds only prices has no market capitalisation and no sector, and the panel states a universe and nothing else. The constructor refuses a panel with neither a field nor a mask, because it carries nothing. It also refuses an empty vector of Panel Fields, because nothing states that the panel has no field.
One panel takes one of two shapes, and its type parameters say which.
- Static: every Panel Field is
assetsorassets × labels, and both masks arenothing. A fundamentals table or a sector classification with no history is this shape. It needs at least one Panel Field, because nothing else would state its asset axis. - Time-varying: every Panel Field adds an observation axis, and both masks are
observations × assets. A numeric or a categorical Panel Field isobservations × assets, as the returns are, and a tensor Panel Field isassets × labels × observations, with the observations last as every rank-3 array holds them. A point-in-time panel is this shape.
So nothing masks mean static, or hand-built, and never gapless. A panel that the ingestion layer emits always carries both masks. When the price table holds no gap and the caller gives no Panel Field, the masks are all true and state nothing that the finiteness of the returns does not state, so the layer emits no panel at all (see attach_universe_masks).
Fields
pf: The Panel Fields, each owning its own values and its own observed mask, ornothingwhen the panel states its two masks alone.
amsk: The active mask (observations × assets): whether the asset is in the universe at that observation.nothingwhen the panel is static.
emsk: The estimation mask (observations × assets): whether the asset enters the cross-sectional estimate at that observation. Always a subset of the active mask.nothingwhen the panel is static.
Constructor
AssetPanel(; pf::Option{<:AbstractVector{<:AbstractPanelField}} = nothing, amsk::Option{<:AbstractMatrix{Bool}} = nothing, emsk::Option{<:AbstractMatrix{Bool}} = nothing)Validation
pfisnothingor a non-empty vector. Raises anIsEmptyError.- The panel carries at least one Panel Field or an active mask. See
panel_axes. - The Panel Field names are non-empty and unique. See
assert_panel_labels. - No two columns of the derived Feature Matrix share a name. See
assert_panel_feature_names. - Every Panel Field shares one
panel_field_axes. Raises aDimensionMismatch. - The masks are both
nothingwhen the Panel Fields are static, and both given when they are time-varying. Seeassert_panel_masks.
Examples
A panel is a Result, so an edit makes a new panel through this constructor, which checks the new panel again. Each edit below starts from this panel.
julia> pnl = AssetPanel(; pf = [NumericPanelField(; name = "mcap", vals = [1.0 2.0 3.0; 4.0 5.0 6.0]), CategoricalPanelField(; name = "sector", levels = ["Tech", "Energy"], codes = [1 2 1; 1 2 1])], amsk = trues(2, 3), emsk = trues(2, 3));To rename a Panel Field, make the field again with the new name, its values, its observed mask and its placeholder mask. A field made without the placeholder mask reads each placeholder as data. A categorical or a tensor Panel Field also keeps its levels, or its axis, labels and groups.
julia> f = panel_field(pnl, "mcap");julia> renamed = AssetPanel(; pf = [NumericPanelField(; name = "size", vals = f.vals, omsk = f.omsk, pmsk = f.pmsk); filter(g -> g.name != "mcap", pnl.pf)], amsk = pnl.amsk, emsk = pnl.emsk);julia> [g.name for g in renamed.pf]2-element Vector{String}: "size" "sector"To drop a Panel Field, keep the other fields.
julia> dropped = AssetPanel(; pf = filter(g -> g.name != "sector", pnl.pf), amsk = pnl.amsk, emsk = pnl.emsk);julia> [g.name for g in dropped.pf]1-element Vector{String}: "mcap"To drop observations or assets, view the panel over the complement indices with port_opt_view.
julia> kept = PortfolioOptimisers.port_opt_view(pnl, setdiff(1:2, [1]), setdiff(1:3, [2]));julia> panel_field(kept, "mcap").vals1×2 view(::Matrix{Float64}, [2], [1, 3]) with eltype Float64: 4.0 6.0To edit the masks, make the panel with the new masks. The estimation mask must stay a subset of the active mask, so intersect it with the new active mask. panel_align_active makes one edit of the active mask from the observed masks of the Panel Fields.
julia> amsk = Bool[1 1 0; 1 1 1];julia> edited = AssetPanel(; pf = pnl.pf, amsk = amsk, emsk = pnl.emsk .& amsk);julia> edited.emsk2×3 BitMatrix: 1 1 0 1 1 1Related
PortfolioOptimisers.NumericPanelField — Type
struct NumericPanelField{__T_name, __T_vals, __T_omsk, __T_pmsk} <: AbstractPanelFieldA Panel Field holding one number per asset, and per observation when it is time-varying.
A market capitalisation, a book-to-price ratio or a trailing volume is this kind. It contributes one column to a derived Feature Matrix, named after the Panel Field itself.
Fields
name: The Panel Field's name, which names its column of a derived Feature Matrix.
vals: Values:assetswhen static,observations × assetswhen time-varying.
omsk: Observed mask, the same size as the values, ornothingwhen the Panel Field cannot blank. A cell istruewhere the raw input carried a value, andfalsewhere a fill policy wrote one.
pmsk: Placeholder mask, the same size as the values, ornothingwhen no cell holds a placeholder. A cell istruewhere it holds the placeholder of a blank that no fill reached. No reader reads such a cell.
Constructor
NumericPanelField(name::AbstractString, vals::AbstractArray{<:Real}, omsk::Option{<:AbstractArray{Bool}} = nothing, pmsk::Option{<:AbstractArray{Bool}} = nothing)An integer element type of vals converts once to its floating-point type through float_if_integer, so every kernel downstream reads floating-point data. The codes of a CategoricalPanelField stay integers.
Validation
!isempty(name). Raises anIsEmptyError.!isempty(vals). Raises anIsEmptyError.ndims(vals) in (1, 2). Raises aDimensionMismatch.- Every value is finite. See
assert_panel_finite. size(omsk) == size(vals)whenomskis given. Raises aDimensionMismatch.pmskmarks only unobserved cells. Seeassert_panel_field_placeholder.
Related
PortfolioOptimisers.CategoricalPanelField — Type
struct CategoricalPanelField{__T_name, __T_levels, __T_codes, __T_omsk, __T_pmsk} <: AbstractPanelFieldA Panel Field holding one category label per asset, and per observation when it is time-varying.
A sector, an industry or a country classification is this kind. It stores integer codes over its levels, not the labels and not a one-hot block. A code is the form of a cross-sectional group label, and panel_onehot builds the one-hot block where a matrix is needed. It contributes one column per level to a derived Feature Matrix, named "<field>=<level>". Its observed mask has no level axis, so it contributes one mask column, see panel_field_observed_labels.
Fields
name: The Panel Field's name, which prefixes each of its columns of a derived Feature Matrix.
levels: The category levels, one per derived column, in column order.
codes: Integer codes overlevels:assetswhen static,observations × assetswhen time-varying.
omsk: Observed mask, the same size as the codes, ornothingwhen the Panel Field cannot blank. A cell istruewhere the raw input carried a label, andfalsewhere a fill policy wrote one.
pmsk: Placeholder mask, the same size as the codes, ornothingwhen no cell holds a placeholder. A cell istruewhere it holds the placeholder of a blank that no fill reached. No reader reads such a cell.
Constructor
CategoricalPanelField(name::AbstractString, levels::VecStr, codes::AbstractArray{<:Integer}, omsk::Option{<:AbstractArray{Bool}} = nothing, pmsk::Option{<:AbstractArray{Bool}} = nothing)Validation
!isempty(name). Raises anIsEmptyError.levelsis non-empty, holds no empty entry and holds no repeat. Seeassert_panel_labels.!isempty(codes). Raises anIsEmptyError.ndims(codes) in (1, 2). Raises aDimensionMismatch.- Every code lies in
1:length(levels). Raises aDomainError. size(omsk) == size(codes)whenomskis given. Raises aDimensionMismatch.pmskmarks only unobserved cells. Seeassert_panel_field_placeholder.
Related
PortfolioOptimisers.TensorPanelField — Type
struct TensorPanelField{__T_name, __T_axis, __T_labels, __T_groups, __T_vals, __T_omsk, __T_pmsk} <: AbstractPanelFieldA Panel Field whose label axis carries its own labels, and optionally its own groups.
A factor exposure tensor is this kind: its label axis is the factors, and its groups are the Factor Families. It contributes one column per label to a derived Feature Matrix, named "<field>=<label>".
The label axis is the second axis in both shapes. A static field is assets × labels, and a time-varying field is assets × labels × observations, with the observations on the last axis as every rank-3 array of the library holds them. So the values of one observation, f.vals[:, :, t], are one contiguous assets × labels block.
A Feature Selector entry name => LabelGroup(g) selects the labels of one group, see LabelGroup. The values of one group are a slice of the label axis, selectdim(f.vals, 2, f.groups .== g), which is f.vals[:, f.groups .== g, :] on a time-varying field and f.vals[:, f.groups .== g] on a static one. No function wraps the slice.
Fields
name: The Panel Field's name, which prefixes each of its columns of a derived Feature Matrix.
axis: Name of what the label axis represents, such as"factor".
labels: Labels of the label-axis entries, one per entry of the second axis of the values.
groups: Optional group of each label-axis entry, such as a Factor Family, one per label.
vals: Values:assets × labelswhen static,assets × labels × observationswhen time-varying.
omsk: Observed mask, the same size as the values, ornothingwhen the Panel Field cannot blank. A cell istruewhere the raw input carried a value, andfalsewhere a fill policy wrote one.
pmsk: Placeholder mask, the same size as the values, ornothingwhen no cell holds a placeholder. A cell istruewhere it holds the placeholder of a blank that no fill reached. No reader reads such a cell.
Constructor
TensorPanelField(name::AbstractString, axis::AbstractString, labels::VecStr, groups::Option{<:VecStr}, vals::AbstractArray{<:Real}, omsk::Option{<:AbstractArray{Bool}} = nothing, pmsk::Option{<:AbstractArray{Bool}} = nothing)An integer element type of vals converts once to its floating-point type through float_if_integer, so every kernel downstream reads floating-point data.
Validation
!isempty(name). Raises anIsEmptyError.!isempty(axis). Raises anIsEmptyError.labelsis non-empty, holds no empty entry and holds no repeat. Seeassert_panel_labels.length(groups) == length(labels)whengroupsis given. Raises aDimensionMismatch.!isempty(vals). Raises anIsEmptyError.ndims(vals) in (2, 3). Raises aDimensionMismatch.- Every value is finite. See
assert_panel_finite. size(vals, 2) == length(labels). Raises aDimensionMismatch.size(omsk) == size(vals)whenomskis given. Raises aDimensionMismatch.pmskmarks only unobserved cells. Seeassert_panel_field_placeholder.
Examples
julia> f = TensorPanelField(; name = "exposures", axis = "factor", labels = ["mom_12", "value", "mom_6"], groups = ["momentum", "value", "momentum"], vals = reshape(collect(1.0:12.0), 2, 3, 2));julia> f.vals[:, f.groups .== "momentum", :]2×2×2 Array{Float64, 3}:[:, :, 1] = 1.0 5.0 2.0 6.0[:, :, 2] = 7.0 11.0 8.0 12.0Related
Functions
PortfolioOptimisers.panel_field — Function
panel_field(pnl::AssetPanel, name::AbstractString) -> AbstractPanelFieldLook one Panel Field up in an AssetPanel by name.
This is the only supported route from the name of a Panel Field to its values. A consumer that parses a derived column name reads a naming convention, not the panel. The parse gives a wrong field when the name of a Panel Field holds the "=" or the "::" of that convention.
A reader that gives returns data with no panel the mask-only panel of returns_with_universe also reaches the method on a panel with no Panel Field, as the fit and the online step of a CrossSectionalFactorPrior do. So the message of that method names both causes: the returns data carries no panel, or a panel of its two universe masks alone.
Algorithm
The method that Julia selects is the algorithm.
- The panel holds no Panel Field (
pnl.pf === nothing): throw aKeyErrorthat names both causes. - Otherwise: find the first Panel Field whose name matches. Throw a
KeyErrornaming the nearest match and the whole panel when none does.
Arguments
pnl: The Asset Panel.name: The Panel Field's name.
Validation
- The panel holds a Panel Field named
name. Raises aKeyError.
Returns
f::AbstractPanelField: The Panel Field.
Related
PortfolioOptimisers.port_opt_view — Method
port_opt_view(pnl::AssetPanel, i) -> AssetPanel
port_opt_view(pnl::AssetPanel, i, j, nx::Option{<:VecStr} = nothing) -> AssetPanelReturn a view of the AssetPanel over the observations i and the assets j.
Every Panel Field owns its values, so an asset view reaches all of them. The form with one index keeps every observation, and its index i selects the assets, as in the one-index form for a ReturnsResult. The form with two indices selects the observations and the assets. A static panel has no observation axis, so it ignores the observation index.
nx holds the asset names of the input data, and it makes the square case derivable. The view cuts the label axis of a tensor Panel Field whose labels are those names by the same asset index. Nothing records the fact, and no PricesResult or ReturnsResult holds a flag for it. features_are_assets states the comparison. The form with one index passes nx = nothing, so it does not cut a label axis.
Algorithm
- View each Panel Field with
panel_field_view. Pass aColonfor the observation index of a static panel, and passnxfor the square case. A panel with no Panel Field keepspf = nothing. - View both masks with
panel_mask_view, which keeps themnothingwhen the panel is static.
Arguments
pnl: The Asset Panel.i: Observation index in the form with two indices, and asset index in the form with one index.j: Asset index.nx: The asset names of the input data, ornothing.
Returns
new_pnl::AssetPanel: An Asset Panel over the selected observations and assets.
Related
PortfolioOptimisers.panel_field_axes — Function
panel_field_axes(f::NumericPanelField) -> Tuple{Vararg{Int}}
panel_field_axes(f::CategoricalPanelField) -> Tuple{Vararg{Int}}
panel_field_axes(f::TensorPanelField) -> Tuple{Vararg{Int}}Return a Panel Field's observation and asset axes, without its label axis.
Every Panel Field of one AssetPanel shares this shape, and the shape tells whether the panel is static. A static Panel Field returns (N,), and a time-varying one returns (T, N), the shape of the universe masks.
Algorithm
The method that Julia selects is the algorithm. A numeric and a categorical Panel Field carry no label axis, so their whole size is returned. A tensor Panel Field is assets × labels or assets × labels × observations, so it drops its label axis and returns the observation axis first.
Arguments
f: The Panel Field.
Returns
ax::Tuple{Vararg{Int}}:(N,)when static,(T, N)when time-varying.
Related
PortfolioOptimisers.panel_field_labels — Function
panel_field_labels(f::NumericPanelField) -> Vector{String}
panel_field_labels(f::CategoricalPanelField) -> Vector{String}
panel_field_labels(f::TensorPanelField) -> Vector{String}Return the column names one Panel Field contributes to a derived Feature Matrix, in column order.
Algorithm
The method that Julia selects is the algorithm. Each kind names its columns differently.
NumericPanelField: one column, named after the Panel Field itself.CategoricalPanelField: one column per level, named"<name>=<level>".TensorPanelField: one column per label, named"<name>=<label>".
Arguments
f: The Panel Field.
Returns
labels::Vector{String}: One name per derived column.
Related
PortfolioOptimisers.panel_field_view — Function
panel_field_view(f::NumericPanelField, i, j, nx) -> NumericPanelField
panel_field_view(f::CategoricalPanelField, i, j, nx) -> CategoricalPanelField
panel_field_view(f::TensorPanelField, i, j, nx) -> TensorPanelFieldReturn a view of one Panel Field over the observations i and the assets j.
A static Panel Field has no observation axis, so its caller passes a Colon for i.
The square case is derived here, by name. A tensor Panel Field whose labels are the asset names of the input data (features_are_assets) is sliced on its label axis by the same asset index, together with its labels and its groups: a group belongs to one label, so the labels that survive keep theirs. Every other field's label axis addresses features, and an asset view does not reach it.
Algorithm
The method that Julia selects is the algorithm, and each kind views its own value array and its own mask: a numeric and a categorical Panel Field through panel_array_view, and a tensor Panel Field through panel_tensor_view, which holds the labels on its second axis.
Arguments
f: The Panel Field.i: Observation index.j: Asset index.nx: The asset names of the input data, ornothing. Read for the square case alone.
Returns
- A Panel Field of the same kind over the selected observations and assets.
Related