GitHub

Feature distance active rows: private API

PortfolioOptimisers.active_window — Function
active_window(amsk, Z::Arr3Num, dims::Integer)

Check the active mask of a window against the window, and give it only when a cell of it is inactive.

The active mask marks, for each observation of the window and each asset, whether the asset is in the universe. A window with no inactive cell reads as nothing, so the collapse takes the path that reads every row, and its result is the one it gives with no mask.

Algorithm

  1. amsk is nothing: return nothing.
  2. Otherwise check that amsk is observations × assets for the layout that dims declares, and return nothing when every entry is true, or amsk otherwise.

Arguments

  • amsk: The active mask of the window, observations × assets, or nothing when every cell is active.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Validation

  • size(amsk) == (size(Z, 1), assets). Raises a DimensionMismatch.

Returns

  • A::Option{<:AbstractMatrix{Bool}}: The active mask, or nothing when every cell is active.

Related

source
PortfolioOptimisers.blank_unreadable — Function
blank_unreadable(Z::Arr3Num, A::AbstractMatrix{Bool}, dims::Integer)

Give a copy of a window of time-varying features with a zero at each cell that the mask A marks unreadable.

A collapse reads each asset, or each pair of assets, only at its own readable rows, so the value of an unreadable cell must reach neither the distance nor a refusal. distance checks this copy with assert_feature_matrix and collapses it. A NaN, or a value outside the domain of the metric, at an unreadable cell then refuses nothing, and no pass of a collapse reads it: StackObservations measures the whole stack before it measures a pair again on its shared rows, and AggregateDistances measures every asset at each row before it keeps the pairs that both are active at. The zero is the value that feature_window writes at a placeholder, and that the panel collapse of a meta-optimiser writes at an inactive cell. The copy keeps the element type of Z.

Arguments

  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • A: The mask of readable cells of the window, observations × assets.
  • dims: Dimension along which to perform the computation.

Returns

  • Zr::Array: The window, the same size and element type as Z, with a zero at each unreadable cell.

Related

source
PortfolioOptimisers.assert_feature_readable — Function
assert_feature_readable(alg, A, nx)

Refuse a window in which the collapse has no value to read for an asset, and name each such asset.

Inside a fit, feature_readable_mask drops such an asset at the entry of the optimiser, so the collapse never meets it. A direct call of distance, cor_and_dist or clusterise has no entry of a fit, so it refuses here.

Arguments

  • alg: The collapse algorithm.
  • A: The active mask of the window, observations × assets, or nothing when every cell is active.
  • nx: The asset names, or nothing.

Validation

  • Every asset is readable by feature_readable. Raises an ArgumentError that names each other asset and the remedy.

Returns

  • nothing.

Related

source
PortfolioOptimisers.active_collapse — Function
active_collapse(alg::AbstractCollapseAlgorithm, Zd::Arr3Num, w, A, nx)

Collapse each asset's features over its own active rows, with the weights of those rows.

The weights are resolved once against the whole window. Each asset keeps the entries of its active rows, and the collapse divides by their sum, so an inactive cell has no weight.

\[\begin{align} \bar{z}_{i,\,k} &= \dfrac{\sum\limits_{t=1}^{T} w_{t} a_{t,\,i} z_{t,\,i,\,k}}{\sum\limits_{t=1}^{T} w_{t} a_{t,\,i}}\,, \end{align}\]

Where:

  • $\bar{z}_{i,\,k}$: Collapsed feature $k$ of asset $i$, the aggregate of $z_{t,\,i,\,k}$ over the observation axis.
  • $z_{t,\,i,\,k}$: Feature window entry: feature $k$ of asset $i$ at observation $t$.
  • $a_{t,\,i}$: Entry of the active mask, 1 when asset $i$ is active at observation $t$ and 0 otherwise.
  • $w_{t}$: Observation weight of observation $t$.
  • $T$: Number of observations.

MedianCollapse takes the same restricted weights.

Algorithm

  1. Find the active rows of each asset.
  2. Check that the weights of each asset's active rows sum to more than zero.
  3. Collapse each asset over its active rows with collapse_features, under the weights of those rows, and stack the results by asset.

Arguments

  • alg: The collapse algorithm.
  • Zd: The window, observations × assets × features.
  • w: The resolved observation weights of the window, or nothing.
  • A: The active mask of the window, observations × assets.
  • nx: The asset names, or nothing.

Validation

  • The weights of the active rows of each asset sum to more than zero. Raises an ArgumentError that names each other asset.

Returns

  • Zc::Matrix{<:Number}: The collapsed features, assets × features.

Related

source
PortfolioOptimisers.stack_rescale — Function
stack_rescale(metric::Distances.SemiMetric, d::Number, idx::AbstractVector{<:Integer}, len::Integer)

Rescale the distance d of a pair, measured on the coordinates idx of a stack, to all len coordinates of the stack.

StackObservations measures a pair on the rows at which both assets are active. A metric that sums over the coordinates then sums over fewer of them, and gives a smaller distance than the whole window would. The rescale is the one that makes the sum an estimate of the sum over the whole window.

MetricRescale
Distances.Cityblock, Distances.TotalVariation, Distances.SqEuclidean, Distances.ChiSqDist$d \cdot L / n$
Distances.Euclidean$d \cdot \sqrt{L / n}$
Distances.Minkowski$d \cdot (L / n)^{1/p}$
A weighted formthe ratio of the weight sums in place of $L / n$
AngularDist, Distances.CosineDist, Distances.CorrDist, Distances.Jaccard, Distances.BrayCurtis, and the three mean deviationsnone

Where $L$ is len and $n$ is the length of idx. Every coordinate of a row carries the features of one observation, so $L / n$ is $T / n$ in rows. Another metric has no method, and it refuses. The message names the method to add.

Arguments

  • metric: The distance metric.
  • d: The distance of the pair on the coordinates idx.
  • idx: The coordinates of the stack that the pair shares.
  • len: The number of coordinates of the stack.

Returns

  • d: The rescaled distance.

Related

source
PortfolioOptimisers.active_feature_distance — Function
active_feature_distance(de::FeatureDistance, win::NamedTuple)

Turn a window of time-varying features that holds an inactive cell into a distance matrix, reading each asset or each pair at its own active rows.

win holds the window Z, its layout dims, its active mask A and the asset names nx. Z holds a zero at each cell that A marks false, from blank_unreadable, so no step of a collapse reads the stored value of such a cell. distance calls this function only when a cell of the window is inactive, and only after assert_feature_readable checked that the collapse can read each asset. A window with no inactive cell takes the path of feature_distance, so its result does not change.

Algorithm

The type of de.alg selects one of four methods.

  1. LastObservation: under LastRow, or when every asset is active at the last row, measure the last row. Otherwise measure each asset at its last active row.
  2. AggregateFeatures: resolve the weights with collapse_weights, collapse each asset over its own active rows with active_collapse, and measure the collapsed matrix.
  3. AggregateDistances: measure each row, and add each entry of a pair only at a row where both assets are active, with that row's weight. Check that the weight total of each asset, on the diagonal, is more than zero. Divide each entry by its own weight total, and give each pair with a zero total to resolve_empty_pairs!.
  4. StackObservations: measure the whole stack. Measure each pair that shares fewer rows than the window again, on the coordinates of its shared rows, with stack_metric, and rescale it with stack_rescale. Give each pair with no shared row to resolve_empty_pairs!.

Arguments

  • de: Feature distance estimator.
  • win: The window, a NamedTuple of Z, dims, A and nx.

Returns

  • D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source
PortfolioOptimisers.shared_stack_distance — Function
shared_stack_distance(metric::Distances.SemiMetric, Zs::MatNum, (i, j), rows::AbstractVector, T::Integer)

Measure the pair (i, j) of a stack Zs, assets × (observations · features), on the coordinates of its shared active rows, and rescale the distance to the whole window of T rows.

Algorithm

  1. Name the coordinates of the shared rows, t + (k - 1) T for each shared row t and each feature k.
  2. Measure the two assets on those coordinates with pair_distance, under the metric restricted by stack_metric.
  3. Rescale the distance to every coordinate of the stack with stack_rescale.

Related

source
PortfolioOptimisers.feature_readable_mask — Function
feature_readable_mask(x, imsk::Option{<:BitVector}, rd) -> Option{BitVector}

Compose the Investable Mask of a prior with the assets that each FeatureDistance inside x can read, at the entry of a fit.

A fit reduces its prior, its optimiser and its returns data to the Investable Mask once, at its entry, with investable_reduction. An asset that the prior admits can still have no value that a FeatureDistance of the fit can read in the window of its Asset Panel. A cell is readable where the asset is active and every value column of the Feature Selector holds data, observed or filled. LastRow reads nothing for an asset whose last cell is not readable, and every collapse reads nothing for an asset with no readable row. A static panel has one row, so an asset with a placeholder has none. This function drops such an asset from the mask before the reduction. The asset then departs as a non-investable asset: the reduction announces it, the fit gives it a zero weight, and it is listed on the Non-Investable Axis. Under DropFewerRows it also drops, from each pair with no shared active row, the asset with fewer active rows, until no pair is empty.

A default prior drops every asset that is not active over its whole window, so under it this function drops only an asset with a placeholder.

Every fit that clusters or builds a phylogeny calls it: HierarchicalRiskParity, HierarchicalEqualRiskContribution, SchurComplementHierarchicalRiskParity, NestedClustered, and a JuMPOptimiser through its phylogeny and centrality estimators. Stacking and SubsetResampling cluster only inside their inner optimisers, and each of those calls it at its own entry.

Algorithm

The method that Julia selects is the algorithm.

  1. A Tuple or a vector: compose the mask of each entry in turn.
  2. An estimator that holds a distance estimator, such as a ClustersEstimator: forward to the estimator it holds.
  3. A FeatureDistance with no producer, on returns data that holds a panel: take the mask of readable cells in the rows that the collapse reads, with entry_activity. Mark the readable assets with feature_readable, keep those that imsk keeps, and hand the result to drop_empty_pairs!. Return nothing when imsk is nothing and every asset stays.
  4. Every other case, a producer included, a panel whose every cell is readable, and no panel: return imsk unchanged.

Arguments

  • x: An estimator, a vector or a Tuple of estimators.
  • imsk: The Investable Mask of the prior, or nothing when every asset is investable.
  • rd: The returns result to use.

Validation

Returns

  • imsk::Option{BitVector}: The composed mask, or nothing when every asset stays.

Related

source
PortfolioOptimisers.feature_window_mask — Function
feature_window_mask(alg, pnl::AssetPanel, cols::AbstractVector{Tuple{Int, Int, Symbol}}) -> (rows, A)

Name the rows that a collapse reads, and cut the mask of the cells it can read to them.

A cell is readable where the asset is active and every value column of cols holds data: readable_cells joins the active mask of the panel with feature_data_cells. A cell that holds the placeholder of a blank that no fill reached holds no data. So a collapse reads it as it reads an inactive cell: each asset and each pair at its own readable rows. readable_window_rows names the rows on this mask, so LastActiveRow reads the last readable row of each asset. window_activity cuts the mask to the rows.

Arguments

  • alg: The collapse algorithm.
  • pnl: The Asset Panel.
  • cols: The resolved columns, from select_fields.

Returns

Related

source
PortfolioOptimisers.last_active_rows — Function
last_active_rows(A)

Name the rows from the earliest last true row of an asset in the mask A, observations × assets, to the last row: r0:nobs, where r0 is nobs when no asset has a true row. These rows hold the last true row of every asset that has one. nothing names every row, Colon().

Related

source
PortfolioOptimisers.static_window — Function
static_window(amsk, Z::MatNum, dims::Integer)

Check the one-row mask of a static Feature Matrix against the matrix. A static matrix has one row of cells, so its mask is 1 × assets, and nothing marks every asset readable.

Validation

  • size(amsk) == (1, assets), where dims names the asset axis of Z. Raises a DimensionMismatch.

Related

source