GitHub

Feature Distance: private API

PortfolioOptimisers.AbstractFeatureCollapseAlgorithm — Type
abstract type AbstractFeatureCollapseAlgorithm <: AbstractAlgorithm

Abstract supertype for all feature collapse algorithms.

A feature collapse algorithm reduces a window of time-varying features, observations × assets × features, to a single assets × assets distance matrix. It is the FeatureDistance alg field, and is inert when the feature matrix is 2-D — a static feature matrix has no observation axis to collapse. At observations == 1 every algorithm in the family agrees exactly.

Related

source
PortfolioOptimisers.assert_metric_domain — Function
assert_metric_domain(metric::Distances.SemiMetric, Z::ArrNum, sym::Symbol = :Z)

Assert that Z lies in metric's domain. The fallback is a no-op: most metrics accept any finite real input, and a blanket non-negativity check would reject signed factor loadings and the FeatureDistance default metric alike.

Distances.Jaccard (the Ruzicka form), Distances.BrayCurtis and Distances.ChiSqDist are the exceptions, all defined only on non-negative reals. The check matters most for Distances.Jaccard, which fails silently: it returns values up to 2 on signed input, with no error, straight into a clustering routine.

Algorithm

  1. Select the method by the type of metric. The three metrics above own one method between them; every other metric reaches the Distances.SemiMetric method, which is a no-op and returns immediately.
  2. On that method, check Z for non-negativity with assert_nonneg, which raises a DomainError naming sym when an entry is negative.

Arguments

  • metric: Distance metric whose domain Z must lie in.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • sym::Symbol = :Z: Name that the error message gives to Z.

Validation

  • Under Distances.Jaccard, Distances.BrayCurtis and Distances.ChiSqDist: all(x -> x >= 0, Z).

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_feature_matrix — Function
assert_feature_matrix(de::FeatureDistance, Z::ArrNum, dims::Integer)

Validate a feature matrix at the distance/cor_and_dist entry point: dims selects a valid axis, Z is non-empty, every entry is finite, and Z lies in the metric's domain.

Non-finite entries are rejected because no metric produces a usable distance from them — the Minkowski family gives Inf and the cosine family gives NaN — and neither can be clustered. Structurally degenerate inputs that a metric can handle are admitted: zero feature vectors are given a documented convention (see AngularDist), and duplicate or constant features are legitimate.

Algorithm

  1. Check dims with assert_dims.
  2. Check that Z is non-empty with assert_nonempty.
  3. Check that every entry of Z is finite with assert_all_finite.
  4. Check that Z lies in de.metric's domain with assert_metric_domain.

Arguments

  • de: Feature distance estimator, read for its metric.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Validation

  • dims in (1, 2).
  • !isempty(Z).
  • all(isfinite, Z).
  • Z lies in de.metric's domain (see assert_metric_domain).

Returns

  • nothing.

Related

source
PortfolioOptimisers.zero_feature_vectors — Function
zero_feature_vectors(Z::MatNum, dims::Integer)

Boolean mask of the assets whose feature vector is entirely zero, in the layout declared by dims.

Algorithm

  1. Select the asset axis from dims: the rows of Z at dims = 1, and its columns at dims = 2.
  2. Test each asset's feature vector with all(iszero, ...), giving one entry of the mask per asset.

Arguments

  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Returns

  • z::Vector{Bool}: Mask, one entry per asset, true where that asset's feature vector is entirely zero.

Related

source
PortfolioOptimisers.patch_zero_feature_vectors! — Function
patch_zero_feature_vectors!(D::MatNum, Z::MatNum, dims::Integer)

Apply the zero-feature-vector convention to D in place: two zero vectors are at distance 0, a zero vector and a non-zero one at distance 1.

Only entries the metric left as NaN are rewritten. A zero feature vector is structurally valid input, so construction-time validation cannot catch it, but it is undefined for the metrics normalised by a norm — the cosine family gives NaN against anything, and Distances.Jaccard/Distances.BrayCurtis give NaN between two zero vectors. It is perfectly well defined for the Minkowski family, which places it at the origin; restricting the patch to NaN entries fixes the former without corrupting the latter.

The convention is the one that keeps $S = \cos(\pi D)$ true on every entry, so AngularSimilarity yields +1 between two zero vectors and -1 against a non-zero one, with a unit diagonal. Distances.pairwise always writes an exact zero diagonal, so self-distance needs no patching.

Algorithm

  1. Build the zero mask z of Z with zero_feature_vectors.
  2. Return D unchanged when no asset is masked, which is the common case.
  3. Otherwise visit every off-diagonal entry of D and rewrite it only when the metric left it as NaN and at least one of its two assets is masked: to zero(T) when both are masked, and to one(T) when exactly one is.

Arguments

  • D: Distance matrix assets × assets, rewritten in place.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Returns

  • D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source
PortfolioOptimisers.feature_distance — Function
feature_distance(metric::Distances.SemiMetric, Z::MatNum, dims::Integer)

Turn a 2-D feature matrix into a distance matrix. This is the shared kernel behind every FeatureDistance entry point: the collapse algorithms differ only in the matrix they hand it, except for AggregateDistances, which calls it once per observation and aggregates the results.

Algorithm

  1. Apply metric to every pair of assets of Z with Distances.pairwise, along the axis dims names, giving D.
  2. Apply the zero-feature-vector convention to D in place with patch_zero_feature_vectors!.

Arguments

  • metric: Distance metric applied to the assets of Z.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Returns

  • D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source
feature_distance(de::FeatureDistance, Z::Arr3Num, dims::Integer)

Turn a window of time-varying features into a distance matrix, by the collapse algorithm in de.alg.

Algorithm

The type of de.alg selects one of four methods. Each is stated on its own type, and the branch is:

  1. LastObservation: hand the last slice of the observation axis to the 2-D kernel.
  2. StackObservations: hand the stacked matrix from stack_observations to the 2-D kernel, along its first axis.
  3. AggregateFeatures: resolve the weights with collapse_weights, collapse the window with collapse_features, and hand the collapsed matrix to the 2-D kernel.
  4. AggregateDistances: resolve the weights with collapse_weights, then accumulate one weighted distance matrix per observation and divide by the weight total.

Arguments

  • de: Feature distance estimator, read for its metric and its alg.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Returns

  • D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source
PortfolioOptimisers.collapse_features — Function
collapse_features(alg::AbstractCollapseAlgorithm, Z::Arr3Num, w::Option{<:VecNum})

Aggregate a window of time-varying features along its leading observation axis, returning a matrix with the two trailing axes of Z unchanged. Used by AggregateFeatures.

The element type of the result follows the aggregate, not the window: the mean and the median of a window of integers are both fractional, and a window with an even observation count has a fractional median even when every value in it is whole. Both routes therefore build their result from the values they compute.

Algorithm

The MeanCollapse route:

  1. Reduce the leading axis of Z with Statistics.mean, weighted by w when w is not nothing.
  2. Drop the reduced axis.

The MedianCollapse route:

  1. For each asset j and each feature k, take the observation series view(Z, :, j, k).
  2. Reduce that series with Statistics.median, weighted by w when w is not nothing, giving one entry of the result.

Arguments

  • alg: Collapse algorithm, the aggregator applied along the observation axis.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • w: Resolved observation weights, one entry per observation, or nothing for an unweighted collapse.

Returns

  • Zc::Matrix{<:Number}: Collapsed feature matrix, the two trailing axes of Z unchanged.

Related

source
PortfolioOptimisers.stack_observations — Function
stack_observations(Z::Arr3Num, dims::Integer)

Reshape a window of time-varying features into an assets × (observations · features) matrix, whose rows are the assets whichever trailing axis dims says they occupy.

Algorithm

  1. Permute Z so the asset axis leads: (2, 1, 3) at dims = 1, and (3, 1, 2) at dims = 2. Both leave the observation axis second and the feature axis third.
  2. Reshape the permuted array to assets × (observations · features), giving one long feature vector per asset.

Arguments

  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
  • dims: Dimension along which to perform the computation.

Returns

  • Za::Matrix{<:Number}: Stacked feature matrix, assets × (observations · features).

Related

source
PortfolioOptimisers.collapse_weights — Function
collapse_weights(w::Option{<:ObsWeights}, Z::Arr3Num)

Resolve the observation weights of a collapse algorithm against a window of time-varying features.

Z is matricised to observations × (assets · features) first, because get_observation_weights's documented interface is VecNum/MatNum and a raw 3-D array matches neither — a user's correct MatNum method would otherwise never fire. There is no caller-side nothing guard: get_observation_weights raises ObservationWeightsError itself when a DynamicAbstractWeights cannot resolve, so nothing here means only that no weights were requested.

Cross-fold weighting requires a DynamicAbstractWeights. It resolves against the Z it is handed, so it is fold-local and correct automatically. A static AbstractWeights is fixed at construction and outlives the fold: a longer one used to be read positionally by AggregateDistances, giving the oldest weights to the newest observations with no bounds error, and a shorter one gave a bare BoundsError. The length check makes both loud.

Algorithm

  1. Reshape Z to observations × (assets · features), so the observation axis leads a matrix.
  2. Resolve w against that matrix with get_observation_weights, along its first axis.
  3. Check the resolved length against the observation count of Z, unless the resolution gave nothing.

Arguments

  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.

Validation

  • length(w) == size(Z, 1) once resolved. Raises a DimensionMismatch naming both lengths.

Returns

  • w::Option{<:VecNum}: Resolved observation weights, one entry per observation, or nothing when no weights were requested.

Related

source
PortfolioOptimisers.collapse_rows — Function
collapse_rows(alg::AbstractFeatureCollapseAlgorithm, pnl::AssetPanel)

Name the observation rows a collapse algorithm reads, so the kernel stacks those rows alone.

A time-varying Feature Matrix is stacked from an Asset Panel and then collapsed along its observation axis, and a collapse that reads one row has no use for the others. LastObservation reads the last row, so it names it, and the stack it is handed is a window of one observation: a lifted static Panel Field, whose values are a RepeatedLeading, is then read once rather than once per observation, and the kernel's cost under the default collapse is the assets × features slice it measures. Every other member answers Colon(), every row. The two aggregates resolve their weights against the stacked window itself (see collapse_weights), so a window they could cut is not known before the stack exists, and StackObservations reads the whole stack by definition.

A static panel has no observation axis, so every member answers Colon() on one, LastObservation included.

Algorithm

The method that Julia selects is the algorithm.

  1. LastObservation on a time-varying panel: the last observation, nobs:nobs, where nobs is the observation count panel_axes reads.
  2. Every other case: Colon().

Arguments

  • alg: The collapse algorithm.
  • pnl: The Asset Panel the Feature Matrix is stacked from.

Returns

  • rows: A one-observation range, or Colon(). The rows keyword of feature_matrix.

Related

source