Asset Panel Estimators

An Asset Panel estimator is a producer: FeatureDistance holds one in its ape slot, and it builds a static AssetPanel at the point of use, from the prior result and the returns of the subproblem that runs it.

PortfolioOptimisers.Proximity — Type

struct Proximity{__T_decay} <: AbstractPhylogenyFeatureAlgorithm

Phylogeny feature algorithm scoring each pair by how far apart it sits.

Z[i, k] = decay(separation(i, k)) inside the budget and 0 beyond, so the score falls off with distance instead of flattening to an indicator. The separation and its budget come from the source NetworkEstimator's sep — hops under HopCount, a summed shortest path under PathLength — and the fall-off from decay.

The score is a function of the separation rather than of the un-clamped walk count sum(A^i). A walk count is degree-biased — a hub accumulates walks combinatorially — so two assets' scores would encode how busy their neighbourhoods are as much as how close they are. It is for the same reason strictly richer than phylogeny_matrix's output, which accumulates that walk count and then clamp!(P, 0, 1) destroys the step count: this is the information this algorithm keeps.

The two knobs

decay shapes the fall-off; sep on the source NetworkEstimator truncates it. They are deliberately separate — an exponential never reaches zero, so a budget cannot be expressed as a fall-off — and the budget is the only place truncation happens. Under the default LinearDecay and HopCount the two coincide in appearance: a direct neighbour scores n, a two-hop neighbour n - 1, the asset itself n + 1, and the score would hit 0 exactly one hop past the budget that already cut it. Under ExponentialDecay or ReciprocalDecay the diagonal is 1 and the fall-off is set by the member's own parameter, independently of how far the budget looks.

NoDecay is the flat end of that dial and is worth stating on its own, because it is what a binary neighbourhood indicator now is: the budget still cuts, so Proximity(; decay = NoDecay()) gives 1 inside it and 0 outside — an indicator, not a matrix of ones.

A zero entry means functionally unreachable: either the pair is disconnected or outside the budget, or the decay has fallen to nothing — the same claim about the pair, since AbstractSeparationDecayAlgorithm forbids anything below zero inside the budget. No shipped decay other than the flat one emits zero there, so for what ships a zero is unreachable-or-out-of-budget and nothing else.

Two separations, and what may be compared across them

A Z graded over hops and a Z graded over path lengths are interchangeable as inputs — both satisfy the same contract, so every consumer takes either — and are not comparable as values. The budgets are in different units, the supports differ, and under LinearDecay so do the scales.

On any real universe they will nevertheless look interchangeable: measured over twenty assets, rho = 0.99 on a minimum spanning tree and 0.95 to 0.98 on a PMFG. That is empirical rather than guaranteed — both structures are selected by distance, so their two readings of it rarely disagree — and it is not a licence to compare one run's numbers against another's.

Under PathLength's default dmax = nothing the budget is the observed diameter, so f(0) is data-dependent for LinearDecay and a diameter that moves between cross-validation folds shifts every entry of Z rather than rescaling it. A fixed dmax buys back the fold-stability, and the decays that pin f(0) = 1 never had the exposure.

Unreachable pairs

An unreachable pair carries separation_matrix's sentinel — typemax, which is typemax(Int) for a hop count and Inf for a path length over Float64 weights — so the budget comparison both selects the 0 and guards the decay call: separation_decay is never evaluated at the sentinel. The guard is load-bearing rather than tidy — ReciprocalDecay overflows 1 + d there, and for a fractional power that is a DomainError rather than a discarded number.

Fields

decay: Separation decay algorithm, the rule by which the score falls off as two assets get further apart. Distinct from the exponentially weighted moment estimators' decay, which is a smoothing constant over observations.

Constructors

Proximity(;    decay::AbstractSeparationDecayAlgorithm = LinearDecay()) -> Proximity

Keywords correspond to the struct's fields.

Examples

julia> Proximity(; decay = ExponentialDecay(; rate = 0.5))Proximity  decay ┼ ExponentialDecay        │   rate ┴ Float64: 0.5

Related

source

PortfolioOptimisers.RegressionPanel — Type

struct RegressionPanel <: AbstractAssetPanelEstimator

Builds an Asset Panel holding the factor loadings the wrapped prior fitted.

RegressionPanel treats pr.rr.L — the coordinate system a factor model places each asset in — as the values of one tensor Panel Field, "loadings" on the axis "factor". It is the cheapest real feature source in the library: a factor prior already computes the loadings, and already refits them per fold, so the features track the fold with no extra plumbing.

The matrix is L, not M

L is assets × reduced_dimensions, set by DimensionReductionRegression: the low-dimensional coordinate system the asset actually lives in. M is the reconstructed full-factor loadings. pr.rr.L always resolves — Regression swaps in M when L is unset — so this producer needs no branch and works behind every regression estimator, the time-series one and the cross-sectional one alike.

Both are assets-major, so the panel's asset axis is the carrier's with no transpose. The trailing axis is the reduced dimensions, which are not assets, so a produced loadings panel is never square in the sense features_are_assets means.

Validation

The call supplies a prior result. Raises an IsNothingError naming the site.
The wrapped prior carries a regression (see assert_prior_regression). Nesting order does not matter: every wrapping estimator forwards rr and the factor block fpr. What throws is a prior that never computed a regression at all, such as EmpiricalPrior.

Examples

julia> RegressionPanel()RegressionPanel()

Related

source

PortfolioOptimisers.PhylogenyPanel — Type

struct PhylogenyPanel{__T_pl, __T_alg} <: AbstractAssetPanelEstimator

Builds an Asset Panel holding a square proximity matrix graded from a graph or a partition.

An assets × assets proximity matrix is a feature block whose feature k reads "is close to asset k", so a metric over its rows measures neighbourhood overlap — a standard notion of topological similarity — and needs almost no new estimation code. PhylogenyPanel returns it as one tensor Panel Field, "proximity" on the axis "asset".

It is the one producer whose trailing axis is the asset axis. That needs no flag anywhere: it refits, so a subproblem gets its own square matrix over its own universe rather than a slice of a larger one, and features_are_assets compares the field's labels against the carrier's asset names where a hand-supplied panel meets a view.

Fields

pl: Structure source, always an estimator so that it refits per fold: a network estimator (a graph, whose sep measures the separations alg grades) or a clustering estimator (a partition, for which alg is inert). A precomputed result is not accepted – an Estimator does not hold a Result.

alg: Phylogeny feature algorithm: the rule turning the source's separations into feature values. Inert for a partition source, which has no separation to grade.

Constructors

PhylogenyPanel(;    pl::NwE_ClE = NetworkEstimator(),    alg::AbstractPhylogenyFeatureAlgorithm = Proximity()) -> PhylogenyPanel

Keywords correspond to the struct's fields.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

pl: Recursively updated via factory.

View parameters

When port_opt_view is called on this type, the following @vprop-tagged fields are automatically subset to the selected indices:

pl: Recursively viewed via port_opt_view.

Why the diagonal includes self

The diagonal is not a convention, it selects between two different algorithms. Measured on a three-node path 1 - 2 - 3 under the default AngularDist, with the decay held flat at NoDecay so that the diagonal is the only thing that changes between the two columns:

pair	zero diagonal	self included
`1`-`3`, 2 hops	0.0	0.333
`1`-`2`, 1 hop	0.5	0.196

The default LinearDecay grades the same graph rather than flattening it, so its numbers differ — 0.436 and 0.239 over a two-hop budget — while the ordering is the one the right-hand column shows.

With a zero diagonal the two non-adjacent endpoints come out identical and the adjacent pairs maximally far: rows are compared on who their neighbours are, never on whether they are each other's. That is structural equivalence — similarity of role — which is a real notion but the opposite of the proximity the name promises.

Including self also keeps subproblems well defined. An asset view of a spanning tree routinely isolates a vertex, and a zero-diagonal row for an isolated asset is a zero row: AngularDist's zero-vector convention then puts every isolated asset at distance 0 from every other isolated asset, clustering them together for no reason. With self included they sit at maximal distance from everything, including each other.

A clustering source is admitted, with a caveat worth reading

pl is bound by NwE_ClE: a graph (NetworkEstimator) or a partition (ClustersEstimator). Both are estimators, so both refit.

A partition carries much less than a graph, and the shortfall is measurable rather than stylistic. Its matrix is P * transpose(P) with the diagonal restored, so row i is the co-membership indicator of asset i and two rows are either identical or disjoint. On a seven-asset universe clustered [1, 1, 1, 2, 2, 3, 3] the whole distance matrix takes two distinct values under the default AngularDist: 0.0 within a cluster and 0.5 across one, whatever the cluster sizes. The raw phylogeny_matrix output, whose - I this producer undoes, takes three — 0.0, 0.333 and 0.5 — because that - I makes each row of a pair a lone 1 pointing at the other member, so the two rows are orthogonal and a size-two cluster's within-cluster distance equals its across-cluster distance. Restoring the diagonal repairs exactly that case; the coarseness remains.

Clustering a re-encoded clustering largely returns the clustering. Prefer a graph source unless the partition is what you actually want to measure.

Provenance

The source is a NetworkEstimator and never a precomputed PhylogenyResult, because an Estimator does not hold a Result (see CONTEXT.md §1). This producer is therefore endogenous: the graph is filtered from the returns correlation, so it measures topology the correlation implies rather than structure outside it, and it refits on every fold and every subproblem.

It reads no prior result, so it is the one producer that runs at a pre-prior site such as preselection.

Examples

julia> PhylogenyPanel()PhylogenyPanel   pl ┼ NetworkEstimator      │    ce ┼ PortfolioOptimisersCovariance      │       │   ce ┼ Covariance      │       │      │    me ┼ SimpleExpectedReturns      │       │      │       │   w ┴ nothing      │       │      │    ce ┼ GeneralCovariance      │       │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)      │       │      │       │    w ┴ nothing      │       │      │   alg ┼ FullMoment()      │       │      │     w ┴ nothing      │       │   mp ┼ MatrixProcessing      │       │      │     pdm ┼ Posdef      │       │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton      │       │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()      │       │      │      dn ┼ nothing      │       │      │      dt ┼ nothing      │       │      │     alg ┼ nothing      │       │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)      │    de ┼ Distance      │       │   power ┼ nothing      │       │     alg ┴ CanonicalDistance()      │   alg ┼ KruskalTree      │       │     args ┼ Tuple{}: ()      │       │   kwargs ┴ @NamedTuple{}: NamedTuple()      │   sep ┼ HopCount      │       │   n ┴ Int64: 1  alg ┼ Proximity      │   decay ┴ LinearDecay()

Related

source

PortfolioOptimisers.phylogeny_features — Function

phylogeny_features(alg::Proximity, pl::AbstractNetworkEstimator,
                   X::MatNum; kwargs...)
phylogeny_features(alg::AbstractPhylogenyFeatureAlgorithm,
                   pl::AbstractClustersEstimator, X::MatNum; kwargs...)

Turn a graph source into a square assets × assets feature matrix.

The kernel behind PhylogenyPanel. Every method returns a matrix in eltype(X), the type of the returns it graded — not the Int or BitMatrix the phylogeny routines produce — so that AngularDist keeps its BLAS gemm path and a Float32 history grades in Float32.

The source is always refit

pl is an estimator — a NetworkEstimator or a ClustersEstimator — never a precomputed PhylogenyResult or Clusters, because an Estimator does not hold a Result (see CONTEXT.md §1). The structure is therefore rebuilt from X on every call, so it tracks a cross-validation fold or a meta-optimiser's subproblem instead of describing a universe it no longer sees.

alg applies to a graph, and is inert for a partition

A graph source has separation structure, so alg decays it — over the separations its sep measures.

A partition has none: two assets are in the same cluster or they are not, and there is nothing between them to decay. Every algorithm therefore gives the same co-membership matrix, and alg is inert rather than an error — the same treatment FeatureDistance's collapse alg gets on a static feature matrix.

The inert surface is alg alone. sep lives on NetworkEstimator, which a clustering source does not have, so there is no second field going quiet here.

The diagonal

Z[i, i] is the top of the scale, never zero: 1 for any clustering source, and separation_decay(decay, 0, dmax) for Proximity over a graph — n + 1 under the default LinearDecay and HopCount, the observed diameter plus one under PathLength's default budget, 1 for the members that pin f(0) = 1. That the diagonal is maximal is a contract on AbstractSeparationDecayAlgorithm, checked before the loop by assert_separation_decay.

Algorithm

Over a graph source, under Proximity:

Build the structure from X through separation_graph, giving g.
Resolve the separation algorithm against the structure through resolve_separation, giving sep.
Measure the separations through separation_matrix, giving d.
Read the budget through separation_budget, and score d through _proximity_features.

Over a partition source, under any algorithm:

Build the co-membership matrix from X through phylogeny_matrix, and convert it to eltype(X).
Add the identity, restoring the diagonal that phylogeny_matrix subtracts.

Arguments

alg: Phylogeny feature algorithm.
pl: Structure source — a network estimator (a graph) or a clustering estimator (a partition).
X: Asset returns matrix observations × assets.
kwargs...: Additional keyword arguments passed to the underlying phylogeny routines.

Returns

Z::Matrix{Float64}: Square assets × assets feature matrix.

Related

source

PortfolioOptimisers.asset_panel — Method

asset_panel(ape::RegressionPanel, pr, rd, X) -> AssetPanel
asset_panel(ape::PhylogenyPanel, pr, rd, X) -> AssetPanel

Build the static AssetPanel a producer returns, at the point of use.

Each method returns a panel holding one TensorPanelField, because a loadings matrix and a proximity matrix are each one quantity with a labelled third axis. The trailing axis is labelled off the data carrier or the regression block where a name exists there, and positionally otherwise; panel_axis_labels and regression_factor_names state the rule.

A producer runs on the subproblem's own prior and returns, so nothing views what it built and a fold refits it. Standalone on a prior fitted on a point-in-time Asset Panel, a RegressionPanel reads the loadings on the prior's Investable Mask and answers the full universe, with a zero row and a false observed mask outside it, so the panel it builds can be handed back to an optimiser; expand_investable_loadings states the rule.

Algorithm

A RegressionPanel takes four steps:

Check that a prior result reached the call, with assert_producer_prior.
Check that the prior carries a regression, with assert_prior_regression.
Reduce the prior to its Investable Mask through investable_mask and port_opt_view, and read the loadings there. A prior fitted on a point-in-time Asset Panel writes NaN on the loadings of every asset outside its mask, and a Panel Field admits no NaN. Inside an optimiser the prior arrives reduced and the view is the whole universe.
Check that every loading on the mask is finite, and refuse otherwise: an asset the check counts has a finite moment and a loadings row that is not, which is a defect of the regression.
Label the loadings axis with panel_axis_labels, from regression_factor_names.
Expand the loadings back onto the full universe with expand_investable_loadings: a zero row and a false observed mask outside the mask, the same rule every uncertainty set fitted standalone on such a prior follows.
Return the panel holding them as the field "loadings" on the axis "factor".

A PhylogenyPanel takes three steps:

Grade the structure into an assets × assets matrix with phylogeny_features.
Label the trailing axis with panel_axis_labels, from carrier_asset_names.
Return the panel holding that matrix as the field "proximity" on the axis "asset".

Arguments

ape: The producer.
pr: Prior result or returns result. Both carry the asset returns matrix X and the feature matrix Z, so either can supply them.
rd: The returns result to use. Read for the axis names alone.
X: Returns matrix of the subproblem, observations × assets.

Validation

A RegressionPanel needs a prior result carrying a regression. Raises an IsNothingError.
A RegressionPanel needs every loading inside the prior's Investable Mask to be finite. Raises an IsNonFiniteError counting the assets whose loadings are not.

Returns

pnl::AssetPanel: A static Asset Panel holding one tensor Panel Field.

Related

source