Distance

PortfolioOptimisers.Distance — Type

struct Distance{__T_power, __T_alg} <: AbstractDistanceEstimator

Pairs a distance algorithm with an optional integer power, and applies it to a correlation matrix or to the data.

This is the estimator every clustering, network and phylogeny routine reaches for a distance matrix. alg chooses the transform; power raises the quantity that transform is built on to the integer power $p$, which sharpens the contrast between a strong relationship and a weak one.

Note

power = 1 reproduces the base distance exactly, for every algorithm. It is the neutral position of the knob, not a way to select the generalised estimator. Only $p \geq 2$ changes the result. power = nothing and power = 1 are kept apart by dispatch alone, so that the base case never raises a matrix to a power.

Mathematical definition

The four correlation-based algorithms (see RhoDistanceAlgorithm) raise the correlation to $p$ inside their own base formula.

\[\begin{align} _{g}d_{i,\,j}^{\mathrm{S}} &= \sqrt{\mathrm{clamp}\left(s\left(1 - \rho_{i,\,j}^{p}\right),\, 0,\, 1\right)}\\ _{g}d_{i,\,j}^{\mathrm{SA}} &= \sqrt{\mathrm{clamp}\left(1 - \lvert\rho_{i,\,j}\rvert^{p},\, 0,\, 1\right)}\\ _{g}d_{i,\,j}^{\mathrm{L}} &= \max\left(-\log{\lvert\rho_{i,\,j}\rvert^{p}},\, 0\right)\\ _{g}d_{i,\,j}^{\mathrm{C}} &= \sqrt{\mathrm{clamp}\left(1 - \rho_{i,\,j}^{p},\, 0,\, 1\right)}\\ s &= \begin{cases} 1/2 & \text{if } p \mod 2 \neq 0\\ 1 & \text{otherwise} \end{cases}\,, \end{align}\]

VariationInfoDistance has no correlation to raise, so it raises the distance itself.

\[\begin{align} _{g}d_{i,\,j}^{\mathrm{VI}} &= \left(d_{i,\,j}^{\mathrm{VI}}\right)^{p}\,, \end{align}\]

Where:

$_{g}d_{i,\,j}$: Generalised distance between assets $i$ and $j$, superscripted by the algorithm: SimpleDistance (S), SimpleAbsoluteDistance (SA), LogDistance (L), CorrelationDistance (C), VariationInfoDistance (VI).
$d_{i,\,j}$: Base distance computed using the specified distance algorithm.
$\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
$p$: Integer power.
$s$: Scaling factor of SimpleDistance alone ($s = 1/2$ if $p \bmod 2 \neq 0$, else $s = 1$).

The two cases of $s$ are not a convention. $s$ is the normalisation that keeps the radicand inside $[0,\,1]$ over the reachable range of $\rho_{i,\,j}^{p}$, so it is $1 / (1 - m)$, where $m$ is the smallest value $\rho_{i,\,j}^{p}$ can take. An odd $p$ keeps the sign, so $m = -1$ and $s = 1/2$. An even $p$ cannot be negative, so $m = 0$ and $s = 1$.

The clamp is inert for the first two algorithms at every $p$: $s(1 - \rho_{i,\,j}^{p})$ and $1 - \lvert\rho_{i,\,j}\rvert^{p}$ never leave $[0,\,1]$. It binds for CorrelationDistance at every odd $p$, where $\rho_{i,\,j}^{p}$ keeps its sign and the radicand runs over $[0,\,2]$; that algorithm's own docstring measures the truncation.

CanonicalDistance is a redirect and owns no formula. It forwards power to the algorithm it selects.

`power` raises two different quantities

The field is one field, and the reader must not assume one meaning. For a correlation-based algorithm it raises the correlation, inside the transform. For VariationInfoDistance there is no correlation to raise, so it raises the distance the algorithm returns. So $_{g}d^{\mathrm{S}}$ at $p = 2$ is not $\left(d^{\mathrm{S}}\right)^{2}$, while $_{g}d^{\mathrm{VI}}$ at $p = 2$ is $\left(d^{\mathrm{VI}}\right)^{2}$.

Section 6.2 of [5] carries the four base formulas alone and ends at the tail dissimilarity, so the $p$ generalisation and the scaling $s$ are this library's own extension of it.

Fields

power: Optional matrix exponent. nothing and 1 both give the base distance, so only power >= 2 changes the result.

alg: Distance algorithm.

Constructors

Distance(;    power::Option{<:Integer} = nothing,    alg::AbstractDistanceAlgorithm = SimpleDistance()) -> Distance

Keywords correspond to the struct's fields.

Why the default `alg` is `SimpleDistance`

CanonicalDistance picks an algorithm from the covariance estimator, and falls back to SimpleDistance when the estimator carries no preference. A bare Distance() also serves the matrix entry point distance(de, rho), which holds no estimator to pick from. The two therefore agree except on the estimators CanonicalDistance treats specially. Library entry points that always hold a covariance estimator default to Distance(; alg = CanonicalDistance()) instead, so that the special cases are honoured.

Validation

If power is not nothing, power >= 1.

Examples

julia> Distance()Distance  power ┼ nothing    alg ┴ SimpleDistance()

Related

References

[5] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 6.2.

source

PortfolioOptimisers.distance — Function

distance(de::Distance{<:Any,
                      <:Union{<:SimpleDistance, <:SimpleAbsoluteDistance, <:LogDistance,
                              <:CorrelationDistance, <:CanonicalDistance}},
         ce::StatsBase.CovarianceEstimator, X::MatNum; dims::Int = 1, kwargs...)

Compute the correlation matrix with ce and X, and transform it into a distance matrix with the algorithm de names.

This is the data entry point of the correlation-based family. cor_and_dist is the same computation with the correlation returned alongside the distance, so a caller that needs both pays for the correlation once.

Algorithm

Compute the correlation matrix from ce and X with Statistics.cor, along dims. kwargs are forwarded to it, and it is what refuses a dims outside $(1,\, 2)$.
Transform that correlation matrix with _dist_from_cor, under de.alg and de.power.

A CanonicalDistance de takes one step first: it rebuilds de with SimpleDistance, carrying de.power over, because a ce that is a plain StatsBase.CovarianceEstimator carries no preference. The four estimators that do carry one have their own methods; see the redirect table on CanonicalDistance.

Arguments

de: Distance estimator.
- de::Distance{<:Any, <:SimpleDistance}: Use the SimpleDistance algorithm.
- de::Distance{<:Any, <:SimpleAbsoluteDistance}: Use the SimpleAbsoluteDistance algorithm.
- de::Distance{<:Any, <:LogDistance}: Use the LogDistance algorithm.
- de::Distance{<:Any, <:CorrelationDistance}: Use the CorrelationDistance algorithm.
- de::Distance{<:Any, <:CanonicalDistance}: Use the CanonicalDistance algorithm.
ce: Covariance estimator.
X: Data matrix (observations × assets).
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments passed to the correlation computation.

Validation

dims in (1, 2). The check is not made by this method: Statistics.cor is what raises the DomainError.

Returns

D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source

distance(de::Distance{Nothing, <:VariationInfoDistance}, ::Any, X::MatNum;
         dims::Int = 1, kwargs...)
distance(de::Distance{<:Integer, <:VariationInfoDistance}, ::Any, X::MatNum;
         dims::Int = 1, kwargs...)

Compute the variation of information distance matrix from the data matrix alone.

This is the one algorithm of the family that reads X rather than a correlation matrix, so the covariance estimator is a placeholder that the method ignores. It captures a non-linear relationship that no correlation coefficient sees.

`power` raises the distance here, not the correlation

The two methods above are the two power cases, and they are the reason the trap exists. For a correlation-based algorithm power raises the correlation inside the transform. There is no correlation here, so power raises the distance the algorithm returns: the result is variation_info(...) .^ de.power. One field, two quantities.

Algorithm

Orient X with dims_oriented, which transposes it when dims is 2 and refuses any other value.
Read de.alg.bins and de.alg.normalise off the algorithm, and pass both to variation_info, which builds the joint histograms and forms the distance. VariationInfoDistance states those steps.
When de.power is an Integer, raise the distance matrix of step 2 to it entry by entry. When de.power is nothing, return that matrix as it stands.

Arguments

de: Distance estimator carrying the VariationInfoDistance algorithm.
::Any: Covariance estimator placeholder for API compatibility. It is ignored.
X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments. They are ignored. bins and normalise come from de.alg, never from a keyword.

Validation

dims in (1, 2). The check is not made by this method: dims_oriented is what raises the DomainError.

Returns

D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

Distance
VariationInfoDistance
variation_info
MutualInfoCovariance
CanonicalDistance: what copies bins and normalise off a MutualInfoCovariance onto de.alg.
cor_and_dist

source

distance(::Distance{<:Any,
                    <:Union{<:SimpleDistance, <:SimpleAbsoluteDistance, <:LogDistance,
                            <:CorrelationDistance, <:CanonicalDistance}},
         rho::MatNum, args...; kwargs...)

Compute the distance matrix from a correlation matrix, or from a covariance matrix.

This is the matrix entry point of the correlation-based family, for a caller that already holds the matrix and needs no covariance estimator. The value of the diagonal decides which of the two it was given; see _as_correlation. The distance is the one the algorithm defines, and it is not Euclidean under any of the four: SimpleDistance and SimpleAbsoluteDistance return an angular distance, and LogDistance an unbounded dissimilarity.

Algorithm

Coerce rho to a correlation matrix with _as_correlation, which also checks that it is square.
Transform that correlation matrix with _dist_from_cor, under de.alg and de.power.

A CanonicalDistance de takes one step first: it rebuilds de with SimpleDistance, carrying de.power over. There is no covariance estimator here to select from, so the redirect table cannot apply and the fallback of the table is taken.

Arguments

de: Distance estimator.
- de::Distance{<:Any, <:SimpleDistance}: Use the SimpleDistance algorithm.
- de::Distance{<:Any, <:SimpleAbsoluteDistance}: Use the SimpleAbsoluteDistance algorithm.
- de::Distance{<:Any, <:LogDistance}: Use the LogDistance algorithm.
- de::Distance{<:Any, <:CorrelationDistance}: Use the CorrelationDistance algorithm.
- de::Distance{<:Any, <:CanonicalDistance}: Use the CanonicalDistance algorithm.
rho: Correlation or covariance matrix. A covariance matrix is converted with StatsBase.cov2cor, and the conversion allocates rather than writing into the argument.
args...: Additional arguments. They are ignored. They exist so that this method and the ce-and-X method above take the same call.
kwargs...: Additional keyword arguments. They are ignored. They exist so that this method and the ce-and-X method above take the same call.

Validation

rho is square.

Returns

D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source

distance(de::Distance{<:Any, <:CanonicalDistance},
         ce::Union{<:MutualInfoCovariance,
                   <:AllInternalMutualInfoCov,
                   <:LTDCov_AllInternalLTDCov,
                   <:DistCov_AllInternalDistCov},
         X::MatNum; dims::Int = 1, kwargs...)

Rebuild de with the distance algorithm that the covariance estimator's own range calls for, and call distance again.

The redirect owns no formula. It exists so that a codependence measure reaches the transform its range needs: a signed correlation must be halved, a mutual information has no correlation to transform at all, and a tail dependence coefficient wants an unbounded distance. de.power is carried over unchanged onto the algorithm that is selected.

Covariance estimator	Algorithm selected	Read from `ce`
`MutualInfoCovariance`	`VariationInfoDistance`	`ce.bins`, `ce.normalise`
`PortfolioOptimisersCovariance` wrapping it	`VariationInfoDistance`	`ce.ce.bins`, `ce.ce.normalise`
`LowerTailDependenceCovariance`, wrapped or not	`LogDistance`	nothing
`DistanceCovariance`, wrapped or not	`CorrelationDistance`	nothing
any other `StatsBase.CovarianceEstimator`	`SimpleDistance`	nothing

The last row is the fallback, and it is also what a bare Distance with SimpleDistance gives. The two agree on every estimator outside the table.

Algorithm

Select the row of the table above by the type of ce. Dispatch does the selection, so a wrapper reaches the same row as the estimator it wraps.
Build a fresh Distance carrying de.power and the algorithm of that row.
On the two mutual-information rows, copy bins and normalise off ce onto the new VariationInfoDistance, one field level deeper for the wrapper. This step is invisible to a reader of the signature, and it is load-bearing: a MutualInfoCovariance built with a non-default bins gives a different distance matrix from one built with the default, and the redirect is what carries that setting across.
Call distance with the rebuilt estimator, the same ce, and the same X, dims and kwargs.

Arguments

de: Distance estimator carrying the CanonicalDistance algorithm.
ce: Covariance estimator.
X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments passed to the selected algorithm.

Returns

D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

Distance
CanonicalDistance
MutualInfoCovariance
LowerTailDependenceCovariance
DistanceCovariance
PortfolioOptimisersCovariance
cor_and_dist: carries the same table, over the same five rows.

source

distance(de::DistanceDistance, ce::StatsBase.CovarianceEstimator, X::MatNum;
         dims::Int = 1, kwargs...)

Compute the distance-of-distances matrix from a covariance estimator and data matrix.

Algorithm

Build a base Distance carrying de.power and de.alg, and call distance on it with ce, X, dims and kwargs, giving the base distance matrix D.
Apply de.metric to D with Distances.pairwise, forwarding the estimator's own de.args and de.kwargs, giving the distance-of-distances matrix.

An alg of CanonicalDistance redirects inside step 1, by the type of ce, so the algorithm that produces D is chosen before the metric is applied. distance carries the redirect table.

Distances.pairwise treats a column of D as one observation. The call passes dims = 2 itself, before the splat of de.kwargs, so a dims in de.kwargs overrides it. Which axis is read does not change the answer here, because a base distance matrix is symmetric. It matters only when de.args supplies a second matrix of a different shape.

Arguments

de: Distance-of-distances estimator.
ce: Covariance estimator.
X: Data matrix (observations × assets).
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments passed to the base distance computation of step 1. They never reach Distances.pairwise, which is served by the kwargs field of de.

Returns

D::Matrix{<:Number}: Matrix of pairwise distances of distances.

Related

source

distance(de::DistanceDistance, rho::MatNum, args...; kwargs...)

Compute the distance-of-distances matrix from a correlation or covariance matrix.

Algorithm

Build a base Distance carrying de.power and de.alg, and call distance on it with rho, args and kwargs, giving the base distance matrix D.
Apply de.metric to D with Distances.pairwise, forwarding the estimator's own de.args and de.kwargs, giving the distance-of-distances matrix.

An alg of CanonicalDistance takes one step before step 1, and rebuilds itself with SimpleDistance. There is no covariance estimator on this route, so the redirect table cannot select a row and its fallback is taken.

Distances.pairwise reads the columns of D, as it does on the covariance-estimator route above. The call passes dims = 2 itself, and a dims in de.kwargs overrides it.

Arguments

de: Distance-of-distances estimator.
rho: Correlation or covariance matrix.
args...: Additional arguments. They are forwarded to the base distance of step 1, which ignores them. They are not the args field of de, which is what reaches Distances.pairwise.
kwargs...: Additional keyword arguments passed to the base distance computation of step 1. They never reach Distances.pairwise, which is served by the kwargs field of de.

Returns

D::Matrix{<:Number}: Matrix of pairwise distances of distances.

Related

source

distance(de::FeatureDistance, Z::MatNum; dims::Int = 1, kwargs...)
distance(de::FeatureDistance, Z::Arr3Num; dims::Int = 1, kwargs...)

Compute the distance matrix from a feature matrix.

The 2-D method never consults de.alg: a static feature matrix has no observation axis to collapse, so the collapse algorithm is inert rather than an error. The 3-D method dispatches on it. Assets whose feature vector is entirely zero are given the convention documented in patch_zero_feature_vectors!.

Algorithm

Validate Z and dims with assert_feature_matrix.
On the 2-D method, hand de.metric and Z to the kernel.
On the 3-D method, hand de and Z to the collapse dispatcher, which selects the branch that de.alg names.

Arguments

de: Feature distance estimator.
Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments (ignored).

Validation

dims in (1, 2).
!isempty(Z).
all(isfinite, Z).
Z lies in de.metric's domain (see assert_metric_domain).

Returns

D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Examples

julia> Z = [1.0 0.0; 0.0 1.0; 1.0 1.0];julia> distance(FeatureDistance(), Z)3×3 Matrix{Float64}: 0.0   0.5   0.25 0.5   0.0   0.25 0.25  0.25  0.0

Related

source

distance(de::FeatureDistance, ::Any, X; pr = nothing, rd = nothing, kwargs...)
cor_and_dist(de::FeatureDistance, ::Any, X; pr = nothing, rd = nothing, kwargs...)

Three-argument entry points, for the clustering and network estimators.

Every consumer in the clustering and network stack calls cor_and_dist(de, ce, X; …) or distance(de, pl, X; …), passing a covariance estimator (or, in logo!'s case, a similarity matrix) and a returns matrix. FeatureDistance uses the covariance positional not at all, and it is typed ::Any rather than bounded — logo! puts a similarity matrix where the others put a covariance estimator. It does read X, which a producer measures.

The two carriers ride the keyword tail as pr and rd, and feature_matrix resolves the panel from them and from de.ape. A forwarder that takes a prior result passes both through; preselection passes rd alone.

dims is ignored and the kernel is called with dims = 1. The ambient dims describes the returns matrix X, and a stacked Feature Matrix is canonically assets-major regardless of it. dims stays meaningful only at the raw-matrix entry point distance(de, Z; dims).

Algorithm

Stack the Feature Matrix with feature_matrix, which resolves the panel and cuts it to de.sel and to the observation rows de.alg reads.
Call the matching two-argument entry point on it, at dims = 1.

Arguments

de: Feature distance estimator.
The second positional: ignored. Present so this estimator matches the signature every consumer calls.
X: Returns matrix of the subproblem, observations × assets.
pr: Prior result or returns result. Both carry the asset returns matrix X and the feature matrix Z, so either can supply them.
rd: The returns result to use.
kwargs...: Additional keyword arguments (ignored).

Validation

The panel resolves. See asset_panel, which raises an IsNothingError naming the site when it does not.

Returns

From distance: D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.
From cor_and_dist: the tuple (S, D) of the similarity matrix and that same distance matrix.

Related

source

PortfolioOptimisers.cor_and_dist — Function

cor_and_dist(de::Distance{<:Any,
                          <:Union{<:SimpleDistance, <:SimpleAbsoluteDistance,
                                  <:LogDistance, <:CorrelationDistance,
                                  <:VariationInfoDistance, <:CanonicalDistance}},
             ce::StatsBase.CovarianceEstimator, X::MatNum; dims::Int = 1, kwargs...)

Compute the correlation matrix and the distance matrix together, from one pass over the data.

It returns the same D that distance returns for the same arguments; that agreement is the claim of having two entry points. Take this one when both matrices are wanted, because the correlation-based family then computes the correlation once instead of twice. The correlation it returns is the one ce computes, untransformed: it is not the magnitude that SimpleAbsoluteDistance and LogDistance take, nor the power of it that a de.power raises.

Algorithm

Which of the three routes runs is decided by de.alg.

A correlation-based algorithm computes the correlation matrix with Statistics.cor, then transforms that same matrix with _dist_from_cor. This is the route that saves the second correlation.
VariationInfoDistance checks dims with assert_dims, computes the correlation matrix for the caller, and calls distance for the distance. The distance reads X, not the correlation, so nothing is shared between the two.
CanonicalDistance rebuilds de with the algorithm its redirect table names for ce, then re-enters at route 1 or route 2. See distance for the table and for the fields the rebuild copies.

Arguments

de: Distance estimator.
ce: Covariance estimator.
X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments passed to the correlation computation.

Validation

dims in (1, 2). Route 2 makes the check itself with assert_dims. Route 1 leaves it to Statistics.cor, which raises the same DomainError.

Returns

rho::MatNum: Correlation matrix assets x assets.
D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Related

source

cor_and_dist(de::DistanceDistance, ce::StatsBase.CovarianceEstimator, X::MatNum;
             dims::Int = 1, kwargs...)

Compute both the correlation matrix and the distance-of-distances matrix from a covariance estimator and data matrix.

Algorithm

Build a base Distance carrying de.power and de.alg, and call cor_and_dist on it with ce, X, dims and kwargs, giving the correlation matrix rho and the base distance matrix D from one pass. This is the pass that the distance sibling cannot share.
Apply de.metric to D with Distances.pairwise, forwarding the estimator's own de.args and de.kwargs, giving the distance-of-distances matrix.
Return rho unchanged beside that matrix. The metric is applied to the distance matrix alone, so the correlation this method returns is the base estimator's own.

The second element is the matrix that distance returns for the same de, ce and X, so a caller that needs both quantities pays for one correlation rather than two.

Arguments

de: Distance-of-distances estimator.
ce: Covariance estimator.
X: Data matrix (observations × assets).
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments passed to the base distance computation of step 1. They never reach Distances.pairwise, which is served by the kwargs field of de.

Returns

(rho::Matrix{<:Number}, D::Matrix{<:Number}): Tuple of correlation matrix and distance-of-distances matrix.

Related

source

cor_and_dist(de::FeatureDistance, Z::MatNum; dims::Int = 1, kwargs...)
cor_and_dist(de::FeatureDistance, Z::Arr3Num; dims::Int = 1, kwargs...)

Compute the similarity and distance matrices from a feature matrix.

The similarity shares the distance's provenance: it is distance_to_similarity(de.sim; D = D), derived from the distance matrix this call just produced, so S and D are two views of one measurement rather than two independent estimates. Deriving it from the aggregated distance is also what keeps the zero-feature-vector convention consistent under AggregateDistances, since $\mathrm{mean}(\cos(\pi D_{t})) \neq \cos(\pi\,\mathrm{mean}(D_{t}))$.

Algorithm

Compute the distance matrix D with distance, which validates Z and dims on the way.
Transform D with distance_to_similarity under de.sim, giving S.

Arguments

de: Feature distance estimator.
Z: Feature matrix assets × features if dims = 1, features × assets when dims = 2. May also be a 3-D array of time-varying features, in which case the observation axis always leads: observations × assets × features if dims = 1, observations × features × assets when dims = 2.
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments (ignored).

Validation

dims in (1, 2).
!isempty(Z).
all(isfinite, Z).
Z lies in de.metric's domain (see assert_metric_domain).

Returns

S::Matrix{<:Number}: Similarity matrix, assets × assets.
D::MatNum: Distance matrix assets x assets, in the units the distance algorithm defines.

Examples

julia> Z = [1.0 0.0; 0.0 1.0; 1.0 1.0];julia> S, D = cor_and_dist(FeatureDistance(), Z);julia> S3×3 Matrix{Float64}: 1.0          6.12323e-17  0.707107 6.12323e-17  1.0          0.707107 0.707107     0.707107     1.0

Related

source

References

[5]: D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).