Base Distance
PortfolioOptimisers.SimpleDistance — Type
struct SimpleDistance <: AbstractDistanceAlgorithmTurns a signed correlation into a distance by $\sqrt{(1 - \rho) / 2}$.
The factor $1/2$ keeps the result in $[0,\,1]$ over the whole signed range of the correlation. So this is the algorithm for a codependence measure on $[-1,\,1]$, such as the Pearson, Spearman and Kendall correlations and the Gerber statistic.
Mathematical definition
\[\begin{align} d_{i,\,j} &= \sqrt{\mathrm{clamp}\left(\dfrac{1 - \rho_{i,\,j}}{2},\, 0,\, 1\right)}\,, \end{align}\]
Where:
- $d_{i,\,j}$: Pairwise distance between assets $i$ and $j$.
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
The clamp is only a numerical guard. $(1 - \rho_{i,\,j}) / 2$ already lies in $[0,\,1]$ for every $\lvert\rho_{i,\,j}\rvert \leq 1$. So the clamp binds only when an estimator that shrinks, denoises or repairs the matrix moves a correlation slightly outside $[-1,\,1]$. In CorrelationDistance, the same clamp binds on every negative correlation.
Lemma 3 of [111] states this distance as $\sin(\theta / 2)$ of the angle $\theta = \arccos(\rho)$. The lemma proves that it is a metric for a Pearson or a Spearman correlation and for a cosine. [112] also states it for the correlation of asset returns. That work gives a separate proof that it is a metric, and a worked example.
Related
AbstractDistanceAlgorithmAbstractDistanceEstimatorSimpleAbsoluteDistanceCorrelationDistancedistancecor_and_dist
References
- [111] S. Van Dongen and A. J. Enright. Metric distances derived from cosine similarity and Pearson and Spearman correlations. arXiv preprint arXiv:1208.3145 (2012). Section 4, Lemma 3.
- [112] M. López de Prado. Building diversified portfolios that outperform out of sample. The Journal of Portfolio Management 42, 59–69 (2016). Online supplementary materials, Stage 1, Example 1, and Appendix A.1.
PortfolioOptimisers.SimpleAbsoluteDistance — Type
struct SimpleAbsoluteDistance <: AbstractDistanceAlgorithmTurns the magnitude of a correlation into a distance by $\sqrt{1 - \lvert\rho\rvert}$.
The magnitude discards the sign. So two assets that move together and two assets that move in opposite directions with the same strength are equally close. This is the algorithm for a codependence measure that is already non-negative, and for the case where only the strength of the relationship matters.
Mathematical definition
\[\begin{align} d_{i,\,j} &= \sqrt{\mathrm{clamp}\left(1 - \lvert\rho_{i,\,j}\rvert,\, 0,\, 1\right)}\,, \end{align}\]
Where:
- $d_{i,\,j}$: Pairwise distance between assets $i$ and $j$.
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
As in SimpleDistance, the clamp is only a numerical guard. $1 - \lvert\rho_{i,\,j}\rvert$ already lies in $[0,\,1]$ for every $\lvert\rho_{i,\,j}\rvert \leq 1$.
Appendix A.1 of the online supplementary materials of [112] states this distance and proves that it is a metric for a Pearson correlation.
Related
AbstractDistanceAlgorithmAbstractDistanceEstimatorSimpleDistanceCorrelationDistancedistancecor_and_dist_absguard: supplies the magnitude. It decides only whether to allocate a copy, and it does not change the mathematics. On a non-negative codependence measure it returns the same matrix and allocates nothing.
References
- [112] M. López de Prado. Building diversified portfolios that outperform out of sample. The Journal of Portfolio Management 42, 59–69 (2016). Online supplementary materials, Appendix A.1.
PortfolioOptimisers.LogDistance — Type
struct LogDistance <: AbstractDistanceAlgorithmTurns the magnitude of a correlation into an unbounded distance by $-\log\lvert\rho\rvert$.
The range is $[0,\,\infty)$ and not $[0,\,1]$, so the distance grows without limit as the relationship weakens. This is the algorithm for a tail dependence coefficient, and CanonicalDistance selects it for LowerTailDependenceCovariance.
Mathematical definition
\[\begin{align} d_{i,\,j} &= \max\left(-\log{\lvert\rho_{i,\,j}\rvert},\, 0\right)\,, \end{align}\]
Where:
- $d_{i,\,j}$: Pairwise distance between assets $i$ and $j$.
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
The floor at zero is necessary. A covariance estimator that shrinks, denoises or repairs a matrix can return $\lvert\rho_{i,\,j}\rvert$ slightly above one, and $-\log$ of that value is negative. The square-root algorithms do not have this problem, because they clamp before the root. A negative distance inverts the order that the distance expresses, and a shortest-path routine that reads it gives a wrong result.
Two uncorrelated assets are infinitely far apart. $\rho_{i,\,j} = 0$ gives $d_{i,\,j} = \infty$, and the algorithm keeps that value.
Equation 2 of Section 3 of [38] proposes $-\log \hat{\lambda}_L$ for an estimated lower tail dependence coefficient $\hat{\lambda}_L$. That section calls it a dissimilarity and not a distance, because it does not satisfy the triangle inequality. A tail dependence coefficient lies in $[0,\,1]$, so the absolute value and the floor at zero are this library's guards.
Related
AbstractDistanceAlgorithmAbstractDistanceEstimatorLowerTailDependenceCovarianceCanonicalDistanceassert_similarity_domain: on the PMFG path, refuses an entry above1underComplementSimilarityand an infinite entry underMaximumDistanceSimilarity. This algorithm can give both.distancecor_and_dist_absguard: supplies the magnitude, as it does forSimpleAbsoluteDistance. It decides only whether to allocate a copy, and it does not change the mathematics.
References
- [38] G. De Luca and P. Zuccolotto. A tail dependence-based dissimilarity measure for financial time series clustering. Advances in Data Analysis and Classification 5, 323–340 (2011). Section 3, Equation 2.
PortfolioOptimisers.CorrelationDistance — Type
struct CorrelationDistance <: AbstractDistanceAlgorithmTurns a non-negative codependence into a distance by $\sqrt{1 - \rho}$, with no factor $1/2$.
It is SimpleAbsoluteDistance without the absolute value. It is the algorithm for a codependence measure whose range is $[0,\,1]$, and CanonicalDistance selects it for DistanceCovariance.
Mathematical definition
\[\begin{align} d_{i,\,j} &= \sqrt{\mathrm{clamp}\left(1 - \rho_{i,\,j},\, 0,\, 1\right)}\,, \end{align}\]
Where:
- $d_{i,\,j}$: Pairwise distance between assets $i$ and $j$.
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
This is the one algorithm in the family whose clamp is not a numerical guard. Without the factor $1/2$, $1 - \rho_{i,\,j}$ runs over $[0,\,2]$ on a signed correlation, so the clamp truncates every negative entry to a distance of exactly 1.
The truncation removes the order of the negative entries. $\rho_{i,\,j} = -0.1$ and $\rho_{i,\,j} = -1$ both give $1$, but $\sqrt{1 - \rho_{i,\,j}}$ gives $\sqrt{1.1}$ and $\sqrt{2}$. So the distance does not separate two negative correlations.
The intended domain has no negative entry. Give a signed correlation to SimpleDistance, which halves the radicand and so never saturates. Or give it to SimpleAbsoluteDistance, which reads the magnitude.
On a non-negative $\rho_{i,\,j}$ the formula is the distance $\sqrt{1 - \lvert\rho_{i,\,j}\rvert}$ of Appendix A.1 of the online supplementary materials of [112]. That appendix proves the metric property for a Pearson correlation alone. Section 3.2 of [26] proves it again for a correlation estimate with $\sigma[X, Y] = \rho[X, Y] \sigma[X] \sigma[Y]$, and it does not name a distance correlation. Section 6.2.1 of [12] gives the formula for a distance correlation. A search found no earlier work that states that use.
Related
AbstractDistanceAlgorithmAbstractDistanceEstimatorSimpleDistanceSimpleAbsoluteDistanceDistanceCovarianceCanonicalDistancedistancecor_and_dist
References
- [112] M. López de Prado. Building diversified portfolios that outperform out of sample. The Journal of Portfolio Management 42, 59–69 (2016). Online supplementary materials, Appendix A.1.
- [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 3.2.
- [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 6.2.1, Equation 6.23. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.VariationInfoDistance — Type
struct VariationInfoDistance{__T_bins, __T_normalise} <: AbstractDistanceAlgorithmMeasures the information one asset loses about another, from the entropies of a joint histogram.
It is the only algorithm in the family that reads the data matrix and not a correlation matrix. So it can measure a non-linear dependence that a correlation coefficient does not show. CanonicalDistance selects it for MutualInfoCovariance.
Mathematical definition
\[\begin{align} d_{i,\,j} &= H(X_{i}) + H(X_{j}) - 2\,I(X_{i};X_{j})\,, \end{align}\]
When normalise is true, the algorithm divides the result by the joint entropy:
\[\begin{align} \tilde{d}_{i,\,j} &= \dfrac{H(X_{i}) + H(X_{j}) - 2\,I(X_{i};X_{j})}{H(X_{i}) + H(X_{j}) - I(X_{i};X_{j})}\,. \end{align}\]
Where:
- $d_{i,\,j}$: Pairwise variation of information between assets $i$ and $j$.
- $\tilde{d}_{i,\,j}$: Normalised pairwise variation of information.
- $H(X_{i})$: Marginal Shannon entropy of asset $i$, estimated from a histogram whose bin count comes from
bins. - $I(X_{i};X_{j})$: Mutual information between assets $i$ and $j$.
The first formula is the variation of information of Equation 19 of [51], which proves that it is a metric. The second is $d = 1 - I(X_{i};X_{j}) / H(X_{i},\,X_{j})$ of Equation 1 of [113], which proves that it is a distance on discrete distributions. Theorem 1 of [114] proves again that it is a metric on $[0,\,1]$. Their Theorem 2 gives a second metric on $[0,\,1]$, which divides by $\max(H(X_{i}),\, H(X_{j}))$ instead. This algorithm does not compute that second metric.
Algorithm
It is the only algorithm of the family that runs a procedure. The other algorithms apply a closed form to each correlation entry. variation_info does these steps for each pair of asset columns, and bins and normalise are the two fields below.
- Take the number of histogram bins from
bins. AnIntegeris that count.HacineGharbiRaviercomputes the count from the sample size and the correlation of the pair. A bin-width rule divides the range of each column by its bin width, and takes the larger of the two counts. - Fit a histogram to each of the two columns over that bin count, and take its Shannon entropy. The results are $H(X_{i})$ and $H(X_{j})$.
- Fit the joint histogram of the two columns over the same bin edges.
- Compute the mutual information $I(X_{i};X_{j})$ from the joint histogram and its two marginal sums.
- Compute the variation of information of the first formula above.
- When
normaliseistrue, divide by the joint entropy $H(X_{i}) + H(X_{j}) - I(X_{i};X_{j})$ to get the second formula. - Floor the result at zero. The diagonal is exactly zero, and the procedure does not estimate it.
Fields
bins: Binning algorithm or fixed number of bins.
normalise: Iftrue, the result is the normalised mutual information or the normalised variation of information.
Constructors
VariationInfoDistance(; bins::Int_Bin = HacineGharbiRavier(), normalise::Bool = true) -> VariationInfoDistanceKeywords correspond to the struct's fields.
Validation
- If
binsis an integer,0 < bins <= RESOURCE_LIMITS[].max_bins. The joint histogram isbins × bins, andRESOURCE_LIMITSsets the cap.
Examples
julia> VariationInfoDistance()VariationInfoDistance bins ┼ HacineGharbiRavier() normalise ┴ Bool: trueRelated
References
- [51] M. Meilă. Comparing clusterings—an information based distance. Journal of Multivariate Analysis 98, 873–895 (2007). Section 3, Equation 19.
- [113] C. Rajski. A metric space of discrete probability distributions. Information and Control 4, 371–377 (1961). Theorem, Equation 1.
- [114] A. Kraskov, H. Stögbauer, R. G. Andrzejak and P. Grassberger. Hierarchical clustering using mutual information. Europhysics Letters 70, 278–284 (2005). Section 2.3, Theorems 1 and 2.
PortfolioOptimisers.CanonicalDistance — Type
struct CanonicalDistance <: AbstractDistanceAlgorithmSelects the distance algorithm that matches the covariance estimator it is given.
It has no formula. It is a redirect that sends each codependence measure to the distance transform that fits its range. A signed correlation needs the factor $1/2$. A mutual information has no correlation to transform. A tail dependence coefficient needs an unbounded distance.
| Covariance estimator | Distance algorithm |
|---|---|
MutualInfoCovariance | VariationInfoDistance |
LowerTailDependenceCovariance | LogDistance |
DistanceCovariance | CorrelationDistance |
StatsBase.CovarianceEstimator | SimpleDistance |
The table also applies to a PortfolioOptimisersCovariance whose ce is one of these estimators.
With a correlation or a covariance matrix and no estimator, it uses SimpleDistance.
Related
References
- [12]
- D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
- [26]
- M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020).
- [38]
- G. De Luca and P. Zuccolotto. A tail dependence-based dissimilarity measure for financial time series clustering. Advances in Data Analysis and Classification 5, 323–340 (2011).
- [51]
- M. Meilă. Comparing clusterings—an information based distance. Journal of Multivariate Analysis 98, 873–895 (2007).
- [111]
- S. Van Dongen and A. J. Enright. Metric distances derived from cosine similarity and Pearson and Spearman correlations, arXiv preprint arXiv:1208.3145 (2012).
- [112]
- M. López de Prado. Building diversified portfolios that outperform out of sample. The Journal of Portfolio Management 42, 59–69 (2016).
- [113]
- C. Rajski. A metric space of discrete probability distributions. Information and Control 4, 371–377 (1961).
- [114]
- A. Kraskov, H. Stögbauer, R. G. Andrzejak and P. Grassberger. Hierarchical clustering using mutual information. Europhysics Letters 70, 278–284 (2005).