Similarity
PortfolioOptimisers.MaximumDistanceSimilarity — Type
struct MaximumDistanceSimilarity <: AbstractNonNegativeSimilarityMatrixAlgorithmSubtracts each squared distance from the largest squared distance, rounded up to an integer.
The ceiling makes the result non-negative for a distance matrix of any scale. The source states the transformation as $\rho_{i,\,j} = 1 - d_{i,\,j}^{2}$. This member gives the same result when $\max(\mathbf{D}) \leq 1$, which holds for SimpleDistance, SimpleAbsoluteDistance and CorrelationDistance. Above that bound the form of the source goes negative, and this member does not.
Mathematical definition
\[\begin{align} S_{i,\,j} &= \left\lceil\max(\mathbf{D})^2\right\rceil - D_{i,\,j}^2\,, \end{align}\]
Where:
- $S_{i,\,j}$: The similarity between assets $i$ and $j$.
- $\mathbf{D}$: The distance matrix.
- $D_{i,\,j}$: The distance between assets $i$ and $j$.
Equation 4 of Section 3 of [115] relates a similarity matrix with a unit diagonal to the distance of its points by $\Delta_{i,\,j}^{2} = 2(1 - S_{i,\,j})$, so $S_{i,\,j} = 1 - \Delta_{i,\,j}^{2} / 2$. That relation halves the squared distance, and this transformation does not. The audit of the sources found no work that states $1 - d_{i,\,j}^{2}$.
The transformation needs a finite distance matrix. At an infinite entry it computes ceil(Inf^2) - Inf^2, which is NaN, and each finite entry becomes Inf. Real data reaches this case. LogDistance maps a correlation of exactly zero to an infinite distance, and this member is the default of DBHT and LoGo. assert_similarity_domain refuses an infinite entry on the PMFG path.
Algorithm
The branch of distance_to_similarity that this tag selects runs these steps.
- Take the largest entry of
D, square that scalar, and round it up withceil. The result is the ceiling. The code squares the maximum, and does not take the maximum of the squares. The two are equal, because a distance is never negative. - Square every entry of
D. - Subtract the squared entries of step 2 from the scalar of step 1. The result is the similarity matrix. It keeps the element type of
D, so aFloat32distance matrix gives aFloat32similarity matrix.
Related
AbstractNonNegativeSimilarityMatrixAlgorithmAbstractSimilarityMatrixAlgorithmExponentialSimilarityGeneralExponentialSimilarityComplementSimilarityAngularSimilarityassert_similarity_domaindistance_to_similarity
References
- [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 13.1.4.2, footnote 7. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [115] J. C. Gower. Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika 53, 325–338 (1966). Section 3, Equation 4.
PortfolioOptimisers.ExponentialSimilarity — Type
struct ExponentialSimilarity <: AbstractNonNegativeSimilarityMatrixAlgorithmMaps a distance of any magnitude into $(0,\,1]$ by $e^{-D}$.
This member has no domain. It is non-negative at every finite distance, and it also takes an infinite distance, where exp(-Inf) is exactly 0. Use it when the distance has no upper bound, as under LogDistance, DistanceDistance or VariationInfoDistance with normalise = false. GeneralExponentialSimilarity is the other member with no domain.
Mathematical definition
\[\begin{align} S_{i,\,j} &= e^{-D_{i,\,j}}\,, \end{align}\]
Where:
- $S_{i,\,j}$: The similarity between assets $i$ and $j$.
- $\mathbf{D}$: The distance matrix.
- $D_{i,\,j}$: The distance between assets $i$ and $j$.
Algorithm
The branch of distance_to_similarity that this tag selects runs these steps.
- Negate every entry of
D. - Take the exponential of each negated entry. The result is the similarity matrix. It keeps the element type of
D, so aFloat32distance matrix gives aFloat32similarity matrix.
Related
PortfolioOptimisers.GeneralExponentialSimilarity — Type
struct GeneralExponentialSimilarity{__T_coef, __T_power} <: AbstractNonNegativeSimilarityMatrixAlgorithmApplies $e^{-c D^{p}}$, the exponential transformation with a scale and an exponent.
coef sets the rate at which the similarity decreases with the distance, and power sets the shape of that decrease. A larger coef lowers the similarity at each positive distance. A larger power moves the similarity towards 1 below D = 1, and towards 0 above D = 1. Like ExponentialSimilarity, this member has no domain, and at coef = 1 and power = 1 the two members give the same result.
Mathematical definition
\[\begin{align} S_{i,\,j} &= e^{-c \cdot D_{i,\,j}^p}\,, \end{align}\]
Where:
- $S_{i,\,j}$: The similarity between assets $i$ and $j$.
- $\mathbf{D}$: The distance matrix.
- $D_{i,\,j}$: The distance between assets $i$ and $j$.
- $c$: The scale factor, the field
coef. - $p$: The exponent, the field
power.
Algorithm
The branch of distance_to_similarity that this tag selects runs these steps.
- Read
powerandcoeffrom the tag. - Raise every entry of
Dtopower. - Multiply the result of step 2 by
-coef.coefis a scalar, so one product scales the whole matrix. - Take the exponential of each entry. The result is the similarity matrix. It keeps the element type of
Dwhen neither field has a wider type. TheIntegerdefaults are not wider, so aFloat32distance matrix gives aFloat32similarity matrix.
Fields
coef: The scale factor of the distance term in the similarity transformation.
power: The exponent of each distance in the similarity transformation.
Constructors
GeneralExponentialSimilarity(; coef::Number = 1, power::Number = 1) -> GeneralExponentialSimilarityKeywords correspond to the struct's fields.
Validation
isfinite(coef) && coef > 0.power > 0.
Examples
julia> GeneralExponentialSimilarity()GeneralExponentialSimilarity coef ┼ Int64: 1 power ┴ Int64: 1Related
PortfolioOptimisers.ComplementSimilarity — Type
struct ComplementSimilarity <: AbstractNonNegativeSimilarityMatrixAlgorithmTakes the linear complement $1 - D$, the exact inverse of a metric that is one minus a similarity.
For many metrics it recovers a similarity with a name, not a monotone function of one, so default_similarity uses it as the fallback. It is the one member that has a domain and also needs the correct metric. The two warnings below are about these two conditions.
Mathematical definition
\[\begin{align} S_{i,\,j} &= 1 - D_{i,\,j}\,, \end{align}\]
Where:
- $S_{i,\,j}$: The similarity between assets $i$ and $j$.
- $\mathbf{D}$: The distance matrix.
- $D_{i,\,j}$: The distance between assets $i$ and $j$.
It recovers the named similarity of each distance that is one minus a similarity. Against Distances.CosineDist it returns the cosine similarity, against Distances.Jaccard the Ruzicka similarity, against Distances.BrayCurtis the Sørensen-Dice similarity, and against Distances.CorrDist the Pearson correlation.
The result is a non-negative similarity only when $\mathbf{D} \in [0,\,1]$. Above 1 the similarity is negative.
The result of a negative value depends on the path. The FeatureDistance path keeps the value, and plot_clusters colours it on the signed scale $[-1,\,1]$. A distance above 2 gives a similarity below -1, and that scale clips it without a warning. On the PMFG path, assert_similarity_domain refuses the same input, because the consumers of PMFG_T2s cannot take a negative weight. On each path the similarity stays symmetric, with a unit diagonal.
The rule is D <= 1. It is not a rule against a metric without an upper bound. Distances.CosineDist and Distances.CorrDist have the upper bound 2, not 1, and the check refuses them at each entry above 1. Distances.CorrDist goes above 1 at each negative correlation.
In the library, LogDistance, DistanceDistance and VariationInfoDistance with normalise = false can go above 1. Under its default Distances.Euclidean, DistanceDistance can go above 1 also when the base distance is in [0, 1]. For a member with no domain, use ExponentialSimilarity or GeneralExponentialSimilarity.
This member is the inverse of the metrics listed above, and of no other metric. With a different distance it returns a number that is in the domain, non-negative and wrong, and no path detects the error.
SimpleDistance is $\sqrt{(1 - \rho) / 2}$. A correlation of 0.003 gives D = 0.706, which is a legal bounded distance, and this member reports a similarity of 0.29. No check can detect that error. On the FeatureDistance path, default_similarity pairs a metric with its inverse. The fields alg of NetworkEstimator, sim of DBHT and sim of LoGo take a member without a reference to the distance estimator of $\mathbf{D}$. On those paths, the caller must pair the two correctly.
Algorithm
The branch of distance_to_similarity that this tag selects runs these steps.
- Make the unit in the element type of
Dwithone(eltype(D)). That unit keeps the element type ofD, so aFloat32distance matrix gives aFloat32similarity matrix. - Subtract every entry of
Dfrom that unit. The result is the similarity matrix.
Related
PortfolioOptimisers.AngularSimilarity — Type
struct AngularSimilarity <: AbstractSimilarityMatrixAlgorithmRecovers a correlation from a normalised angular distance by $\cos(\pi D)$.
It returns the correlation itself, not a monotone function of it. It is the one member that goes negative at a distance in [0, 1], in this case above D = 0.5. Both facts have one cause. It is the exact algebraic inverse of AngularDist, and a correlation can be negative.
Mathematical definition
\[\begin{align} S_{i,\,j} &= \cos\left(\pi D_{i,\,j}\right)\,, \end{align}\]
Where:
- $S_{i,\,j}$: The similarity between assets $i$ and $j$.
- $\mathbf{D}$: The distance matrix.
- $D_{i,\,j}$: The distance between assets $i$ and $j$.
For an angular distance $D_{i,\,j} = \arccos(\rho_{i,\,j}) / \pi$ this member recovers $\rho_{i,\,j}$ exactly, without the data of the distance. No source states the inverse, because it is algebra. The references below are the sources of the angular distance, which AngularDist also names. Against AngularDist, the recovered cosine and the cosine of the features agree to floating-point precision. The member maps $[0,\,1] \to [1,\,-1]$, so the similarity is bounded, and its diagonal is 1 when the diagonal of the distance matrix is zero.
Where this member is correct, and where it is refused
It is correct on the FeatureDistance path, which is the one path that pairs a metric with its inverse. On that path, default_similarity selects this member for AngularDist, and the recovered $\rho$ is exact.
It is not a member of AbstractNonNegativeSimilarityMatrixAlgorithm, so alg of NetworkEstimator, sim of DBHT and sim of LoGo refuse it at construction. No domain precondition can admit it. With the correct metric, this member is negative at each $\rho_{i,\,j} < 0$, which is ordinary data, and the consumers of PMFG_T2s cannot take a negative weight.
With a distance that is not angular, this member returns a number that is not a correlation. SimpleDistance is $\sqrt{(1 - \rho) / 2}$, and its range is $[0,\,1]$, the same range as the domain of this member. No check can find the error. A correlation of 0.003 gives D = 0.706, and this member reports -0.603, which is not a weak negative correlation but a wrong number.
Algorithm
The branch of distance_to_similarity that this tag selects runs these steps.
- Multiply every entry of
Dbypi.piis anIrrational, which takes the element type of the number that it multiplies, so aFloat32distance matrix staysFloat32. - Take the cosine of each scaled entry. The result is the similarity matrix.
The sign comes from step 2 alone. $\cos(\pi D)$ is positive below $D = 0.5$, zero at $D = 0.5$ and negative above it. Thus AbstractNonNegativeSimilarityMatrixAlgorithm does not admit this member.
Related
AbstractSimilarityMatrixAlgorithmAbstractNonNegativeSimilarityMatrixAlgorithmAngularDistMaximumDistanceSimilarityExponentialSimilarityGeneralExponentialSimilarityComplementSimilaritydistance_to_similaritydefault_similarity
References
- [111] S. Van Dongen and A. J. Enright. Metric distances derived from cosine similarity and Pearson and Spearman correlations. arXiv preprint arXiv:1208.3145 (2012). Section 4, Lemma 3.
- [116] M. S. Charikar. Similarity estimation techniques from rounding algorithms. In: Proceedings of the Thirty-Fourth Annual ACM Symposium on Theory of Computing (2002); pp. 380–388. Section 1.
References
- [12]
- D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
- [111]
- S. Van Dongen and A. J. Enright. Metric distances derived from cosine similarity and Pearson and Spearman correlations, arXiv preprint arXiv:1208.3145 (2012).
- [115]
- J. C. Gower. Some distance properties of latent root and vector methods used in multivariate analysis. Biometrika 53, 325–338 (1966).
- [116]
- M. S. Charikar. Similarity estimation techniques from rounding algorithms. In: Proceedings of the Thirty-Fourth Annual ACM Symposium on Theory of Computing (2002); pp. 380–388.