Similarity Matrices: private API

PortfolioOptimisers.AbstractSimilarityMatrixAlgorithm — Type

abstract type AbstractSimilarityMatrixAlgorithm <: AbstractAlgorithm

Abstract supertype for all similarity matrix algorithms.

Every subtype defines a pure transformation of a distance matrix into a similarity matrix, applied by distance_to_similarity. The family is consumed in two places: the Planar Maximally Filtered Graph (PMFG) construction, which the similarity branch of NetworkEstimator and both DBHT and LoGo all reach, and the similarity slot returned by cor_and_dist.

The family is open: an extension defines a member and a distance_to_similarity method for it.

The first consumer needs a similarity that cannot go below zero, so all three of the fields feeding it — NetworkEstimator's alg, DBHT's sim and LoGo's sim — take the narrower AbstractNonNegativeSimilarityMatrixAlgorithm rather than this type. The second takes any member: FeatureDistance's sim keeps all five, and never reaches a PMFG, because every PMFG entry point recomputes the similarity from its own field and discards the one cor_and_dist returned.

Related

source

PortfolioOptimisers.AbstractNonNegativeSimilarityMatrixAlgorithm — Type

abstract type AbstractNonNegativeSimilarityMatrixAlgorithm <: AbstractSimilarityMatrixAlgorithm

Abstract supertype for the similarity matrix algorithms admitted on the PMFG path.

A member of this family promises that distance_to_similarity returns no negative entry for any finite distance matrix in the member's own domain. Tree_SimMat, DBHT's sim and LoGo's sim are bounded by this type, so a similarity that can go negative fails at construction rather than inside PMFG_T2s.

Where the requirement comes from

Not from PMFG_T2s, which runs on signed input: its gain argmax compares sums of exactly three weights, and its seed mask is shift-invariant. The requirement comes from downstream DBHT. DirectHb compares unnormalised weight sums over different-sized bubbles and writes the winner into Hc, where a separating bubble is detected by an exact zero row sum — so cancelling signs manufacture one. BubbleMember divides by a signed total that can reach zero, Inf or a negative, in which case argmax picks the worst bubble. Both failures are silent: the caller gets wrong clusters and no error.

The type bound is deliberately wider than that provenance. calc_adjacency binarises the PMFG, clusterise takes matrix powers of it, and logo! reads structure only, so none of them reaches the bubble machinery. They are bound anyway, because relaxing the one guard that presently makes them safe is a change with its own justification to make.

Non-negative, not positive

exp(-Inf) is 0 exactly, and that route is live — LogDistance maps an exactly zero correlation to an infinite distance. A zero similarity is admissible; a negative one is not.

The guarantee is per member, and it carries a domain

The type says which algorithm. It cannot say which data, so a member whose guarantee needs a precondition on D declares it through assert_similarity_domain, which the PMFG entry points call before they transform.

Member	Holds when
`ExponentialSimilarity`	always
`GeneralExponentialSimilarity`	always
`MaximumDistanceSimilarity`	`D` is finite
`ComplementSimilarity`	`all(D .<= 1)`

AngularSimilarity is excluded permanently, and not for want of a precondition: it is negative wherever the correlation is negative, which is ordinary data.

Open by declaration, not by proof

The family is open, and membership is a claim a subtype makes rather than one the library verifies. A probe cannot check it: the contract quantifies over every admissible distance matrix, so a probe that passes ComplementSimilarity at D = 0.5 still misses its failure at D = 7. PMFG_T2s's own non-negativity check is therefore kept as the backstop against an extension that claims membership and does not keep it.

Related

source

PortfolioOptimisers.distance_to_similarity — Function

distance_to_similarity(se::AbstractSimilarityMatrixAlgorithm; D::MatNum, kwargs...)

Compute a similarity matrix from a distance matrix using the specified similarity algorithm.

Algorithm

Select the method by the type of se. Every member of AbstractSimilarityMatrixAlgorithm owns one method, and the member's own docstring carries the closed form and the steps of its branch.
Apply that transformation to D entrywise, giving a new matrix of the same size. D is never written to, and no method reads a keyword.

The function is a pure transformation with no domain of its own. It never calls assert_similarity_domain, so a member whose non-negativity needs a precondition on D is checked by the caller, at the five PMFG entry points, and not here.

Arguments

se: Similarity matrix algorithm.
- se::MaximumDistanceSimilarity: Uses the maximum distance transformation.
- se::ExponentialSimilarity: Uses the exponential transformation.
- se::GeneralExponentialSimilarity: Uses a generalised exponential transformation.
- se::ComplementSimilarity: Uses the linear complement of the distance.
- se::AngularSimilarity: Inverts a normalised angular distance.
D: Distance matrix.
kwargs...: Additional keyword arguments (not used).

Returns

S::Matrix{<:Number}: Similarity matrix of the same size as D.

Examples

julia> D = [0.0 0.25 0.5; 0.25 0.0 1.0; 0.5 1.0 0.0];julia> PortfolioOptimisers.distance_to_similarity(ComplementSimilarity(); D = D)3×3 Matrix{Float64}: 1.0   0.75  0.5 0.75  1.0   0.0 0.5   0.0   1.0julia> D = [0.0 1/3 2/3; 1/3 0.0 1.0; 2/3 1.0 0.0];julia> PortfolioOptimisers.distance_to_similarity(AngularSimilarity(); D = D)3×3 Matrix{Float64}:  1.0   0.5  -0.5  0.5   1.0  -1.0 -0.5  -1.0   1.0

Related

source

PortfolioOptimisers.assert_similarity_domain — Function

assert_similarity_domain(sim::AbstractSimilarityMatrixAlgorithm,
                         de::AbstractDistanceEstimator, D::MatNum)
assert_similarity_domain(sim::ComplementSimilarity, de::AbstractDistanceEstimator,
                         D::MatNum)
assert_similarity_domain(sim::MaximumDistanceSimilarity,
                         de::AbstractDistanceEstimator, D::MatNum)

Assert that D lies in sim's domain, so that sim's non-negativity guarantee holds on it.

The other half of AbstractNonNegativeSimilarityMatrixAlgorithm's enforcement. The type bound says which algorithm may reach the PMFG path; it cannot say which data, and two of the four admitted members are non-negative only over part of the distances a caller can produce.

Member	Precondition
`ComplementSimilarity`	`all(D .<= 1)`
`MaximumDistanceSimilarity`	`all(isfinite, D)`
every other member	none — the fallback is a no-op

Interface-scoped, not member-wide

The five call sites are the PMFG entry points, and nothing else: calc_weighted_adjacency_graph, calc_distance_weighted_graph, both clusterise methods that build a PMFG, and logo!. It is deliberately not called inside distance_to_similarity, which stays a pure transformation with no domain of its own.

The scope follows the failure, which is path-dependent rather than member-wide. ComplementSimilarity against an unbounded distance is documented, tested behaviour on the FeatureDistance path — every Distances.SemiMetric yields a similarity there and nothing throws — and it is only on the PMFG path that the same value is unusable.

What it costs a caller

Nothing that works today. Every pairing this refuses already throws, at PMFG_T2s's own check and one transformation later, with a message that names W rather than the configuration that produced it. The gain is the message: it names both halves, which is why de is passed in — the distance estimator that produced the offending value, and the similarity that refused it.

Algorithm

Select the method by the type of sim. The two members of the table above own a method each; every other member reaches the method of AbstractSimilarityMatrixAlgorithm, which is a no-op and returns immediately.
Test that member's precondition over the whole of D.
Raise a DomainError when the test fails. The message carries maximum(D), the name of the similarity that refused it, that similarity's closed form, and the concrete type of de. Return nothing when it holds.

Arguments

sim: Similarity matrix algorithm.
de: Distance estimator that produced D. Read only to name it in the error.
D: Distance matrix.

Validation

Under ComplementSimilarity: all(x -> x <= 1, D).
Under MaximumDistanceSimilarity: all(isfinite, D).

Returns

nothing.

Related

source

PortfolioOptimisers.default_similarity — Function

default_similarity(metric::Distances.SemiMetric)
default_similarity(metric::AngularDist)

Select the similarity matrix algorithm that is the natural counterpart of a distance metric.

Used to default the similarity field of a distance algorithm from its metric, so that the resolved value is visible on the printed object rather than hidden inside the distance kernel. The fallback is ComplementSimilarity, which is the named counterpart of every metric expressible as one minus a similarity; metrics whose inversion is not linear add their own method.

Algorithm

Select the method by the type of metric. Dispatch does the selection, so a metric whose inversion is not linear adds a method of its own and reaches it without a branch here.
Return the similarity that method names. There are two methods today:
- metric::Distances.SemiMetric is the fallback, and returns ComplementSimilarity. Every metric that is itself one minus a similarity lands here, and the complement recovers that similarity by name.
- metric::AngularDist returns AngularSimilarity, which is the exact algebraic inverse of the angular distance. That method is declared in src/06_Distance/05_FeatureDistance.jl, beside AngularDist itself, and this docstring speaks for it.

The pairing this function makes is correct because it selects the inverse of the metric it is given. A similarity chosen by hand carries no such guarantee; ComplementSimilarity and AngularSimilarity both state what a wrong pairing returns.

Arguments

metric: Distance metric.

Returns

sim::AbstractSimilarityMatrixAlgorithm: Similarity matrix algorithm.

Examples

julia> PortfolioOptimisers.default_similarity(PortfolioOptimisers.Distances.CosineDist())ComplementSimilarity()julia> PortfolioOptimisers.default_similarity(PortfolioOptimisers.AngularDist())AngularSimilarity()

Related

source