Distance
PortfolioOptimisers.Distance — Type
struct Distance{__T_power, __T_alg} <: AbstractDistanceEstimatorPairs a distance algorithm with an optional integer power, and applies it to a correlation matrix or to the data.
Every clustering, network and phylogeny routine uses this estimator to get a distance matrix. alg selects the transform. power raises the input of that transform to the integer power $p$.
power = 1 gives the base distance exactly, for every algorithm. Only $p \geq 2$ changes the result. power = nothing and power = 1 select different methods, so the base case never raises a matrix to a power.
Mathematical definition
The four correlation-based algorithms of RhoDistanceAlgorithm raise the correlation to $p$ inside their base formula.
\[\begin{align} _{g}d_{i,\,j}^{\mathrm{S}} &= \sqrt{\mathrm{clamp}\left(s\left(1 - \rho_{i,\,j}^{p}\right),\, 0,\, 1\right)}\\ _{g}d_{i,\,j}^{\mathrm{SA}} &= \sqrt{\mathrm{clamp}\left(1 - \lvert\rho_{i,\,j}\rvert^{p},\, 0,\, 1\right)}\\ _{g}d_{i,\,j}^{\mathrm{L}} &= \max\left(-\log{\lvert\rho_{i,\,j}\rvert^{p}},\, 0\right)\\ _{g}d_{i,\,j}^{\mathrm{C}} &= \sqrt{\mathrm{clamp}\left(1 - \rho_{i,\,j}^{p},\, 0,\, 1\right)}\\ s &= \begin{cases} 1/2 & \text{if } p \mod 2 \neq 0\\ 1 & \text{otherwise} \end{cases}\,, \end{align}\]
VariationInfoDistance has no correlation to raise, so it raises the distance itself.
\[\begin{align} _{g}d_{i,\,j}^{\mathrm{VI}} &= \left(d_{i,\,j}^{\mathrm{VI}}\right)^{p}\,, \end{align}\]
Where:
- $_{g}d_{i,\,j}$: Generalised distance between assets $i$ and $j$. The superscript names the algorithm: S for
SimpleDistance, SA forSimpleAbsoluteDistance, L forLogDistance, C forCorrelationDistanceand VI forVariationInfoDistance. - $d_{i,\,j}$: Base distance of the algorithm.
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
- $p$: Integer power.
- $s$: Scale factor of
SimpleDistanceonly. $s = 1/2$ for an odd $p$, and $s = 1$ for an even $p$.
The two values of $s$ come from the range of $\rho_{i,\,j}^{p}$. $s$ keeps the radicand inside $[0,\,1]$, so $s = 1 / (1 - m)$, where $m$ is the smallest value that $\rho_{i,\,j}^{p}$ can take. An odd $p$ keeps the sign, so $m = -1$ and $s = 1/2$. At an even $p$, $\rho_{i,\,j}^{p}$ cannot be negative, so $m = 0$ and $s = 1$.
The clamp has no effect on the first two algorithms at any $p$, because $s(1 - \rho_{i,\,j}^{p})$ and $1 - \lvert\rho_{i,\,j}\rvert^{p}$ never leave $[0,\,1]$. It binds for CorrelationDistance at every odd $p$. There, $\rho_{i,\,j}^{p}$ keeps its sign and the radicand runs over $[0,\,2]$. The docstring of CorrelationDistance shows the truncation.
CanonicalDistance is a redirect and has no formula. It gives power to the algorithm that it selects.
power is one field with two meanings. For a correlation-based algorithm, it raises the correlation inside the transform. For VariationInfoDistance, it raises the distance that the algorithm returns, because there is no correlation. So at $p = 2$, $_{g}d^{\mathrm{S}}$ is not $\left(d^{\mathrm{S}}\right)^{2}$, but $_{g}d^{\mathrm{VI}}$ is $\left(d^{\mathrm{VI}}\right)^{2}$.
The docstring of each algorithm states its base formula and its original source. The $p$ generalisation and the scale factor $s$ are this library's own extension of those formulas.
Fields
power: Optional integer exponent, which the distance algorithm applies entry by entry.Distancestates which quantity it raises.nothingand1both give the base distance, so onlypower >= 2changes the result.
alg: The distance algorithm.
Constructors
Distance(; power::Option{<:Integer} = nothing, alg::AbstractDistanceAlgorithm = SimpleDistance()) -> DistanceKeywords correspond to the struct's fields.
CanonicalDistance selects an algorithm from the covariance estimator. It uses SimpleDistance for every estimator outside its table. A bare Distance() also works at the matrix entry point distance(de, rho), which has no estimator to select from. So the two defaults agree, except on the estimators in the table of CanonicalDistance. A library entry point that always has a covariance estimator uses Distance(; alg = CanonicalDistance()) as its default, so that each estimator in the table gets its own algorithm.
Validation
- If
poweris notnothing,power >= 1.
Examples
julia> Distance()Distance power ┼ nothing alg ┴ SimpleDistance()Related
PortfolioOptimisers.distance — Function
distance(de::Distance{<:Any,
<:Union{SimpleDistance, SimpleAbsoluteDistance, LogDistance,
CorrelationDistance, CanonicalDistance}},
ce::StatsBase.CovarianceEstimator, X::MatNum; dims::Int = 1, kwargs...)Compute the correlation matrix with ce and X, and transform it into a distance matrix with the algorithm de names.
This is the data entry point of the correlation-based family. cor_and_dist does the same computation and also returns the correlation, so a caller that needs both matrices computes the correlation once.
Algorithm
- Compute the correlation matrix from
ceandXwithStatistics.cor, alongdims. The method giveskwargstoStatistics.cor, andStatistics.correfuses adimsoutside $(1,\, 2)$. - Transform that correlation matrix with
_dist_from_cor, underde.algandde.power.
A CanonicalDistance de does one step first. It rebuilds de with SimpleDistance and keeps de.power, because the last row of the redirect table gives that algorithm to a plain StatsBase.CovarianceEstimator. The estimators of the other rows have their own methods, and the table on CanonicalDistance lists them.
Arguments
de: Distance estimator.de::Distance{<:Any, <:SimpleDistance}: Use theSimpleDistancealgorithm.de::Distance{<:Any, <:SimpleAbsoluteDistance}: Use theSimpleAbsoluteDistancealgorithm.de::Distance{<:Any, <:LogDistance}: Use theLogDistancealgorithm.de::Distance{<:Any, <:CorrelationDistance}: Use theCorrelationDistancealgorithm.de::Distance{<:Any, <:CanonicalDistance}: Use theCanonicalDistancealgorithm.
ce: Covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments that the method gives toStatistics.cor.
Validation
dims in (1, 2). This method does not do the check.Statistics.corraises theDomainError.
Returns
D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
distance(de::Distance{Nothing, <:VariationInfoDistance}, ::Any, X::MatNum;
dims::Int = 1, kwargs...)
distance(de::Distance{<:Integer, <:VariationInfoDistance}, ::Any, X::MatNum;
dims::Int = 1, kwargs...)Compute the variation of information distance matrix from the data matrix alone.
It is the one algorithm of the family that reads X and not a correlation matrix. So the covariance estimator is a placeholder, and the method ignores it.
The two signatures above are the two power cases. For a correlation-based algorithm, power raises the correlation inside the transform. Here there is no correlation, so power raises the distance that the algorithm returns. The result is variation_info(...) .^ de.power.
Algorithm
- Orient
Xwithdims_oriented, which transposes it whendimsis2and refuses any other value. - Read
de.alg.binsandde.alg.normaliseoff the algorithm, and pass both tovariation_info, which builds the joint histograms and forms the distance.VariationInfoDistancestates those steps. - When
de.poweris anInteger, raise the distance matrix of step 2 to it entry by entry. Whende.powerisnothing, return that matrix unchanged.
Arguments
de: Distance estimator that holds theVariationInfoDistancealgorithm.::Any: Placeholder for the covariance estimator, so that this method takes the same call as the other methods. The method ignores it.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments that the method ignores.binsandnormalisecome fromde.alg, never from a keyword.
Validation
dims in (1, 2). This method does not do the check.dims_orientedraises theDomainError.
Returns
D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
DistanceVariationInfoDistancevariation_infoMutualInfoCovarianceCanonicalDistance: copiesbinsandnormalisefrom aMutualInfoCovariancetode.alg.cor_and_dist
distance(::Distance{<:Any,
<:Union{SimpleDistance, SimpleAbsoluteDistance, LogDistance,
CorrelationDistance, CanonicalDistance}},
rho::MatNum, args...; kwargs...)Compute the distance matrix from a correlation matrix, or from a covariance matrix.
This is the matrix entry point of the correlation-based family, for a caller that already has the matrix. The diagonal tells a correlation matrix from a covariance matrix, as _as_correlation states. The result is the distance that the algorithm defines. It is not the Euclidean distance between the columns of the data. SimpleDistance and SimpleAbsoluteDistance return an angular distance, and LogDistance returns an unbounded dissimilarity.
Algorithm
- Convert
rhoto a correlation matrix with_as_correlation, which also checks that it is square. - Transform that correlation matrix with
_dist_from_cor, underde.algandde.power.
A CanonicalDistance de does one step first. It rebuilds de with SimpleDistance and keeps de.power. Here there is no covariance estimator to select from, so the method uses the last row of the redirect table.
Arguments
de: Distance estimator.de::Distance{<:Any, <:SimpleDistance}: Use theSimpleDistancealgorithm.de::Distance{<:Any, <:SimpleAbsoluteDistance}: Use theSimpleAbsoluteDistancealgorithm.de::Distance{<:Any, <:LogDistance}: Use theLogDistancealgorithm.de::Distance{<:Any, <:CorrelationDistance}: Use theCorrelationDistancealgorithm.de::Distance{<:Any, <:CanonicalDistance}: Use theCanonicalDistancealgorithm.
rho: Correlation or covariance matrix._as_correlationconverts a covariance matrix into a new matrix withStatsBase.cov2cor, and does not write into the argument.args...: Arguments that the method ignores. They let this method and thece-and-Xmethod above take the same call.kwargs...: Keyword arguments that the method ignores. They let this method and thece-and-Xmethod above take the same call.
Validation
rhois square.
Returns
D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
distance(de::Distance{<:Any, <:CanonicalDistance},
ce::Union{MutualInfoCovariance,
AllInternalMutualInfoCov,
LTDCov_AllInternalLTDCov,
DistCov_AllInternalDistCov},
X::MatNum; dims::Int = 1, kwargs...)Rebuild de with the distance algorithm that fits the range of the covariance estimator, and call distance again.
The redirect has no formula. It sends each codependence measure to the transform that fits its range. A signed correlation needs the factor $1/2$. A mutual information has no correlation to transform. A tail dependence coefficient needs an unbounded distance. The selected algorithm gets de.power unchanged.
| Covariance estimator | Algorithm selected | Read from ce |
|---|---|---|
MutualInfoCovariance | VariationInfoDistance | ce.bins, ce.normalise |
PortfolioOptimisersCovariance that wraps it | VariationInfoDistance | ce.ce.bins, ce.ce.normalise |
LowerTailDependenceCovariance, wrapped or not | LogDistance | nothing |
DistanceCovariance, wrapped or not | CorrelationDistance | nothing |
any other StatsBase.CovarianceEstimator | SimpleDistance | nothing |
The last row is the fallback. On every estimator of that row, the redirect and a bare Distance with SimpleDistance give the same result.
Algorithm
- Select the row of the table above by the type of
ce. Dispatch does the selection, so a wrapper reaches the same row as the estimator that it wraps. - Build a new
Distancewithde.powerand the algorithm of that row. - On the two mutual information rows, copy
binsandnormalisefromceto the newVariationInfoDistance. For the wrapper, read them fromce.ce. The signature does not show this step, but the result depends on it. AMutualInfoCovariancewith a non-defaultbinsgives a different distance matrix from one with the defaultbins. - Call
distancewith the rebuilt estimator, the samece, and the sameX,dimsandkwargs.
Arguments
de: Distance estimator that holds theCanonicalDistancealgorithm.ce: Covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments that the method gives to thedistancemethod of the selected algorithm.
Returns
D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
DistanceCanonicalDistanceMutualInfoCovarianceLowerTailDependenceCovarianceDistanceCovariancePortfolioOptimisersCovariancecor_and_dist: applies the same table, with the same five rows.
distance(de::DistanceDistance, ce::StatsBase.CovarianceEstimator, X::MatNum;
dims::Int = 1, kwargs...)Compute the distance-of-distances matrix from a covariance estimator and a data matrix.
Algorithm
- Build a base
Distancewithde.powerandde.alg. Calldistanceon it withce,X,dimsandkwargs. The result is the base distance matrixD. - Apply
de.metrictoDwithDistances.pairwise, and give it the fieldsde.argsandde.kwargs. The result is the distance-of-distances matrix.
When alg is a CanonicalDistance, step 1 selects the algorithm of D by the type of ce. The metric runs after that selection. distance holds the redirect table.
Distances.pairwise reads each column of D as one observation. The call passes dims = 2 before the splat of de.kwargs, so a dims in de.kwargs replaces it. The axis does not change the result, because a base distance matrix is symmetric. The axis matters only when de.args holds a second matrix of a different shape.
Arguments
de: The distance-of-distances estimator.ce: The covariance estimator.X: The data matrix, with one row per observation and one column per asset whendims = 1.dims: Dimension along which the computation runs.kwargs...: More keyword arguments for the base distance of step 1. They do not reachDistances.pairwise, which reads the fieldde.kwargs.
Returns
D::Matrix{<:Number}: The matrix of the pairwise distances of distances.
Related
distance(de::DistanceDistance, rho::MatNum, args...; kwargs...)Compute the distance-of-distances matrix from a correlation or covariance matrix.
Algorithm
- Build a base
Distancewithde.powerandde.alg. Calldistanceon it withrho,argsandkwargs. The result is the base distance matrixD. - Apply
de.metrictoDwithDistances.pairwise, and give it the fieldsde.argsandde.kwargs. The result is the distance-of-distances matrix.
When alg is a CanonicalDistance, the base distance replaces it with SimpleDistance before step 1. This route has no covariance estimator, so the redirect table cannot select a row, and the base distance takes the fallback of the table.
Distances.pairwise reads the columns of D, as it does on the route of the covariance estimator above. The call passes dims = 2, and a dims in de.kwargs replaces it.
Arguments
de: The distance-of-distances estimator.rho: The correlation or covariance matrix.args...: More positional arguments. The method gives them to the base distance of step 1, which ignores them. They are not the fieldde.args, which goes toDistances.pairwise.kwargs...: More keyword arguments for the base distance of step 1. They do not reachDistances.pairwise, which reads the fieldde.kwargs.
Returns
D::Matrix{<:Number}: The matrix of the pairwise distances of distances.
Related
distance(de::FeatureDistance, Z::MatNum; dims::Int = 1, amsk = nothing, nx = nothing,
kwargs...)
distance(de::FeatureDistance, Z::Arr3Num; dims::Int = 1, amsk = nothing, nx = nothing,
kwargs...)Compute the distance matrix from a feature matrix.
The 2-D method collapses nothing. A static feature matrix has no observation axis, so the collapse algorithm reads its one row. Its amsk is 1 × assets. The mask marks the assets whose every value column holds data, and the method refuses an asset that the mask marks false. The 3-D method dispatches on the collapse algorithm. An asset whose feature vector is all zero takes the convention that patch_zero_feature_vectors! states.
The 3-D method reads the mask amsk of the window. A collapse then reads each asset, or each pair of assets, at its own active rows, as each member of AbstractFeatureCollapseAlgorithm states. A window with no inactive cell gives the same result as no mask. The checks read the same cells as the collapse, so the value of a cell that amsk marks false changes neither the distance nor a refusal.
Algorithm
- Check
dimswithassert_dims. - On the 2-D method, check the one-row mask with
static_windowandassert_feature_readable. ValidateZwithassert_feature_matrix, and passde.metricandZto the kernel. - On the 3-D method, check the mask with
active_window, and check that the collapse can read each asset withassert_feature_readable. - With no inactive cell, validate
Zwithassert_feature_matrix. PassdeandZtofeature_distance, which selects the method thatde.algnames. - Otherwise, put a zero at each inactive cell of a copy of
Zwithblank_unreadable. Validate the copy withassert_feature_matrix, and pass it toactive_feature_distance.
Arguments
de: Feature distance estimator.Z: Feature matrix,assets × featureswhendims = 1andfeatures × assetswhendims = 2. It can also be a 3-D array of time-varying features, whose last axis is always the observation axis:assets × features × observationswhendims = 1, andfeatures × assets × observationswhendims = 2.dims: Dimension along which the computation runs.amsk: The mask of readable cells of the window,observations × assetson the 3-D method and1 × assetson the 2-D method, ornothingwhen every cell is readable.nx: The asset names that a refusal quotes, ornothingto quote positions.kwargs...: Additional keyword arguments (ignored).
Validation
dims in (1, 2).amskisobservations × assetson the 3-D method and1 × assetson the 2-D method, and the collapse can read each asset. Seeassert_feature_readable.!isempty(Z).- Each cell of
Zthatamskmarks readable is finite and lies in the domain ofde.metric. Seeassert_metric_domain. The check reads no cell thatamskmarksfalse. - On the 3-D method, a pair of assets with no shared active row takes the rule of the
pairfield ofAggregateDistancesandStackObservations.
Returns
D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Examples
julia> Z = [1.0 0.0; 0.0 1.0; 1.0 1.0];julia> distance(FeatureDistance(), Z)3×3 Matrix{Float64}: 0.0 0.5 0.25 0.5 0.0 0.25 0.25 0.25 0.0Related
distance(de::FeatureDistance, ::Any, X; pr = nothing, rd = nothing, kwargs...)Compute the distance matrix of the Feature Matrix that feature_matrix stacks from pr and rd, for the clustering and network estimators.
Each consumer in the clustering and network estimators calls cor_and_dist(de, ce, X; …) or distance(de, pl, X; …), with a covariance estimator and a returns matrix. logo! passes a similarity matrix in place of the covariance estimator, so the type of the second positional argument is ::Any. FeatureDistance does not read that argument. It reads X, because a producer measures it.
The prior result and the returns data arrive as the keywords pr and rd. feature_window resolves the panel from them and from de.ape. A caller that takes a prior result passes both. Preselection passes rd alone.
This method ignores dims and calls the kernel with dims = 1. The dims of the caller describes the returns matrix X, and the stacked Feature Matrix always has the layout of dims = 1. dims has a meaning only at the entry point for a raw matrix, distance(de, Z; dims).
Algorithm
- Stack the Feature Matrix with
feature_window, which resolves the panel and cuts it tode.seland to the observation rows thatde.algreads. It also returns the mask of the readable cells of those rows. A cell is readable when the asset is active and every value column ofde.selholds data, observed or filled. - Compute the distance matrix
Dof that stack with the two-argument method atdims = 1, with that mask and the asset names fromfeature_asset_names.
Arguments
de: Feature distance estimator.- The second positional argument: ignored. It exists so that this method matches the signature that every consumer calls.
X: Returns matrix of the subproblem,observations × assets. A producer reads it.pr: Prior result or returns result. Both carry the asset returns matrixX, so either can supply it.rd: The returns result to use.kwargs...: Additional keyword arguments (ignored),dimsamong them.
Validation
- The panel resolves.
asset_panelraises anIsNothingErrorthat names the site when it does not. - Under
de.strict = true, every entry ofde.selnames a Panel Field, a level, a label or a group that the panel holds. Raises anArgumentError. Underde.strict = false, such an entry warns, and the estimator drops it. - The stacked Feature Matrix passes
assert_feature_matrixatdims = 1at its readable cells. It is not empty, and each cell that the mask marks readable is finite and lies in the domain ofde.metric. The value of an unreadable cell causes no refusal. - The collapse can read each asset at the readable rows of the window. See
assert_feature_readable. A pair with no shared readable row takes thepairrule of its collapse. A fit drops an unreadable asset at its entry withfeature_readable_mask, and this call refuses it.
Returns
D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
FeatureDistancecor_and_dist: the same stack, with the similarity matrix as well.feature_matrixfeature_labelsasset_panelphylogeny_matrix
PortfolioOptimisers.cor_and_dist — Function
cor_and_dist(de::Distance{<:Any,
<:Union{SimpleDistance, SimpleAbsoluteDistance,
LogDistance, CorrelationDistance,
VariationInfoDistance, CanonicalDistance}},
ce::StatsBase.CovarianceEstimator, X::MatNum; dims::Int = 1, kwargs...)Compute the correlation matrix and the distance matrix in one call.
It returns the same D that distance returns for the same arguments. Use it when the caller needs both matrices, because a correlation-based algorithm then computes the correlation once and not twice. The returned rho is the correlation that ce computes, with no transform. It is not the magnitude that SimpleAbsoluteDistance and LogDistance take, and de.power does not raise it.
Algorithm
de.alg selects one of three routes.
- A correlation-based algorithm computes the correlation matrix with
Statistics.cor. Then it transforms that same matrix with_dist_from_cor. On this route the correlation is computed once. VariationInfoDistancechecksdimswithassert_dims, computes the correlation matrix for the caller, and callsdistancefor the distance. The distance readsXand not the correlation, so the two computations share nothing.CanonicalDistancerebuildsdewith the algorithm that its redirect table names force. Then it enters route 1 or route 2. TheCanonicalDistancemethod ofdistancegives the table and the fields that the rebuild copies.
Arguments
de: Distance estimator.ce: Covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments that the method gives toStatistics.cor.
Validation
dims in (1, 2). Route 2 does the check withassert_dims. On route 1,Statistics.cordoes the check and raises the sameDomainError.
Returns
rho::MatNum: Correlation matrixassets x assets.D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
cor_and_dist(de::DistanceDistance, ce::StatsBase.CovarianceEstimator, X::MatNum;
dims::Int = 1, kwargs...)Compute the correlation matrix and the distance-of-distances matrix from a covariance estimator and a data matrix.
Algorithm
- Build a base
Distancewithde.powerandde.alg. Callcor_and_diston it withce,X,dimsandkwargs. One pass gives the correlation matrixrhoand the base distance matrixD. - Apply
de.metrictoDwithDistances.pairwise, and give it the fieldsde.argsandde.kwargs. The result is the distance-of-distances matrix. - Return
rhowithout a change, together with that matrix. The metric reads the distance matrix alone, so the method returns the correlation of the base estimator.
The second element is the matrix that distance returns for the same de, ce and X. A caller that needs the two matrices then computes one correlation, not two.
Arguments
de: The distance-of-distances estimator.ce: The covariance estimator.X: The data matrix, with one row per observation and one column per asset whendims = 1.dims: Dimension along which the computation runs.kwargs...: More keyword arguments for the base distance of step 1. They do not reachDistances.pairwise, which reads the fieldde.kwargs.
Returns
(rho::Matrix{<:Number}, D::Matrix{<:Number}): The tuple of the correlation matrix and the distance-of-distances matrix.
Related
cor_and_dist(de::FeatureDistance, Z::MatNum; dims::Int = 1, kwargs...)
cor_and_dist(de::FeatureDistance, Z::Arr3Num; dims::Int = 1, kwargs...)Compute the similarity and distance matrices from a feature matrix.
The similarity is distance_to_similarity(de.sim; D = D), which the call computes from the distance matrix that it returns. S and D come from one measurement, not from two independent estimates. Under AggregateDistances, the similarity comes from the aggregated distance, so the convention for a zero feature vector holds in S too, because $\mathrm{mean}(\cos(\pi D_{t})) \neq \cos(\pi\,\mathrm{mean}(D_{t}))$.
Algorithm
- Compute the distance matrix
Dwithdistance, which also validatesZanddims. - Transform
Dwithdistance_to_similarityunderde.simto getS.
Arguments
de: Feature distance estimator.Z: Feature matrix,assets × featureswhendims = 1andfeatures × assetswhendims = 2. It can also be a 3-D array of time-varying features, whose last axis is always the observation axis:assets × features × observationswhendims = 1, andfeatures × assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Additional keyword arguments (ignored).
Validation
- The checks of
distance: the mask,!isempty(Z), and each readable cell ofZis finite and lies in the domain ofde.metric. Seeassert_metric_domain.
Returns
S::Matrix{<:Number}: Similarity matrix,assets × assets.D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Examples
julia> Z = [1.0 0.0; 0.0 1.0; 1.0 1.0];julia> S, D = cor_and_dist(FeatureDistance(), Z);julia> S3×3 Matrix{Float64}: 1.0 6.12323e-17 0.707107 6.12323e-17 1.0 0.707107 0.707107 0.707107 1.0Related
cor_and_dist(de::FeatureDistance, ::Any, X; pr = nothing, rd = nothing, kwargs...)Compute the similarity and distance matrices of the Feature Matrix that feature_matrix stacks from pr and rd, for the clustering and network estimators.
clusterise and the network estimators call this method. It reads its arguments as the three-argument distance method does. It ignores the second positional argument and dims, it passes X to a producer, and it resolves the panel from pr and rd.
Algorithm
- Stack the Feature Matrix with
feature_window, which resolves the panel and cuts it tode.seland to the observation rows thatde.algreads. It also returns the mask of the readable cells of those rows. A cell is readable when the asset is active and every value column ofde.selholds data, observed or filled. - Compute the similarity matrix
Sand the distance matrixDof that stack with the two-argument method atdims = 1, with that mask and the asset names fromfeature_asset_names.
Arguments
de: Feature distance estimator.- The second positional argument: ignored. It exists so that this method matches the signature that every consumer calls.
X: Returns matrix of the subproblem,observations × assets. A producer reads it.pr: Prior result or returns result. Both carry the asset returns matrixX, so either can supply it.rd: The returns result to use.kwargs...: Additional keyword arguments (ignored),dimsamong them.
Validation
- The panel resolves.
asset_panelraises anIsNothingErrorthat names the site when it does not. - Under
de.strict = true, every entry ofde.selnames a Panel Field, a level, a label or a group that the panel holds. Raises anArgumentError. Underde.strict = false, such an entry warns, and the estimator drops it. - The stacked Feature Matrix passes
assert_feature_matrixatdims = 1at its readable cells. It is not empty, and each cell that the mask marks readable is finite and lies in the domain ofde.metric. The value of an unreadable cell causes no refusal. - The collapse can read each asset at the readable rows of the window. See
assert_feature_readable. A pair with no shared readable row takes thepairrule of its collapse. A fit drops an unreadable asset at its entry withfeature_readable_mask, and this call refuses it.
Returns
S::Matrix{<:Number}: Similarity matrix,assets × assets, derived fromDunderde.sim.D::MatNum: Distance matrixassets x assets, in the units the distance algorithm defines.
Related
FeatureDistancedistance: the same stack, without the similarity matrix.feature_matrixfeature_labelsclusterise