LoGo: private API

abstract type InverseMatrixSparsificationAlgorithm <: AbstractMatrixProcessingAlgorithm

Abstract supertype for all inverse matrix sparsification algorithms.

A member of this family imposes a sparsity pattern on the inverse of a covariance matrix rather than on the matrix itself. The covariance that comes back is dense; what is sparse is its precision, and the zeros there are the conditional independences the information filtering network selected.

The family declares no seam of its own, and no method dispatches on this supertype. A concrete subtype is reached through the matrix_processing_algorithm! of AbstractMatrixProcessingAlgorithm, which is the interface it inherits and which src/04_MatrixProcessing/04_MatrixProcessing.jl owns. LoGo is the shipped member, and matrix_processing_algorithm! states the contract that method satisfies.

Related

• AbstractMatrixProcessingAlgorithm
• LoGo

References

• [54] W. Barfuss, G. P. Massara, T. Di Matteo and T. Aste. Parsimonious modeling with information filtering networks. Phys. Rev. E 94, 062306 (2016).

const DVarInfo_DDVarInfo = Union{<:Distance{<:Any, <:VariationInfoDistance},
                                 <:DistanceDistance{<:Any, <:Any, <:Any, <:Any,
                                                    <:VariationInfoDistance}}

Alias for distance types using variation of information metrics.

Matches either a VariationInfoDistance-based Distance or a VariationInfoDistance-based DistanceDistance. Used for dispatch in DBHT-based phylogeny computation.

Related

• VariationInfoDistance
• Distance
• DistanceDistance

jlogo!(jlogo::MatNum, sigma::MatNum, source::MatNum, sign::Integer)

Efficiently accumulate contributions to the sparse inverse covariance matrix for LoGo/DBHT.

This internal function updates the jlogo matrix in-place by iterating over a list of cliques or separators (source), extracting the corresponding submatrix from the covariance matrix sigma, inverting it, and adding (or subtracting) the result to the appropriate block in jlogo, scaled by sign.

Every row of source names the same number of vertices, because tmp is allocated once at size(source, 2) and reused. PMFG_T2s satisfies that: its 3-cliques all carry three vertices and its 4-cliques all carry four.

Algorithm

1. Allocate tmp, one square block of the width of a row of source.
2. For each row i of source, read the index set v and gather the submatrix sigma[v, v] into tmp.
3. Invert tmp.
4. Add sign times each entry of the inverse into jlogo, at the pair of v that entry belongs to.

Arguments

• jlogo: N × N matrix to be updated in-place. It is added to, never cleared, so the caller sets what it starts from.
• sigma: N × N covariance matrix. Only the blocks the rows of source name are read.
• source: Ns×k index matrix. Each row holds the k vertices of one clique or separator, and k is 4 for the cliques and 3 for the separators of a PMFG.
• sign: +1 for cliques, -1 for separators.

Returns

• nothing. Updates jlogo in-place.

Related

• J_LoGo
• LoGo

J_LoGo(sigma::MatNum, separators::MatNum, cliques::MatNum)

Compute the sparse inverse covariance matrix using the LoGo (Local-Global) algorithm [54].

This function implements the LoGo sparse inverse covariance estimation by combining clique and separator contributions from a Planar Maximally Filtered Graph (PMFG) or similar clique tree structure. It efficiently accumulates the inverses of covariance submatrices corresponding to cliques and separators, producing a sparse precision (inverse covariance) matrix suitable for robust portfolio optimization and risk management.

Mathematical definition

\[J_{i,\,j} = \sum_{c \in \mathcal{C}} \mathbf{1}\left[i \in c \land j \in c\right] \left(\mathbf{\Sigma}_{c,\,c}\right)^{-1}_{i,\,j} - \sum_{s \in \mathcal{S}} \mathbf{1}\left[i \in s \land j \in s\right] \left(\mathbf{\Sigma}_{s,\,s}\right)^{-1}_{i,\,j}\,.\]

Where:

• $\mathbf{J}$: LoGo precision matrix, $N \times N$.
• $\mathbf{\Sigma}$: Covariance matrix, $N \times N$.
• $\mathbf{\Sigma}_{c,\,c}$: Its submatrix on the index set $c$.
• $\mathcal{C}$: Set of the cliques of the network.
• $\mathcal{S}$: Set of its separators.
• $N$: Number of assets.

$J_{i,\,j}$ is exactly zero for a pair that no clique holds together, so the sparsity pattern of $\mathbf{J}$ is the edge set of the network. That is the conditional independence the filtering states, and it survives in the precision alone.

Algorithm

1. Set jlogo to a zero matrix of the size of sigma.
2. Add the inverse of every clique block with jlogo! at sign = 1.
3. Subtract the inverse of every separator block with jlogo! at sign = -1.

Arguments

• sigma: N × N covariance matrix.
• separators: Ns×3 index matrix. Each row holds the vertices of one separator, which are the 3-cliques of a PMFG.
• cliques: Nq×4 index matrix. Each row holds the vertices of one clique, which are the 4-cliques of a PMFG.

Returns

• jlogo::Matrix{<:Number}: N × N LoGo sparse precision matrix. The covariance it stands for is its inverse, and that inverse is dense.

Related

• jlogo!
• LoGo

LoGo_dist_assert(de::DVarInfo_DDVarInfo, sigma::MatNum, X::MatNum)

Validate compatibility of the distance estimator and covariance matrix for LoGo sparse inverse covariance estimation by checking size(sigma, 1) == size(X, 2).

The check runs for a VariationInfoDistance estimator alone, which is the only family that reads X rather than the correlation matrix. Every other estimator takes the no-op fallback, so a mismatched X passes. The narrow signature is what makes that so: it is bounded by DVarInfo_DDVarInfo, and the configurations that reach it are a Distance or a DistanceDistance whose algorithm is a VariationInfoDistance.

Arguments

• de: Distance estimator whose algorithm is a VariationInfoDistance.
• sigma: N × N covariance matrix.
• X: T × N data matrix. size(X, 2) is the asset axis, which is the axis the check reads.

Validation

• size(sigma, 1) == size(X, 2).

Returns

• nothing.

Related

• LoGo
• logo!

LoGo_dist_assert(args...)

No-op fallback for other distance estimators.

Every distance estimator outside DVarInfo_DDVarInfo derives its distance from the correlation matrix and never reads X, so there is no shape of X for it to disagree with. This method makes that the default and leaves the check to the one family that owns it.

Algorithm

1. Return nothing. No shape is read, and args is discarded.

Arguments

• args...: The distance estimator, the covariance matrix and the data matrix (all ignored).

Returns

• nothing.

Related

• DVarInfo_DDVarInfo
• LoGo
• logo!

logo!(::Nothing, args...; kwargs...)

No-op fallback: return nothing when no LoGo algorithm is configured.

This is the branch a matrix processing pipeline takes when its sparsification field is nothing, so a caller composes the step in and out without a branch of its own.

Algorithm

1. Return nothing. No matrix is read and no matrix is written, and args and kwargs are discarded.

Arguments

• ::Nothing: No LoGo algorithm configured.
• args...: Optional arguments (ignored).
• kwargs...: Optional keyword arguments (ignored).

Returns

• nothing. The caller's sigma is left as it stands.

Related

• LoGo
• logo!

logo!(je::LoGo, sigma::MatNum, X::MatNum;
      dims::Int = 1, kwargs...)

Compute the LoGo (Local-Global) covariance matrix and update sigma in-place.

This method implements the LoGo algorithm for sparse inverse covariance estimation using the Planar Maximally Filtered Graph (PMFG) and clique-based decomposition. It validates inputs, computes the similarity and distance matrices, constructs the PMFG, identifies cliques and separators, and updates the input covariance matrix sigma in-place by inverting the LoGo sparse inverse covariance estimate. The result is projected to the nearest positive definite matrix if a Posdef estimator is not nothing.

Algorithm

1. Check that sigma is square, and check its asset axis against X through LoGo_dist_assert.
2. Read the diagonal of sigma into s. When any entry of s is not one, sigma is a covariance matrix: replace s with its square roots and derive the correlation matrix S with StatsBase.cov2cor. sigma itself stays a covariance, and it is what step 6 decomposes.
3. Take the distance matrix D from S and X with je.de, and check that D lies in the domain je.sim needs.
4. Map D to the non-negative similarity S with je.sim, through distance_to_similarity.
5. Build the TMFG on S with PMFG_T2s at nargout = 4, and take its 3-cliques as the separators and its 4-cliques as the cliques.
6. Build the LoGo precision matrix from sigma with J_LoGo, invert it, and write the result into sigma.
7. Repair sigma with posdef! through je.pdm, which does nothing when je.pdm is nothing.

Arguments

• je: LoGo algorithm instance.
• sigma: Covariance matrix (N × N), updated in-place with the LoGo sparse inverse covariance.
• X: Data matrix (T × N).
• dims: Dimension along which to perform the computation.
• kwargs...: Additional keyword arguments passed to distance and similarity estimators.

Validation

• size(sigma, 1) == size(sigma, 2).
• size(sigma, 1) == size(X, 2), only when je.de reads X. LoGo_dist_assert carries the check, and it has a method for the variation-of-information estimators alone; every other estimator takes the no-op fallback. A default LoGo() therefore accepts a 20 × 20 sigma beside a 400 × 10 X and returns without raising, because CanonicalDistance derives the distance from the correlation matrix and never touches X.

Returns

• nothing. The input sigma is updated in-place.

Related

• LoGo
• J_LoGo
• LoGo_dist_assert
• PMFG_T2s
• distance_to_similarity
• Posdef

logo(je::LoGo, sigma::MatNum, X::MatNum; dims::Int = 1, kwargs...) -> MatNum

Apply the LoGo (Local-Global) transformation to the covariance matrix and return the result as a new matrix.

This is the non-mutating variant of logo!. It copies sigma before applying the transformation.

Algorithm

1. Copy sigma.
2. Run logo! on the copy, which carries every step and every check of this transformation.
3. Return the copy.

Arguments

• je::LoGo: LoGo algorithm configuration.
• sigma::MatNum: N × N covariance matrix to transform (not mutated).
• X::MatNum: T × N returns data matrix.
• dims: Dimension along which to perform the computation.
• kwargs...: Additional keyword arguments passed to distance and similarity estimators.

Validation

• Every check of logo! applies, and it raises from step 2.

Returns

• sigma::MatNum: N × N copy of the input with the LoGo transformation applied.

Related

• logo!
• LoGo
• J_LoGo