LoGo: private API

PortfolioOptimisers.InverseMatrixSparsificationAlgorithm — Type
abstract type InverseMatrixSparsificationAlgorithm <: AbstractMatrixProcessingAlgorithm

Abstract supertype for all inverse matrix sparsification algorithms.

A member of this family makes the inverse of a covariance matrix sparse, and it does not make the matrix itself sparse. The covariance matrix that it writes is dense. What is sparse is its precision matrix, and each zero of the precision matrix is a conditional independence that the information filtering network selects.

The family declares no interface of its own, and no method dispatches on this supertype. The pipeline reaches a concrete subtype through the matrix_processing_algorithm! method of AbstractMatrixProcessingAlgorithm. The family gets that interface from its supertype, and src/04_MatrixProcessing/04_MatrixProcessing.jl owns it. LoGo is the one member that the library ships, and its matrix_processing_algorithm! method states the contract that it satisfies.

Related

References

  • [130] W. Barfuss, G. P. Massara, T. Di Matteo and T. Aste. Parsimonious modeling with information filtering networks. Phys. Rev. E 94, 062306 (2016). Section II.C, Equation 4.
source
PortfolioOptimisers.DVarInfo_DDVarInfo — Type
const DVarInfo_DDVarInfo = Union{Distance{<:Any, <:VariationInfoDistance},
                                 DistanceDistance{<:Any, <:Any, <:Any, <:Any,
                                                    <:VariationInfoDistance}}

Alias for the distance estimators whose algorithm is a VariationInfoDistance.

The alias matches a Distance or a DistanceDistance whose algorithm is a VariationInfoDistance. LoGo_dist_assert dispatches on it, because the distance of these estimators reads the asset axis of X.

Related

source
PortfolioOptimisers.jlogo! — Function
jlogo!(jlogo::MatNum, sigma::MatNum, source::MatNum, sign::Integer)

Add sign times the inverse of each block of sigma that a row of source names into the same block of jlogo.

J_LoGo calls this function two times, once on the cliques and once on the separators. For each row of source, the function takes the submatrix of sigma on the vertices of the row and inverts it. It then adds sign times the inverse into jlogo in place.

Every row of source must name the same number of vertices, because the function allocates tmp once at size(source, 2) and uses it again for each row. PMFG_T2s satisfies that. Each of its 3-cliques has three vertices, and each of its 4-cliques has four.

Algorithm

  1. Allocate tmp, one square block of the width of a row of source.
  2. For each row i of source, read the index set v and gather the submatrix sigma[v, v] into tmp.
  3. Invert tmp.
  4. Add sign times each entry of the inverse into jlogo, at the pair of v that entry belongs to.

Arguments

  • jlogo: N × N matrix that the function changes in place. The function adds to it and never clears it, so the caller sets its first value.
  • sigma: N × N covariance matrix. The function reads only the blocks that the rows of source name.
  • source: Ns×k index matrix. Each row holds the k vertices of one clique or separator. k is 4 for the cliques and 3 for the separators of a TMFG.
  • sign: +1 for the cliques, and -1 for the separators.

Returns

  • nothing. The function changes jlogo in place.

Related

source
J_LoGo(sigma::MatNum, separators::MatNum, cliques::MatNum)

Compute the sparse precision matrix of the LoGo (Local-Global) method of [130].

The function adds the inverse of the covariance block of each clique of a Triangulated Maximally Filtered Graph (TMFG), and it subtracts the inverse of the covariance block of each separator. The precision matrix is the inverse of a covariance matrix.

Mathematical definition

\[J_{i,\,j} = \sum_{c \in \mathcal{C}} \mathbf{1}\left[i \in c \land j \in c\right] \left(\mathbf{\Sigma}_{c,\,c}\right)^{-1}_{i,\,j} - \sum_{s \in \mathcal{S}} \mathbf{1}\left[i \in s \land j \in s\right] \left(\mathbf{\Sigma}_{s,\,s}\right)^{-1}_{i,\,j}\,.\]

Where:

  • $\mathbf{J}$: LoGo precision matrix, $N \times N$.
  • $\mathbf{\Sigma}$: Covariance matrix, $N \times N$.
  • $\mathbf{\Sigma}_{c,\,c}$: Its submatrix on the index set $c$.
  • $\mathcal{C}$: Set of the cliques of the network.
  • $\mathcal{S}$: Set of its separators.
  • $N$: Number of assets.

$J_{i,\,j}$ is exactly zero for a pair that no clique holds together, so the sparsity pattern of $\mathbf{J}$ is the edge set of the network. Each zero is a conditional independence that the filtering states, and only the precision matrix holds it.

Equation 4 of [130] weights the inverse of each separator block by $k(S) - 1$, where $k(S)$ is the number of components that the removal of the separator leaves. In a TMFG every separator joins exactly two cliques, so the weight is one and the formula above is Equation 4. On another clique tree the formula needs that weight.

Algorithm

  1. Set jlogo to a zero matrix of the size of sigma.
  2. Add the inverse of every clique block with jlogo! at sign = 1.
  3. Subtract the inverse of every separator block with jlogo! at sign = -1.

Arguments

  • sigma: N × N covariance matrix.
  • separators: Ns×3 index matrix. Each row holds the vertices of one separator, which are the 3-cliques of a TMFG.
  • cliques: Nq×4 index matrix. Each row holds the vertices of one clique, which are the 4-cliques of a TMFG.

Returns

  • jlogo::Matrix{<:Number}: N × N LoGo sparse precision matrix. Its inverse is the covariance matrix, and that inverse is dense.

Related

source
PortfolioOptimisers.LoGo_dist_assert — Function
LoGo_dist_assert(de::DVarInfo_DDVarInfo, sigma::MatNum, X::MatNum; dims::Int = 1)

Check that the asset axis of X has the size of sigma, for a distance estimator that reads X.

The check runs for a VariationInfoDistance estimator alone, because the distance of that family reads the asset axis of X. Every other distance estimator takes the method on AbstractDistanceEstimator, which does nothing, so a mismatched X passes. The narrow signature selects the check. Its bound is DVarInfo_DDVarInfo, so a Distance or a DistanceDistance whose algorithm is a VariationInfoDistance reaches it.

Arguments

  • de: Distance estimator whose algorithm is a VariationInfoDistance.
  • sigma: N × N covariance matrix.
  • X: T × N data matrix when dims is 1, and N × T when dims is 2. The check reads the asset axis, size(X, 2) when dims is 1 and size(X, 1) when dims is 2.
  • dims: Dimension along which the computation runs.

Validation

  • size(sigma, 1) == size(X, isone(dims) ? 2 : 1), raising DimensionMismatch.

Returns

  • nothing.

Related

source
LoGo_dist_assert(de::AbstractDistanceEstimator, sigma::MatNum, X::MatNum; dims::Int = 1)

Do nothing for a distance estimator that does not read the asset axis of X.

A distance estimator of the library outside DVarInfo_DDVarInfo derives its distance from the correlation matrix, so logo! reads no axis of X for it. A FeatureDistance is the exception, and it raises an IsNothingError in logo!, because matrix processing gives it no panel. This method is the no-op of the value check on the set that reaches it, and the narrow method holds the check for the one family that needs it. Any other type or arity is a MethodError.

Algorithm

  1. Return nothing. The method reads no size.

Arguments

  • de: Distance estimator outside DVarInfo_DDVarInfo, which the method ignores.
  • sigma: N × N covariance matrix, which the method ignores.
  • X: Data matrix, which the method ignores.
  • dims: Dimension along which the computation runs. The method ignores it.

Returns

  • nothing.

Related

source
PortfolioOptimisers.logo! — Function
logo!(::Nothing, args...; kwargs...)

Return nothing when no LoGo algorithm is set.

No method of the library calls this method. When the algorithm of the matrix processing pipeline is nothing, the pipeline calls the nothing method of matrix_processing_algorithm! instead.

Algorithm

  1. Return nothing. The method reads no matrix and writes no matrix, and it ignores args and kwargs.

Arguments

  • ::Nothing: No LoGo algorithm is set.
  • args...: Positional arguments, which the method ignores.
  • kwargs...: Keyword arguments, which the method ignores.

Returns

  • nothing. The method does not change the sigma of the caller.

Related

source
logo!(je::LoGo, sigma::MatNum, X::MatNum;
      dims::Int = 1, kwargs...)

Write the LoGo (Local-Global) covariance matrix into sigma.

The method builds a Triangulated Maximally Filtered Graph (TMFG) on a similarity matrix of the assets. It computes the LoGo precision matrix from the clique and separator blocks of sigma, and it writes the inverse of that precision matrix into sigma. When je.pdm is not nothing, posdef! then repairs sigma to the nearest positive definite matrix.

Algorithm

  1. Check that sigma is square. Check with LoGo_dist_assert that the asset axis of X has the size of sigma.
  2. Read the diagonal of sigma into s. When an entry of s is not one, sigma is a covariance matrix. Then replace s with its square roots, and get the correlation matrix S with StatsBase.cov2cor. sigma stays a covariance matrix, and step 6 decomposes it.
  3. Take the distance matrix D from S and X with je.de. Check with assert_similarity_domain that D lies in the domain that je.sim needs.
  4. Map D to the non-negative similarity S with je.sim, through distance_to_similarity.
  5. Build the TMFG on S with PMFG_T2s at nargout = 4, and take its 3-cliques as the separators and its 4-cliques as the cliques.
  6. Build the LoGo precision matrix from sigma with J_LoGo, invert it, and write the result into sigma.
  7. Repair sigma with posdef! through je.pdm, which does nothing when je.pdm is nothing.

Arguments

  • je: The LoGo algorithm.
  • sigma: N × N covariance or correlation matrix. The method writes the LoGo covariance matrix into it.
  • X: T × N data matrix when dims is 1, and N × T when dims is 2.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keyword arguments that the method passes to distance.

Validation

  • size(sigma, 1) == size(sigma, 2).
  • The asset axis of X has size(sigma, 1) entries, when je.de reads X. LoGo_dist_assert holds this check, and it has a method for the variation-of-information estimators alone. Every other distance estimator takes the method on AbstractDistanceEstimator, which does nothing. Thus a default LoGo() accepts a 20 × 20 sigma with a 400 × 10 X and raises no error, because CanonicalDistance derives the distance from the correlation matrix and does not read X.

Returns

  • nothing. The method changes sigma in place.

Related

source
logo(je::LoGo, sigma::MatNum, X::MatNum; dims::Int = 1, kwargs...) -> MatNum

Return a copy of sigma with the LoGo (Local-Global) transformation applied.

This is the out-of-place form of logo!. It copies sigma, and it runs logo! on the copy.

Algorithm

  1. Copy sigma.
  2. Run logo! on the copy. It does every step and every check of the transformation.
  3. Return the copy.

Arguments

  • je::LoGo: The LoGo algorithm.
  • sigma::MatNum: N × N covariance or correlation matrix. The function does not change it.
  • X::MatNum: T × N returns data matrix when dims is 1, and N × T when dims is 2.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keyword arguments that logo! passes to distance.

Validation

  • Every check of logo! applies. The error comes from step 2.

Returns

  • sigma::MatNum: N × N copy of the input with the LoGo transformation applied.

Related

source

References

[130]
W. Barfuss, G. P. Massara, T. Di Matteo and T. Aste. Parsimonious modeling with information filtering networks. Phys. Rev. E 94, 062306 (2016).