LoGo: private API
PortfolioOptimisers.InverseMatrixSparsificationAlgorithm — Type
abstract type InverseMatrixSparsificationAlgorithm <: AbstractMatrixProcessingAlgorithmAbstract supertype for all inverse matrix sparsification algorithms.
A member of this family makes the inverse of a covariance matrix sparse, and it does not make the matrix itself sparse. The covariance matrix that it writes is dense. What is sparse is its precision matrix, and each zero of the precision matrix is a conditional independence that the information filtering network selects.
The family declares no interface of its own, and no method dispatches on this supertype. The pipeline reaches a concrete subtype through the matrix_processing_algorithm! method of AbstractMatrixProcessingAlgorithm. The family gets that interface from its supertype, and src/04_MatrixProcessing/04_MatrixProcessing.jl owns it. LoGo is the one member that the library ships, and its matrix_processing_algorithm! method states the contract that it satisfies.
Related
References
- [130] W. Barfuss, G. P. Massara, T. Di Matteo and T. Aste. Parsimonious modeling with information filtering networks. Phys. Rev. E 94, 062306 (2016). Section II.C, Equation 4.
PortfolioOptimisers.DVarInfo_DDVarInfo — Type
const DVarInfo_DDVarInfo = Union{Distance{<:Any, <:VariationInfoDistance},
DistanceDistance{<:Any, <:Any, <:Any, <:Any,
<:VariationInfoDistance}}Alias for the distance estimators whose algorithm is a VariationInfoDistance.
The alias matches a Distance or a DistanceDistance whose algorithm is a VariationInfoDistance. LoGo_dist_assert dispatches on it, because the distance of these estimators reads the asset axis of X.
Related
PortfolioOptimisers.jlogo! — Function
jlogo!(jlogo::MatNum, sigma::MatNum, source::MatNum, sign::Integer)Add sign times the inverse of each block of sigma that a row of source names into the same block of jlogo.
J_LoGo calls this function two times, once on the cliques and once on the separators. For each row of source, the function takes the submatrix of sigma on the vertices of the row and inverts it. It then adds sign times the inverse into jlogo in place.
Every row of source must name the same number of vertices, because the function allocates tmp once at size(source, 2) and uses it again for each row. PMFG_T2s satisfies that. Each of its 3-cliques has three vertices, and each of its 4-cliques has four.
Algorithm
- Allocate
tmp, one square block of the width of a row ofsource. - For each row
iofsource, read the index setvand gather the submatrixsigma[v, v]intotmp. - Invert
tmp. - Add
signtimes each entry of the inverse intojlogo, at the pair ofvthat entry belongs to.
Arguments
jlogo:N × Nmatrix that the function changes in place. The function adds to it and never clears it, so the caller sets its first value.sigma:N × Ncovariance matrix. The function reads only the blocks that the rows ofsourcename.source:Ns×kindex matrix. Each row holds thekvertices of one clique or separator.kis4for the cliques and3for the separators of a TMFG.sign:+1for the cliques, and-1for the separators.
Returns
nothing. The function changesjlogoin place.
Related
PortfolioOptimisers.J_LoGo — Function
J_LoGo(sigma::MatNum, separators::MatNum, cliques::MatNum)Compute the sparse precision matrix of the LoGo (Local-Global) method of [130].
The function adds the inverse of the covariance block of each clique of a Triangulated Maximally Filtered Graph (TMFG), and it subtracts the inverse of the covariance block of each separator. The precision matrix is the inverse of a covariance matrix.
Mathematical definition
\[J_{i,\,j} = \sum_{c \in \mathcal{C}} \mathbf{1}\left[i \in c \land j \in c\right] \left(\mathbf{\Sigma}_{c,\,c}\right)^{-1}_{i,\,j} - \sum_{s \in \mathcal{S}} \mathbf{1}\left[i \in s \land j \in s\right] \left(\mathbf{\Sigma}_{s,\,s}\right)^{-1}_{i,\,j}\,.\]
Where:
- $\mathbf{J}$: LoGo precision matrix, $N \times N$.
- $\mathbf{\Sigma}$: Covariance matrix, $N \times N$.
- $\mathbf{\Sigma}_{c,\,c}$: Its submatrix on the index set $c$.
- $\mathcal{C}$: Set of the cliques of the network.
- $\mathcal{S}$: Set of its separators.
- $N$: Number of assets.
$J_{i,\,j}$ is exactly zero for a pair that no clique holds together, so the sparsity pattern of $\mathbf{J}$ is the edge set of the network. Each zero is a conditional independence that the filtering states, and only the precision matrix holds it.
Equation 4 of [130] weights the inverse of each separator block by $k(S) - 1$, where $k(S)$ is the number of components that the removal of the separator leaves. In a TMFG every separator joins exactly two cliques, so the weight is one and the formula above is Equation 4. On another clique tree the formula needs that weight.
Algorithm
- Set
jlogoto a zero matrix of the size ofsigma. - Add the inverse of every clique block with
jlogo!atsign = 1. - Subtract the inverse of every separator block with
jlogo!atsign = -1.
Arguments
sigma:N × Ncovariance matrix.separators:Ns×3index matrix. Each row holds the vertices of one separator, which are the 3-cliques of a TMFG.cliques:Nq×4index matrix. Each row holds the vertices of one clique, which are the 4-cliques of a TMFG.
Returns
jlogo::Matrix{<:Number}:N × NLoGo sparse precision matrix. Its inverse is the covariance matrix, and that inverse is dense.
Related
PortfolioOptimisers.LoGo_dist_assert — Function
LoGo_dist_assert(de::DVarInfo_DDVarInfo, sigma::MatNum, X::MatNum; dims::Int = 1)Check that the asset axis of X has the size of sigma, for a distance estimator that reads X.
The check runs for a VariationInfoDistance estimator alone, because the distance of that family reads the asset axis of X. Every other distance estimator takes the method on AbstractDistanceEstimator, which does nothing, so a mismatched X passes. The narrow signature selects the check. Its bound is DVarInfo_DDVarInfo, so a Distance or a DistanceDistance whose algorithm is a VariationInfoDistance reaches it.
Arguments
de: Distance estimator whose algorithm is aVariationInfoDistance.sigma:N × Ncovariance matrix.X:T × Ndata matrix whendimsis1, andN × Twhendimsis2. The check reads the asset axis,size(X, 2)whendimsis1andsize(X, 1)whendimsis2.dims: Dimension along which the computation runs.
Validation
size(sigma, 1) == size(X, isone(dims) ? 2 : 1), raisingDimensionMismatch.
Returns
nothing.
Related
LoGo_dist_assert(de::AbstractDistanceEstimator, sigma::MatNum, X::MatNum; dims::Int = 1)Do nothing for a distance estimator that does not read the asset axis of X.
A distance estimator of the library outside DVarInfo_DDVarInfo derives its distance from the correlation matrix, so logo! reads no axis of X for it. A FeatureDistance is the exception, and it raises an IsNothingError in logo!, because matrix processing gives it no panel. This method is the no-op of the value check on the set that reaches it, and the narrow method holds the check for the one family that needs it. Any other type or arity is a MethodError.
Algorithm
- Return
nothing. The method reads no size.
Arguments
de: Distance estimator outsideDVarInfo_DDVarInfo, which the method ignores.sigma:N × Ncovariance matrix, which the method ignores.X: Data matrix, which the method ignores.dims: Dimension along which the computation runs. The method ignores it.
Returns
nothing.
Related
PortfolioOptimisers.logo! — Function
logo!(::Nothing, args...; kwargs...)
Return nothing when no LoGo algorithm is set.
No method of the library calls this method. When the algorithm of the matrix processing pipeline is nothing, the pipeline calls the nothing method of matrix_processing_algorithm! instead.
Algorithm
- Return
nothing. The method reads no matrix and writes no matrix, and it ignoresargsandkwargs.
Arguments
::Nothing: No LoGo algorithm is set.args...: Positional arguments, which the method ignores.kwargs...: Keyword arguments, which the method ignores.
Returns
nothing. The method does not change thesigmaof the caller.
Related
logo!(je::LoGo, sigma::MatNum, X::MatNum;
dims::Int = 1, kwargs...)Write the LoGo (Local-Global) covariance matrix into sigma.
The method builds a Triangulated Maximally Filtered Graph (TMFG) on a similarity matrix of the assets. It computes the LoGo precision matrix from the clique and separator blocks of sigma, and it writes the inverse of that precision matrix into sigma. When je.pdm is not nothing, posdef! then repairs sigma to the nearest positive definite matrix.
Algorithm
- Check that
sigmais square. Check withLoGo_dist_assertthat the asset axis ofXhas the size ofsigma. - Read the diagonal of
sigmaintos. When an entry ofsis not one,sigmais a covariance matrix. Then replaceswith its square roots, and get the correlation matrixSwithStatsBase.cov2cor.sigmastays a covariance matrix, and step 6 decomposes it. - Take the distance matrix
DfromSandXwithje.de. Check withassert_similarity_domainthatDlies in the domain thatje.simneeds. - Map
Dto the non-negative similaritySwithje.sim, throughdistance_to_similarity. - Build the TMFG on
SwithPMFG_T2satnargout = 4, and take its 3-cliques as the separators and its 4-cliques as the cliques. - Build the LoGo precision matrix from
sigmawithJ_LoGo, invert it, and write the result intosigma. - Repair
sigmawithposdef!throughje.pdm, which does nothing whenje.pdmisnothing.
Arguments
je: The LoGo algorithm.sigma:N × Ncovariance or correlation matrix. The method writes the LoGo covariance matrix into it.X:T × Ndata matrix whendimsis1, andN × Twhendimsis2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments that the method passes todistance.
Validation
size(sigma, 1) == size(sigma, 2).- The asset axis of
Xhassize(sigma, 1)entries, whenje.dereadsX.LoGo_dist_assertholds this check, and it has a method for the variation-of-information estimators alone. Every other distance estimator takes the method onAbstractDistanceEstimator, which does nothing. Thus a defaultLoGo()accepts a20 × 20sigmawith a400 × 10Xand raises no error, becauseCanonicalDistancederives the distance from the correlation matrix and does not readX.
Returns
nothing. The method changessigmain place.
Related
PortfolioOptimisers.logo — Function
logo(je::LoGo, sigma::MatNum, X::MatNum; dims::Int = 1, kwargs...) -> MatNumReturn a copy of sigma with the LoGo (Local-Global) transformation applied.
This is the out-of-place form of logo!. It copies sigma, and it runs logo! on the copy.
Algorithm
- Copy
sigma. - Run
logo!on the copy. It does every step and every check of the transformation. - Return the copy.
Arguments
je::LoGo: The LoGo algorithm.sigma::MatNum:N × Ncovariance or correlation matrix. The function does not change it.X::MatNum:T × Nreturns data matrix whendimsis1, andN × Twhendimsis2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments thatlogo!passes todistance.
Validation
- Every check of
logo!applies. The error comes from step 2.
Returns
sigma::MatNum:N × Ncopy of the input with the LoGo transformation applied.
Related
References
- [130]
- W. Barfuss, G. P. Massara, T. Di Matteo and T. Aste. Parsimonious modeling with information filtering networks. Phys. Rev. E 94, 062306 (2016).