Mutual Information Covariance

struct MutualInfoCovariance{__T_ve, __T_bins, __T_normalise} <: AbstractCovarianceEstimator

Measures codependence with mutual information, which captures a non-linear relationship a correlation misses.

The mutual information matrix is optionally normalised by the smaller of the two marginal entropies, then rescaled by the marginal standard deviations of ve to give a covariance matrix.

Mutual information is non-negative, so every entry of the matrix is non-negative and no pair is ever reported as opposed. A negative linear relationship reads as a strong one, not as a negative one. When normalise is false the correlation matrix is unbounded above and its diagonal carries the marginal entropy in nats rather than one, so it is a codependence matrix rather than a correlation matrix in the usual sense.

Mathematical definition

\[\begin{align} \hat{\boldsymbol{\rho}}_{ij} &= \mathrm{MI}(X_i,\, X_j)\,, \\ \hat{\mathbf{\Sigma}}_{ij} &= \begin{cases} \hat{\sigma}_i^2 & i = j \\ \hat{\boldsymbol{\rho}}_{ij}\,\hat{\sigma}_i\,\hat{\sigma}_j & i \neq j \end{cases}\,. \end{align}\]

Where:

• $\hat{\boldsymbol{\rho}}_{ij}$: Mutual information-based correlation between assets $i$ and $j$.
• $\hat{\mathbf{\Sigma}}_{ij}$: Covariance between assets $i$ and $j$.
• $\mathrm{MI}(X_i, X_j)$: Mutual information between assets $i$ and $j$, computed by mutual_info. When normalise is true it is divided by $\min(H(X_i), H(X_j))$, which bounds it to $[0,\, 1]$.
• $\hat{\sigma}_i$: Marginal standard deviation of asset $i$ from the variance estimator ve.

The diagonal of $\hat{\mathbf{\Sigma}}$ is the variance whatever $\hat{\boldsymbol{\rho}}$ carries there, so the two values of normalise give the same diagonal and differ only off it.

Fields

• ve: Variance estimator.

• bins: Binning algorithm or fixed number of bins.

• normalise: Whether to normalise the mutual and/or variation of information calculation.

Constructors

MutualInfoCovariance(;    ve::AbstractVarianceEstimator = SimpleVariance(),    bins::Int_Bin = HacineGharbiRavier(),    normalise::Bool = true) -> MutualInfoCovariance

Keywords correspond to the struct's fields.

Validation

• If bins is an integer, 0 < bins <= RESOURCE_LIMITS[].max_bins (the joint histogram is bins × bins; see RESOURCE_LIMITS).

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

• ve: Recursively updated via factory.

View parameters

When port_opt_view is called on this type, the following @vprop-tagged fields are automatically subset to the selected indices:

• ve: Recursively viewed via port_opt_view.

Examples

julia> MutualInfoCovariance()MutualInfoCovariance         ve ┼ SimpleVariance            │          me ┼ SimpleExpectedReturns            │             │   w ┴ nothing            │           w ┼ nothing            │   corrected ┴ Bool: true       bins ┼ HacineGharbiRavier()  normalise ┴ Bool: true

Related

• AbstractVarianceEstimator
• AbstractBins
• mutual_info
• factory
• port_opt_view

References

• [24] C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948).
• [5] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 6.1.6, equations 6.18 and 6.19.

Statistics.cor(ce::MutualInfoCovariance, X::MatNum; dims::Int = 1, kwargs...)

Compute the mutual information (MI) correlation matrix using a MutualInfoCovariance estimator.

This method computes the pairwise mutual information correlation matrix for the input data matrix X, using the binning strategy and normalisation specified in ce. The MI correlation captures both linear and nonlinear dependencies between asset returns, making it robust to complex relationships that may not be detected by traditional correlation measures.

The result is bounded by $[0, 1]$ with a unit diagonal only when ce.normalise is true. When it is false the entries are mutual information in nats and the diagonal is the marginal entropy, as mutual_info states.

Algorithm

1. Orient X to observations × assets with dims_oriented, which validates dims and transposes when dims is 2.
2. Return mutual_info of the oriented matrix, under the binning algorithm ce.bins and the flag ce.normalise.

Arguments

• ce: Covariance estimator.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• dims: Dimension along which to perform the computation.
• kwargs...: Additional keyword arguments (currently unused).

Validation

• dims in (1, 2).

Returns

• rho::MatNum: Correlation matrix assets x assets.

Related

• MutualInfoCovariance
• mutual_info
• Int_Bin
• dims_oriented
• cov(ce::AbstractCovarianceEstimator, X::MatNum; dims::Int = 1, kwargs...)