Mutual info covariance

PortfolioOptimisers.MutualInfoCovariance — Type
struct MutualInfoCovariance{__T_ve, __T_bins, __T_normalise, __T_cache} <: AbstractCovarianceEstimator

Measures codependence with mutual information, which detects a non-linear relationship that a correlation can miss.

The estimator computes the mutual information matrix of the returns with mutual_info. When normalise is true, it divides each entry by the smaller of the two marginal entropies. It then scales the matrix by the marginal standard deviations from ve to get a covariance matrix.

Mutual information is non-negative, so every entry of the matrix is non-negative, and the matrix shows no pair as opposed. A strong negative linear relationship gives a large positive entry. When normalise is false, the correlation matrix has no upper bound, and its diagonal is the marginal entropy in nats, not one. It is then a codependence matrix, not a correlation matrix in the usual sense.

Section 12 of [50] defines the rate of transmission $R = H(x) + H(y) - H(x,\,y)$, which is the mutual information, with the entropies of Section 6. The estimate here replaces those entropies with histogram plug-in values. The division by the smaller marginal entropy is Equation 6.19 of [12]. Equation 18 in Section 3 of [51] proves $I \leq \min(H(C),\, H(C'))$ for clusterings, and this bound puts the quotient in $[0,\, 1]$. No work that first states this normalisation was found.

Mathematical definition

\[\begin{align} \hat{\boldsymbol{\rho}}_{ij} &= \mathrm{MI}(X_i,\, X_j)\,, \\ \hat{\mathbf{\Sigma}}_{ij} &= \begin{cases} \hat{\sigma}_i^2 & i = j \\ \hat{\boldsymbol{\rho}}_{ij}\,\hat{\sigma}_i\,\hat{\sigma}_j & i \neq j \end{cases}\,. \end{align}\]

Where:

  • $\hat{\boldsymbol{\rho}}_{ij}$: Mutual information-based correlation between assets $i$ and $j$.
  • $\hat{\mathbf{\Sigma}}_{ij}$: Covariance between assets $i$ and $j$.
  • $\mathrm{MI}(X_i, X_j)$: Mutual information between assets $i$ and $j$, computed by mutual_info. When normalise is true, mutual_info divides it by $\min(H(X_i), H(X_j))$, which bounds it to $[0,\, 1]$.
  • $\hat{\sigma}_i$: Marginal standard deviation of asset $i$ from the variance estimator ve.

The diagonal of $\hat{\mathbf{\Sigma}}$ is the variance whatever value $\hat{\boldsymbol{\rho}}$ has there, so the two values of normalise give the same diagonal and differ only off it.

Fields

  • ve: Variance estimator.
  • bins: Binning algorithm or fixed number of bins.
  • normalise: If true, the result is the normalised mutual information or the normalised variation of information.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it. factory keeps it unchanged, because a factory call resolves the configuration, not the sample. port_opt_view copies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone. obs_weights_view removes it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.

Constructors

MutualInfoCovariance(;    ve::AbstractVarianceEstimator = SimpleVariance(),    bins::Int_Bin = HacineGharbiRavier(),    normalise::Bool = true,    cache::Option{<:AbstractPartialFitState} = nothing) -> MutualInfoCovariance

Keywords correspond to the struct's fields.

Validation

  • If bins is an integer, 0 < bins <= RESOURCE_LIMITS[].max_bins. The joint histogram is bins × bins, and RESOURCE_LIMITS sets the cap.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • ve: Recursively updated through factory.
  • cache: Carried unchanged through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Examples

julia> MutualInfoCovariance()MutualInfoCovariance         ve ┼ SimpleVariance            │          me ┼ SimpleExpectedReturns            │             │   w ┴ nothing            │           w ┼ nothing            │   corrected ┴ Bool: true       bins ┼ HacineGharbiRavier()  normalise ┴ Bool: true

Related

References

  • [50] C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948). Section 12.
  • [51] M. Meilă. Comparing clusterings—an information based distance. Journal of Multivariate Analysis 98, 873–895 (2007). Section 3, Equation 18.
  • [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 6.1.6, Equation 6.19. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
Statistics.cor — Method
Statistics.cor(ce::MutualInfoCovariance, X::MatNum; dims::Int = 1, kwargs...)

Compute the mutual information correlation matrix with a MutualInfoCovariance estimator.

The method computes the mutual information of each pair of columns of X with mutual_info, under the binning algorithm ce.bins and the flag ce.normalise. The mutual information detects a linear and a non-linear dependence, and the Pearson correlation detects only a linear one.

The result is in $[0, 1]$ with a unit diagonal only when ce.normalise is true. When it is false, the entries are mutual information in nats, and the diagonal is the marginal entropy, as mutual_info states.

Algorithm

  1. Orient X to observations × assets with dims_oriented, which checks dims and transposes X when dims is 2.
  2. Refuse a gapped sample with assert_finite_sample.
  3. Return mutual_info of the oriented matrix, under the binning algorithm ce.bins and the flag ce.normalise.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which the computation runs.
  • kwargs...: Ignored keyword arguments.

Validation

  • dims in (1, 2).

Returns

  • rho::MatNum: Correlation matrix assets x assets.

Related

source

References

[12]
D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
[50]
C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948).
[51]
M. Meilă. Comparing clusterings—an information based distance. Journal of Multivariate Analysis 98, 873–895 (2007).