Mutual info covariance
PortfolioOptimisers.MutualInfoCovariance — Type
struct MutualInfoCovariance{__T_ve, __T_bins, __T_normalise, __T_cache} <: AbstractCovarianceEstimatorMeasures codependence with mutual information, which detects a non-linear relationship that a correlation can miss.
The estimator computes the mutual information matrix of the returns with mutual_info. When normalise is true, it divides each entry by the smaller of the two marginal entropies. It then scales the matrix by the marginal standard deviations from ve to get a covariance matrix.
Mutual information is non-negative, so every entry of the matrix is non-negative, and the matrix shows no pair as opposed. A strong negative linear relationship gives a large positive entry. When normalise is false, the correlation matrix has no upper bound, and its diagonal is the marginal entropy in nats, not one. It is then a codependence matrix, not a correlation matrix in the usual sense.
Section 12 of [50] defines the rate of transmission $R = H(x) + H(y) - H(x,\,y)$, which is the mutual information, with the entropies of Section 6. The estimate here replaces those entropies with histogram plug-in values. The division by the smaller marginal entropy is Equation 6.19 of [12]. Equation 18 in Section 3 of [51] proves $I \leq \min(H(C),\, H(C'))$ for clusterings, and this bound puts the quotient in $[0,\, 1]$. No work that first states this normalisation was found.
Mathematical definition
\[\begin{align} \hat{\boldsymbol{\rho}}_{ij} &= \mathrm{MI}(X_i,\, X_j)\,, \\ \hat{\mathbf{\Sigma}}_{ij} &= \begin{cases} \hat{\sigma}_i^2 & i = j \\ \hat{\boldsymbol{\rho}}_{ij}\,\hat{\sigma}_i\,\hat{\sigma}_j & i \neq j \end{cases}\,. \end{align}\]
Where:
- $\hat{\boldsymbol{\rho}}_{ij}$: Mutual information-based correlation between assets $i$ and $j$.
- $\hat{\mathbf{\Sigma}}_{ij}$: Covariance between assets $i$ and $j$.
- $\mathrm{MI}(X_i, X_j)$: Mutual information between assets $i$ and $j$, computed by
mutual_info. Whennormaliseistrue,mutual_infodivides it by $\min(H(X_i), H(X_j))$, which bounds it to $[0,\, 1]$. - $\hat{\sigma}_i$: Marginal standard deviation of asset $i$ from the variance estimator
ve.
The diagonal of $\hat{\mathbf{\Sigma}}$ is the variance whatever value $\hat{\boldsymbol{\rho}}$ has there, so the two values of normalise give the same diagonal and differ only off it.
Fields
ve: Variance estimator.
bins: Binning algorithm or fixed number of bins.
normalise: Iftrue, the result is the normalised mutual information or the normalised variation of information.
cache: Optional partial-fit state. It isnothinguntilpartial_fit!writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it.factorykeeps it unchanged, because a factory call resolves the configuration, not the sample.port_opt_viewcopies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone.obs_weights_viewremoves it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.
Constructors
MutualInfoCovariance(; ve::AbstractVarianceEstimator = SimpleVariance(), bins::Int_Bin = HacineGharbiRavier(), normalise::Bool = true, cache::Option{<:AbstractPartialFitState} = nothing) -> MutualInfoCovarianceKeywords correspond to the struct's fields.
Validation
- If
binsis an integer,0 < bins <= RESOURCE_LIMITS[].max_bins. The joint histogram isbins × bins, andRESOURCE_LIMITSsets the cap.
Propagated parameters
factory passes its arguments on to these @fprop fields:
View parameters
port_opt_view takes the view of these @vprop fields at the selected indices:
ve: Recursively viewed throughport_opt_view.cache: Sliced to the selected assets throughport_opt_view.
Examples
julia> MutualInfoCovariance()MutualInfoCovariance ve ┼ SimpleVariance │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ w ┼ nothing │ corrected ┴ Bool: true bins ┼ HacineGharbiRavier() normalise ┴ Bool: trueRelated
References
- [50] C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948). Section 12.
- [51] M. Meilă. Comparing clusterings—an information based distance. Journal of Multivariate Analysis 98, 873–895 (2007). Section 3, Equation 18.
- [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 6.1.6, Equation 6.19. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
Statistics.cor — Method
Statistics.cor(ce::MutualInfoCovariance, X::MatNum; dims::Int = 1, kwargs...)Compute the mutual information correlation matrix with a MutualInfoCovariance estimator.
The method computes the mutual information of each pair of columns of X with mutual_info, under the binning algorithm ce.bins and the flag ce.normalise. The mutual information detects a linear and a non-linear dependence, and the Pearson correlation detects only a linear one.
The result is in $[0, 1]$ with a unit diagonal only when ce.normalise is true. When it is false, the entries are mutual information in nats, and the diagonal is the marginal entropy, as mutual_info states.
Algorithm
- Orient
Xtoobservations × assetswithdims_oriented, which checksdimsand transposesXwhendimsis2. - Refuse a gapped sample with
assert_finite_sample. - Return
mutual_infoof the oriented matrix, under the binning algorithmce.binsand the flagce.normalise.
Arguments
ce: Covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Ignored keyword arguments.
Validation
dims in (1, 2).
Returns
rho::MatNum: Correlation matrixassets x assets.
Related
References
- [12]
- D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
- [50]
- C. E. Shannon. A mathematical theory of communication. The Bell System Technical Journal 27, 379–423 (1948).
- [51]
- M. Meilă. Comparing clusterings—an information based distance. Journal of Multivariate Analysis 98, 873–895 (2007).