Denoise

A covariance matrix estimated from returns mixes the relations between the assets with sampling noise. For a given number of observations and assets, random matrix theory gives the range of eigenvalues that pure noise produces.

Denoising changes the small eigenvalues inside that range, so they have less effect on the result [26, 27]. It also lowers the condition number of the matrix, so a solver that uses the matrix is more stable.

PortfolioOptimisers.AbstractDenoiseEstimator — Type
abstract type AbstractDenoiseEstimator <: AbstractEstimator

Abstract supertype for all denoising estimator types.

A type that denoises a covariance-like or correlation-like matrix is a subtype of AbstractDenoiseEstimator.

Interfaces

To add a denoising estimator, subtype AbstractDenoiseEstimator, keep its parameters in the fields of the struct, and add these methods:

  • denoise!(dn::AbstractDenoiseEstimator, X::MatNum, q::Number) -> MatNum: Denoises X in place.
  • denoise(dn::AbstractDenoiseEstimator, X::MatNum, q::Number) -> MatNum: Optional. Returns the denoised X in a new matrix. A fallback method copies X and calls denoise!, so add this method only when it avoids the copy.

Arguments

  • dn: Matrix denoising estimator.
  • X: Covariance-like or correlation-like matrix assets × assets.
  • q: The effective sample ratio observations / assets, which sets the threshold of the spectrum.

Returns

  • X::MatNum: The denoised input matrix X.

Examples

The example below defines a minimal denoising estimator.

julia> struct MyDenoiseEstimator <: PortfolioOptimisers.AbstractDenoiseEstimator endjulia> function PortfolioOptimisers.denoise!(dn::MyDenoiseEstimator, X::PortfolioOptimisers.MatNum,                                             q::Number)           # Implement your in-place denoising estimator here.           println("Denoising matrix in-place...")           return X       endjulia> function PortfolioOptimisers.denoise(dn::MyDenoiseEstimator, X::PortfolioOptimisers.MatNum,                                            q::Number)           X = copy(X)           println("Copy X...")           denoise!(dn, X, q)           return X       endjulia> denoise!(MyDenoiseEstimator(), [1.0 2.0; 2.0 1.0], 2.0)Denoising matrix in-place...2×2 Matrix{Float64}: 1.0  2.0 2.0  1.0julia> denoise(MyDenoiseEstimator(), [1.0 2.0; 2.0 1.0], 2.0)Copy X...Denoising matrix in-place...2×2 Matrix{Float64}: 1.0  2.0 2.0  1.0

Related

source
PortfolioOptimisers.AbstractDenoiseAlgorithm — Type
abstract type AbstractDenoiseAlgorithm <: AbstractAlgorithm

Abstract supertype for all denoising algorithm types.

A type that implements one denoising algorithm is a subtype of AbstractDenoiseAlgorithm.

Interfaces

To add a denoising algorithm that Denoise can use, subtype AbstractDenoiseAlgorithm, keep its parameters in the fields of the struct, and add a method of _denoise!(alg::AbstractDenoiseAlgorithm, X::MatNum, vals::VecNum, vecs::MatNum, num_factors::Integer) -> MatNum. The method denoises the correlation matrix X in place under the algorithm. The leading underscore shows that only the dispatch of denoise! and denoise calls _denoise!, and a user of the library does not call it directly.

Arguments

  • alg: The denoising algorithm.
  • X: Covariance-like or correlation-like matrix assets × assets.
  • vals: The eigenvalues of X, in ascending order.
  • vecs: The eigenvectors of X, in the order of vals.
  • num_factors: The number of eigenvalues that are noise.

Returns

  • X::MatNum: The input matrix X, modified in place.

Examples

The example below defines a minimal denoising algorithm.

julia> struct MyDenoiseAlgorithm <: PortfolioOptimisers.AbstractDenoiseAlgorithm endjulia> function PortfolioOptimisers._denoise!(dn::MyDenoiseAlgorithm,                                              X::PortfolioOptimisers.MatNum,                                              vals::PortfolioOptimisers.VecNum,                                              vecs::PortfolioOptimisers.MatNum,                                              num_factors::Integer)           # Implement your in-place denoising logic here.           println("Denoising matrix using custom algorithm...")           return X       endjulia> denoise!(Denoise(; alg = MyDenoiseAlgorithm()), [2.0 1.0; 1.0 2.0], 1 / 100)Denoising matrix using custom algorithm...2×2 Matrix{Float64}: 2.0  1.0 1.0  2.0julia> denoise(Denoise(; alg = MyDenoiseAlgorithm()), [2.0 1.0; 1.0 2.0], 1 / 100)Denoising matrix using custom algorithm...2×2 Matrix{Float64}: 2.0  1.0 1.0  2.0

Related

source
PortfolioOptimisers.SpectralDenoise — Type
struct SpectralDenoise <: AbstractDenoiseAlgorithm

Denoises a matrix by setting its noise eigenvalues to zero. It removes the principal components that random matrix theory attributes to noise, and then rescales the reconstruction to a unit diagonal.

Mathematical definition

The algorithm sets the noise eigenvalues to zero, rebuilds the matrix from the remaining spectrum, and rescales the result to a unit diagonal:

\[\begin{align} \tilde{\lambda}_i &= \begin{cases} 0 & \lambda_i \leq \lambda_+ \\ \lambda_i & \lambda_i > \lambda_+ \end{cases}\,, \\ \mathbf{C}_{\mathrm{signal}} &= \mathbf{V} \, \mathrm{Diag}(\tilde{\boldsymbol{\lambda}}) \, \mathbf{V}^\intercal\,, \\ \tilde{X}_{ij} &= \frac{(C_{\mathrm{signal}})_{ij}}{\sqrt{(C_{\mathrm{signal}})_{ii} \, (C_{\mathrm{signal}})_{jj}}}\,. \end{align}\]

Where:

  • $\tilde{\lambda}_i$: Denoised $i$-th eigenvalue.
  • $\lambda_i$: $i$-th eigenvalue of the input matrix.
  • $\lambda_+$: Marčenko-Pastur upper bound of the noise band. An eigenvalue is noise when $\lambda_i \leq \lambda_+$, and signal when $\lambda_i > \lambda_+$.
  • $\mathbf{V}$: Eigenvector matrix of the input.
  • $\mathbf{C}_{\mathrm{signal}}$: Reconstruction from the signal eigenpairs alone.
  • $\tilde{\mathbf{X}}$: Denoised matrix.

When the noise eigenvalues become zero, the diagonal of $\mathbf{C}_{\mathrm{signal}}$ falls below one. So the rescaling changes every entry of the matrix.

When every eigenvalue is at or below $\lambda_+$, $\mathbf{C}_{\mathrm{signal}}$ is the zero matrix and the rescaling is undefined. The denoised matrix is the identity in that case:

\[\tilde{\mathbf{X}} = \mathbf{I} \quad \textrm{if} \quad \lambda_i \leq \lambda_+ \quad \forall \, i\,.\]

No signal survives, so every asset keeps its own variance and no pair keeps a correlation. FixedDenoise returns the identity on the same input, so the two tags agree on this case.

Section VIII of [28] sets the noise eigenvalues to zero, and then it overwrites the diagonal with one. This tag rescales the reconstruction instead, as Equation 3.55 of [12] states. The book names [26] as the source of the method, but that work does not state it. Its Section 2.5 gives only the constant residual eigenvalue and the targeted shrinkage, and Exercise 6 of its Section 2.9 only asks what happens when the components below the threshold are dropped. A search of the literature found no work that states the rescale after the zeroed eigenvalues, so the rescale follows the book.

Algorithm

The branch of _denoise! that this tag selects runs these steps.

  1. When num_factors equals length(vals), write the identity into X and return it. Every eigenvalue is noise, and steps 2 to 4 divide zero by zero on such a spectrum.
  2. Set the num_factors smallest entries of vals to zero. vals is sorted ascending, so those are the noise eigenvalues.
  3. Rebuild the matrix as vecs * Diagonal(vals) * transpose(vecs), which is the signal-only reconstruction $\mathbf{C}_{\mathrm{signal}}$.
  4. Rescale the reconstruction to unit diagonal with StatsBase.cov2cor, and write the result into X. The rescaling also sheds the round-off of the eigendecomposition, so this branch never pins the diagonal by hand.

Constructors

SpectralDenoise() -> SpectralDenoise

Examples

julia> SpectralDenoise()SpectralDenoise()

Related

References

  • [28] V. Plerou, P. Gopikrishnan, B. Rosenow, L. A. Amaral, T. Guhr and H. E. Stanley. Random matrix approach to cross correlations in financial data. Phys. Rev. E 65, 066126 (2002). Section VIII.
  • [27] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967). Section 1, Example 1.
  • [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 3.5.2.2, Equations 3.54 and 3.55. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 2.9, Exercise 6.
source
PortfolioOptimisers.FixedDenoise — Type
struct FixedDenoise <: AbstractDenoiseAlgorithm

Denoises a matrix by replacing its noise eigenvalues with their own mean. It flattens the principal components that random matrix theory attributes to noise, and does not remove them. Then it rescales the reconstruction to a unit diagonal.

Mathematical definition

The algorithm replaces the noise eigenvalues by their own mean, rebuilds the matrix from the flattened spectrum, and rescales the result to a unit diagonal:

\[\begin{align} \tilde{\lambda}_i &= \begin{cases} \bar{\lambda}_\text{noise} & \lambda_i \leq \lambda_+ \\ \lambda_i & \lambda_i > \lambda_+ \end{cases}\,, \\ \mathbf{C} &= \mathbf{V} \, \mathrm{Diag}(\tilde{\boldsymbol{\lambda}}) \, \mathbf{V}^\intercal\,, \\ \tilde{X}_{ij} &= \frac{C_{ij}}{\sqrt{C_{ii} C_{jj}}}\,. \end{align}\]

Where:

  • $\tilde{\lambda}_i$: Denoised $i$-th eigenvalue.
  • $\lambda_i$: $i$-th eigenvalue of the input matrix.
  • $\lambda_+$: Marčenko-Pastur upper bound of the noise band. An eigenvalue is noise when $\lambda_i \leq \lambda_+$, and signal when $\lambda_i > \lambda_+$.
  • $\bar{\lambda}_\text{noise}$: The mean of the noise eigenvalues.
  • $\mathbf{V}$: Eigenvector matrix of the input.
  • $\mathbf{C}$: The reconstruction from the flattened spectrum.
  • $\tilde{\mathbf{X}}$: Denoised matrix.

The flat noise spectrum keeps the trace but not the diagonal. So the rescaling changes every entry of the matrix.

Page 1469 of [29] gives each noise eigenvector one constant eigenvalue, chosen so that the trace is kept, and that constant is the mean of the noise eigenvalues. That work does not rescale. Snippet 3 of Section 4.2 of [30] first adds the rescale to unit diagonal, and Section 2.5.1 of [26] states the method as an equation.

Algorithm

The branch of _denoise! that this tag selects runs these steps.

  1. Replace the num_factors smallest entries of vals by their own mean. vals is sorted ascending, so those are the noise eigenvalues.
  2. Rebuild the matrix as vecs * Diagonal(vals) * transpose(vecs), which is the reconstruction $\mathbf{C}$ from the flattened spectrum.
  3. Rescale the reconstruction to unit diagonal with StatsBase.cov2cor, and write the result into X. The rescaling also sheds the round-off of the eigendecomposition, so this branch never pins the diagonal by hand.

Constructors

FixedDenoise() -> FixedDenoise

Examples

julia> FixedDenoise()FixedDenoise()

Related

References

  • [29] L. Laloux, P. Cizeau, J.-P. Bouchaud and M. Potters. Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83, 1467–1470 (1999). Page 1469.
  • [30] M. López de Prado. A robust estimator of the efficient frontier. SSRN Electronic Journal (2019). Section 4.2, Snippet 3.
  • [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 2.5.1, Snippet 2.5.
  • [27] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967). Section 1, Example 1.
source
PortfolioOptimisers.ShrunkDenoise — Type
struct ShrunkDenoise{__T_alpha} <: AbstractDenoiseAlgorithm

Shrinks the off-diagonal part of the noise block of a matrix towards zero, and keeps the diagonal of that block. alpha is the weight that the algorithm keeps on that off-diagonal part, which is the reverse of what its name suggests. So alpha = 0 is total shrinkage, and alpha = 1 returns the input unchanged. The default alpha = 0.0 is total shrinkage.

Mathematical definition

The algorithm splits the spectrum at the Marčenko-Pastur upper bound. It rebuilds the signal block whole, and scales only the off-diagonal part of the noise block by $\alpha$:

\[\begin{align} \mathbf{C}_{\mathrm{signal}} &= \mathbf{V}_{\mathrm{signal}} \, \mathrm{Diag}(\boldsymbol{\lambda}_{\mathrm{signal}}) \, \mathbf{V}_{\mathrm{signal}}^\intercal\,, \\ \mathbf{C}_{\mathrm{noise}} &= \mathbf{V}_{\mathrm{noise}} \, \mathrm{Diag}(\boldsymbol{\lambda}_{\mathrm{noise}}) \, \mathbf{V}_{\mathrm{noise}}^\intercal\,, \\ \tilde{\mathbf{X}} &= \mathbf{C}_{\mathrm{signal}} + \alpha \mathbf{C}_{\mathrm{noise}} + (1 - \alpha) \, \mathrm{Diag}(\mathbf{C}_{\mathrm{noise}})\,. \end{align}\]

Where:

  • $\lambda_i$: $i$-th eigenvalue of the input matrix.
  • $\lambda_+$: Marčenko-Pastur upper bound of the noise band. An eigenvalue is noise when $\lambda_i \leq \lambda_+$, and signal when $\lambda_i > \lambda_+$.
  • $\mathbf{V}_{\mathrm{signal}}$: Eigenvector block of the signal eigenpairs.
  • $\mathbf{V}_{\mathrm{noise}}$: The block of the eigenvectors of the noise eigenpairs.
  • $\boldsymbol{\lambda}_{\mathrm{signal}}$: Signal eigenvalues.
  • $\boldsymbol{\lambda}_{\mathrm{noise}}$: The noise eigenvalues.
  • $\mathbf{C}_{\mathrm{signal}}$: Reconstruction from the signal eigenpairs alone.
  • $\mathbf{C}_{\mathrm{noise}}$: The reconstruction from the noise eigenpairs alone.
  • $\alpha \in [0, 1]$: The weight kept on the off-diagonal part of the noise block. $\alpha = 0$ keeps only its diagonal, which is total shrinkage. $\alpha = 1$ keeps the block whole, so $\tilde{\mathbf{X}} = \mathbf{X}$.
  • $\tilde{\mathbf{X}}$: Denoised matrix.

The two $\alpha$ weights sum to one on the diagonal, so the reconstruction keeps the diagonal in exact arithmetic.

Section 2.5.2 of [26] states this method, with $\alpha \to 0$ for total shrinkage. A search of the literature found no earlier work that states it. That section does not pin the diagonal, which the reconstruction keeps in exact arithmetic.

Algorithm

The branch of _denoise! that this tag selects runs these steps.

  1. Split vals and vecs at num_factors. The first num_factors entries are the noise block vals_l and vecs_l, and the rest are the signal block vals_r and vecs_r.
  2. Build corr0 from the signal block, which is $\mathbf{C}_{\mathrm{signal}}$.
  3. Build corr1 from the noise block, which is $\mathbf{C}_{\mathrm{noise}}$.
  4. Write corr0 + alpha * corr1 + (1 - alpha) * Diagonal(corr1) into X.
  5. Set the diagonal of X to one. The reconstruction preserves the diagonal in exact arithmetic, so this step sheds the round-off of the eigendecomposition. This branch reconstructs directly and not through StatsBase.cov2cor, so it is the only branch that must pin its own diagonal.

Fields

  • alpha: Weight kept on the off-diagonal part of the noise block, $\alpha \in [0, 1]$. It is the weight that the algorithm keeps, not the weight that it removes. 0 keeps only the diagonal of that block, which is total shrinkage. 1 keeps the block whole and returns the input unchanged.

Constructors

ShrunkDenoise(;    alpha::Number = 0.0,) -> ShrunkDenoise

Keywords correspond to the struct's fields.

Validation

  • 0 <= alpha <= 1.

Examples

julia> ShrunkDenoise(; alpha = 0.5)ShrunkDenoise  alpha ┴ Float64: 0.5

Related

References

  • [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 2.5.2, Snippet 2.6.
  • [27] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967). Section 1, Example 1.
source
PortfolioOptimisers.Denoise — Type
struct Denoise{__T_pdm, __T_alg, __T_args, __T_kwargs, __T_kernel, __T_m, __T_n} <: AbstractDenoiseEstimator

Applies a denoising algorithm to a covariance or correlation matrix.

Denoise holds the parameters that denoise! and denoise read. These are the denoising algorithm, the arguments of the search for the noise variance, the settings of the average shifted histogram, and an optional positive definite projection.

Fields

  • pdm: Optional positive definite matrix estimator.
  • alg: The denoising algorithm.

Constructors

Denoise(;    pdm::Option{<:AbstractPosdefEstimator} = Posdef(),    alg::AbstractDenoiseAlgorithm = ShrunkDenoise(),    args::Tuple = (),    kwargs::NamedTuple = (;),    kernel = AverageShiftedHistograms.Kernels.gaussian,    m::Integer = 10,    n::Integer = 1000) -> Denoise

Keywords correspond to the struct's fields.

Validation

  • m > 1.
  • n > 1.

Examples

julia> Denoise()Denoise     pdm ┼ Posdef         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton         │   kwargs ┴ @NamedTuple{}: NamedTuple()     alg ┼ ShrunkDenoise         │   alpha ┴ Float64: 0.0    args ┼ Tuple{}: ()  kwargs ┼ @NamedTuple{}: NamedTuple()  kernel ┼ typeof(AverageShiftedHistograms.Kernels.gaussian): AverageShiftedHistograms.Kernels.gaussian       m ┼ Int64: 10       n ┴ Int64: 1000julia> Denoise(; alg = SpectralDenoise(), m = 20, n = 500)Denoise     pdm ┼ Posdef         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton         │   kwargs ┴ @NamedTuple{}: NamedTuple()     alg ┼ SpectralDenoise()    args ┼ Tuple{}: ()  kwargs ┼ @NamedTuple{}: NamedTuple()  kernel ┼ typeof(AverageShiftedHistograms.Kernels.gaussian): AverageShiftedHistograms.Kernels.gaussian       m ┼ Int64: 20       n ┴ Int64: 500

Related

References

  • [27] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967). Section 1, Example 1.
  • [29] L. Laloux, P. Cizeau, J.-P. Bouchaud and M. Potters. Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83, 1467–1470 (1999). Equation 3.
  • [30] M. López de Prado. A robust estimator of the efficient frontier. SSRN Electronic Journal (2019). Section 4.2, Snippet 2.
  • [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 2.4, Snippet 2.4.
source
PortfolioOptimisers.denoise! — Function
denoise!(dn::Option{<:AbstractDenoiseEstimator}, X::MatNum, q::Number) -> MatNum

Denoises a covariance or correlation matrix in place under a Denoise estimator.

When the diagonal of X is not all ones, the function converts X to a correlation matrix, applies the algorithm, and scales the result back to a covariance.

Mathematical definition

The spectrum of $\mathbf{X}$ is split at the Marčenko-Pastur upper bound:

\[\begin{align} \lambda_{+} &= \sigma^2 \left(1 + \sqrt{\frac{1}{q}}\right)^2\,. \end{align}\]

Where:

  • $\lambda_+$: Marčenko-Pastur upper bound of the noise band. An eigenvalue is noise when $\lambda_i \leq \lambda_+$, and signal when $\lambda_i > \lambda_+$.
  • $\sigma^2$: Variance attributed to noise. A correlation matrix has $\sigma^2 = 1$.
  • $q = T/N$: Effective sample ratio, observations to assets.

This function adds only the split. The closed form of dn.alg states what happens on each side of it.

Algorithm

  1. Check that X is square.
  2. When X has a zero variance, set its row and column to zero and denoise the block of the positive variances with a recursive call, through zero_variance_block!, and return X. A zero variance has no correlation, so step 3 cannot standardise it. The ratio q reads the number of assets, so the block of n assets takes the ratio q * N / n, as matrix_processing_block! does.
  3. Read the diagonal of X into s. When any entry of s is not one, X is a covariance matrix. Then replace s with its square roots and convert X to a correlation matrix with StatsBase.cov2cor!. The test is any(!isone, s), so it is the value of the diagonal that decides, never the type of X.
  4. Eigendecompose X into the ascending eigenvalues vals and the eigenvectors vecs.
  5. Fit the Marčenko-Pastur density to vals with find_max_eval to get max_val, the upper edge of the noise band.
  6. Count the eigenvalues that are not larger than max_val. The count is num_factors, the number of noise eigenvalues.
  7. Rebuild X from the split spectrum with _denoise!, through the branch that dn.alg selects.
  8. Repair the rebuilt matrix with posdef!, under dn.pdm.
  9. When step 3 converted a covariance matrix, convert X back with StatsBase.cor2cov!. The standard deviations are the ones read in step 3, so the original diagonal returns exactly.

Arguments

  • dn: Optional matrix denoising estimator.
    • ::Denoise: Denoises X in place under dn.
    • ::Nothing: Returns X unchanged.
  • X: Covariance-like or correlation-like matrix assets × assets.
  • q: The effective sample ratio observations / assets, which sets the threshold of the spectrum.

Validation

  • assert_matrix_issquare checks that X is square. The ::Nothing method returns before the check, so a dn of nothing accepts any X.

Returns

  • X::MatNum: The input matrix X, modified in place.

Examples

julia> using StableRNGsjulia> rng = StableRNG(123456789);julia> X = rand(rng, 10, 5);julia> X = X' * X5×5 Matrix{Float64}: 3.29494  2.0765   1.73334  2.01524  1.77493 2.0765   2.46967  1.39953  1.97242  2.07886 1.73334  1.39953  1.90712  1.17071  1.30459 2.01524  1.97242  1.17071  2.24818  1.87091 1.77493  2.07886  1.30459  1.87091  2.44414julia> denoise!(Denoise(), X, 10 / 5)5×5 Matrix{Float64}: 3.29494  2.28883  1.70633  2.12343  2.17377 2.28883  2.46967  1.59575  1.98583  2.0329 1.70633  1.59575  1.90712  1.48044  1.51553 2.12343  1.98583  1.48044  2.24818  1.886 2.17377  2.0329   1.51553  1.886    2.44414

Related

References

  • [27] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967). Section 1, Example 1.
  • [29] L. Laloux, P. Cizeau, J.-P. Bouchaud and M. Potters. Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83, 1467–1470 (1999). Equation 3.
  • [30] M. López de Prado. A robust estimator of the efficient frontier. SSRN Electronic Journal (2019). Section 4.2, Snippet 2.
  • [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 2.4, Snippet 2.4.
source
PortfolioOptimisers.denoise — Function
denoise(dn::Option{<:AbstractDenoiseEstimator}, X::MatNum, q::Number) -> MatNum

Out-of-place version of denoise!.

Algorithm

  1. Copy X.
  2. Apply denoise! to the copy, and return it. The function never changes the input.

Arguments

  • dn: Optional matrix denoising estimator.
    • ::Denoise: Denoises a copy of X under dn.
    • ::Nothing: Returns X unchanged.
  • X: Covariance-like or correlation-like matrix assets × assets.
  • q: The effective sample ratio observations / assets, which sets the threshold of the spectrum.

Returns

  • X::MatNum: A new matrix equal to the denoised version of the input.

Examples

julia> using StableRNGsjulia> rng = StableRNG(123456789);julia> X = rand(rng, 10, 5);       X = X' * X;julia> Xd = denoise(Denoise(), X, 10 / 5);julia> size(Xd)(5, 5)

Related

References

  • [27] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967). Section 1, Example 1.
  • [29] L. Laloux, P. Cizeau, J.-P. Bouchaud and M. Potters. Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83, 1467–1470 (1999). Equation 3.
  • [30] M. López de Prado. A robust estimator of the efficient frontier. SSRN Electronic Journal (2019). Section 4.2, Snippet 2.
  • [26] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Section 2.4, Snippet 2.4.
source

References

[12]
D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
[26]
M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020).
[27]
V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967).
[28]
V. Plerou, P. Gopikrishnan, B. Rosenow, L. A. Amaral, T. Guhr and H. E. Stanley. Random matrix approach to cross correlations in financial data. Phys. Rev. E 65, 066126 (2002).
[29]
L. Laloux, P. Cizeau, J.-P. Bouchaud and M. Potters. Noise dressing of financial correlation matrices. Phys. Rev. Lett. 83, 1467–1470 (1999).
[30]
M. López de Prado. A robust estimator of the efficient frontier. SSRN Electronic Journal (2019).