Denoise

PortfolioOptimisers.SpectralDenoise — Type
SpectralDenoise <: AbstractDenoiseAlgorithm

A denoising algorithm that sets the smallest num_factors eigenvalues of a covariance or correlation matrix to zero, effectively removing the principal components relating to random noise according to random matrix theory-based approaches.

Examples

julia> alg = SpectralDenoise()
SpectralDenoise()

Related

source
PortfolioOptimisers.FixedDenoise — Type
FixedDenoise <: AbstractDenoiseAlgorithm

A denoising algorithm that replaces the smallest num_factors eigenvalues of a covariance or correlation matrix with their average, effectively averaging the principal components relating to random noise according to random matrix theory-based approaches.

Examples

julia> alg = FixedDenoise()
FixedDenoise()

Related

source
PortfolioOptimisers.ShrunkDenoise — Type
struct ShrunkDenoise{T1} <: AbstractDenoiseAlgorithm
    alpha::T1
end

A denoising algorithm that shrinks the smallest num_factors eigenvalues of a covariance or correlation matrix towards their diagonal, controlled by the shrinkage parameter alpha. This approach interpolates between no shrinkage (alpha = 0) and full shrinkage (alpha = 1), providing a flexible way to regularize noisy eigenvalues.

Constructor

ShrunkDenoise(; alpha::Real = 0.0)

Creates a new ShrunkDenoise with the specified shrinkage parameter alpha.

Related

source
PortfolioOptimisers.ShrunkDenoise — Method
ShrunkDenoise(; alpha::Real = 0.0)

Constructor for ShrunkDenoise.

Arguments

  • alpha: The shrinkage parameter controlling the degree of shrinkage applied to the smallest eigenvalues. Must be in the range [0, 1], where 0 means no shrinkage and 1 means full shrinkage.

Validation

  • Throws an error if alpha is not in [0, 1].

Examples

julia> alg = ShrunkDenoise(; alpha = 0.5)
ShrunkDenoise
  alpha | Float64: 0.5
source
PortfolioOptimisers.Denoise — Type
struct Denoise{T1, T2, T3, T4, T5, T6} <: AbstractDenoiseEstimator
    alg::T1
    args::T2
    kwargs::T3
    kernel::T4
    m::T5
    n::T6
end

A flexible container type for configuring and applying denoising algorithms to covariance or correlation matrices in PortfolioOptimisers.jl.

Denoise encapsulates all parameters required for matrix denoising, including the kernel and its arguments for spectral density estimation, the denoising algorithm, and matrix dimensions. It is the standard estimator type for denoising routines and supports a variety of algorithms (SpectralDenoise, FixedDenoise, ShrunkDenoise).

Fields

Constructor

Denoise(; alg::AbstractDenoiseAlgorithm = ShrunkDenoise(),
         m::Integer = 10,
         n::Integer = 1000,
         kernel = AverageShiftedHistograms.Kernels.gaussian,
         args::Tuple = (),
         kwargs::NamedTuple = (;))

Keyword arguments correspond to the fields above. The constructor infers types and sets defaults for robust denoising.

Related

source
PortfolioOptimisers.Denoise — Method
Denoise(; kernel = AverageShiftedHistograms.Kernels.gaussian,
          args::Tuple = (),
          kwargs::NamedTuple = (;),
          alg::AbstractDenoiseAlgorithm = ShrunkDenoise(),
          m::Integer = 10,
          n::Integer = 1000)

Construct a Denoise object, configuring all parameters for matrix denoising in PortfolioOptimisers.jl.

Arguments

Validation

  • Types are inferred from provided arguments.
  • All keyword arguments correspond to fields in Denoise.

Examples

julia> de = Denoise(;)
Denoise
     alg | ShrunkDenoise
         |   alpha | Float64: 0.0
    args | Tuple{}: ()
  kwargs | @NamedTuple{}: NamedTuple()
  kernel | typeof(AverageShiftedHistograms.Kernels.gaussian): AverageShiftedHistograms.Kernels.gaussian
       m | Int64: 10
       n | Int64: 1000

julia> de = Denoise(; alg = SpectralDenoise(), m = 20, n = 500)
Denoise
     alg | SpectralDenoise()
    args | Tuple{}: ()
  kwargs | @NamedTuple{}: NamedTuple()
  kernel | typeof(AverageShiftedHistograms.Kernels.gaussian): AverageShiftedHistograms.Kernels.gaussian
       m | Int64: 20
       n | Int64: 500

Related

source
PortfolioOptimisers.denoise! — Function
denoise!(de::Denoise, X::AbstractMatrix, q::Real, pdm::Union{Nothing, <:Posdef} = Posdef())
denoise!(::Nothing, args...)

In-place denoising of a covariance or correlation matrix using a Denoise estimator.

  • If de is nothing, this is a no-op and returns nothing.
  • If de is a Denoise object, the specified denoising algorithm is applied to X in-place. Optionally, a Posdef can be provided to ensure the output is positive definite.

Arguments

  • de::Denoise: The denoising estimator specifying the algorithm and kernel parameters.
  • pdm::Union{Nothing, <:Posdef}: Optional positive definite matrix estimator. If provided, ensures the output is positive definite.
  • X::AbstractMatrix: The covariance or correlation matrix to be denoised (modified in-place).
  • q::Real: The effective sample ratio (e.g., n_obs / n_assets), used for spectral thresholding.

ReturnsResult

  • nothing. The input matrix X is modified in-place.

Validation

  • If X is a covariance matrix, it is internally converted to a correlation matrix for denoising and then rescaled.
  • The number of factors is determined automatically from the spectrum and kernel parameters.
  • If pdm is provided, the result is projected to the nearest positive definite matrix.

Examples

julia> using StableRNGs

julia> rng = StableRNG(123456789);

julia> X = rand(rng, 10, 5);

julia> X = X' * X
5×5 Matrix{Float64}:
 3.29494  2.0765   1.73334  2.01524  1.77493
 2.0765   2.46967  1.39953  1.97242  2.07886
 1.73334  1.39953  1.90712  1.17071  1.30459
 2.01524  1.97242  1.17071  2.24818  1.87091
 1.77493  2.07886  1.30459  1.87091  2.44414

julia> denoise!(Denoise(), X, 10 / 5)

julia> X
5×5 Matrix{Float64}:
 3.29494  2.28883  1.70633  2.12343  2.17377
 2.28883  2.46967  1.59575  1.98583  2.0329
 1.70633  1.59575  1.90712  1.48044  1.51553
 2.12343  1.98583  1.48044  2.24818  1.886
 2.17377  2.0329   1.51553  1.886    2.44414

Related

source
PortfolioOptimisers.denoise — Function
denoise(de::Denoise, X::AbstractMatrix, q::Real, pdm::Union{Nothing, <:Posdef} = Posdef())
denoise(::Nothing, args...)

Same as denoise!, but returns a new matrix instead of modifying X in-place.

  • If de is nothing, this is a no-op and returns nothing.

Examples

julia> using StableRNGs

julia> rng = StableRNG(123456789);

julia> X = rand(rng, 10, 5);

julia> X = X' * X
5×5 Matrix{Float64}:
 3.29494  2.0765   1.73334  2.01524  1.77493
 2.0765   2.46967  1.39953  1.97242  2.07886
 1.73334  1.39953  1.90712  1.17071  1.30459
 2.01524  1.97242  1.17071  2.24818  1.87091
 1.77493  2.07886  1.30459  1.87091  2.44414

julia> Xd = denoise(Denoise(), X, 10 / 5)
5×5 Matrix{Float64}:
 3.29494  2.28883  1.70633  2.12343  2.17377
 2.28883  2.46967  1.59575  1.98583  2.0329
 1.70633  1.59575  1.90712  1.48044  1.51553
 2.12343  1.98583  1.48044  2.24818  1.886
 2.17377  2.0329   1.51553  1.886    2.44414

Related

source
PortfolioOptimisers.AbstractDenoiseEstimator — Type
AbstractDenoiseEstimator <: AbstractEstimator

Abstract supertype for all denoising estimator types in PortfolioOptimisers.jl.

All concrete types that implement denoising of covariance or correlation matrices (e.g., via spectral, fixed, or shrinkage methods) should subtype AbstractDenoiseEstimator. This enables a consistent interface for denoising routines throughout the package.

Related

source
PortfolioOptimisers._denoise! — Function
_denoise!(alg::SpectralDenoise, X::AbstractMatrix, vals::AbstractVector, vecs::AbstractMatrix, num_factors::Integer)
_denoise!(alg::FixedDenoise, X::AbstractMatrix, vals::AbstractVector, vecs::AbstractMatrix, num_factors::Integer)
_denoise!(alg::ShrunkDenoise, X::AbstractMatrix, vals::AbstractVector, vecs::AbstractMatrix, num_factors::Integer)

In-place denoising of a covariance or correlation matrix using a specific denoising algorithm.

These methods are called internally by denoise! and denoise when a Denoise estimator is used, and should not typically be called directly.

Arguments

  • alg::SpectralDenoise: Sets the smallest num_factors eigenvalues to zero.
  • alg::FixedDenoise: Replaces the smallest num_factors eigenvalues with their average.
  • alg::ShrunkDenoise: Shrinks the smallest num_factors eigenvalues towards the diagonal, controlled by alg.alpha.
  • X::AbstractMatrix: The matrix to be denoised (modified in-place).
  • vals::AbstractVector: Eigenvalues of X, sorted in ascending order.
  • vecs::AbstractMatrix: Corresponding eigenvectors of X.
  • num_factors::Integer: Number of eigenvalues to treat as noise.

ReturnsResult

  • nothing. The input matrix X is modified in-place.

Validation

  • Assumes vals and vecs are from the eigendecomposition of a correlation matrix.
  • The denoised matrix is projected back to correlation space.

Related

source
PortfolioOptimisers.errPDF — Function
errPDF(x, vals; kernel = AverageShiftedHistograms.Kernels.gaussian, m = 10, n = 1000, q = 1000)

Compute the sum of squared errors (SSE) between the theoretical Marčenko–Pastur (MP) eigenvalue density and the empirical eigenvalue density estimated from observed eigenvalues.

This function is used internally to fit the MP distribution to the observed spectrum, as part of the denoising procedure.

Arguments

  • x: Scale parameter for the MP distribution [0, 1].
  • vals::AbstractVector: Observed eigenvalues.
  • kernel: Kernel function for AverageShiftedHistograms.ash.
  • m::Integer: Number of adjacent histograms to smooth over.
  • n::Integer: Number of points in the range of eigenvalues for density estimation.
  • q::Real: Effective sample ratio (e.g., n_obs / n_assets).

ReturnsResult

  • sse::Real: The sum of squared errors between the empirical and theoretical densities.

Validation

  • Assumes vals are eigenvalues of a correlation or covariance matrix.
  • The empirical density is estimated using average shifted histograms.

Related

source
PortfolioOptimisers.find_max_eval — Function
find_max_eval(vals, q; kernel = AverageShiftedHistograms.Kernels.gaussian, m::Integer = 10, n::Integer = 1000, args = (), kwargs = (;))

Estimate the upper edge of the Marčenko–Pastur (MP) distribution for a set of eigenvalues, used to separate signal from noise in random matrix denoising.

This function fits the MP distribution to the observed spectrum by minimizing the sum of squared errors between the empirical and theoretical densities, and returns the estimated maximum eigenvalue for noise.

Arguments

  • vals::AbstractVector: Observed eigenvalues (typically sorted in ascending order).
  • q::Real: Effective sample ratio (e.g., n_obs / n_assets).
  • kernel: Kernel function for AverageShiftedHistograms.ash.
  • m::Integer: Number of adjacent histograms to smooth over.
  • n::Integer: Number of points in the range of eigenvalues for density estimation.
  • args: Additional positional arguments for Optim.optimize.
  • kwargs: Additional keyword arguments for Optim.optimize.

ReturnsResult

  • (e_max, x): Tuple containing the estimated upper edge of the noise eigenvalue spectrum (e_max) and the fitted scale parameter (x).

Validation

  • Uses Optim.optimize to fit the MP distribution.
  • Assumes vals are eigenvalues of a correlation or covariance matrix.

Related

source