PortfolioOptimisers.jl

Appearance

6

Matrix processing ​

Co-moment matrices can be post-processed after being computed. These processes are often complementary but there is no set order.

At the base level, we have four possible post processing steps:

  1. Positive definite projection.

  2. Denoising.

  3. Detoning.

  4. Custom process.

The only set order is that positive definite projection should come first. This is because the other post-processing methods work best with positive definite matrices, and may use positive definite projection internally.

Aside from this, there is no set canonical order, the closest to a heuristic we can justify is to denoise before detoning. As such we provide all permutations of ordering for the last three steps.

PortfolioOptimisers.AbstractMatrixProcessingEstimator Type
julia
abstract type AbstractMatrixProcessingEstimator <: AbstractEstimator end

Abstract supertype for all matrix processing estimator types in PortfolioOptimisers.jl.

All concrete types that implement matrix processing routines—such as covariance matrix cleaning, denoising, or detoning—should subtype AbstractMatrixProcessingEstimator. This enables a consistent interface for matrix processing estimators throughout the package.

Related

source
PortfolioOptimisers.AbstractMatrixProcessingAlgorithm Type
julia
abstract type AbstractMatrixProcessingAlgorithm <: AbstractAlgorithm end

Abstract supertype for all matrix processing algorithm types in PortfolioOptimisers.jl.

All concrete types that implement a specific matrix processing algorithm (e.g., custom cleaning or transformation routines) should subtype AbstractMatrixProcessingAlgorithm. This enables flexible extension and dispatch of matrix processing routines.

Related

source
PortfolioOptimisers.AbstractMatrixProcessingOrder Type
julia
abstract type AbstractMatrixProcessingOrder <: AbstractAlgorithm end

Abstract supertype for matrix processing order types in PortfolioOptimisers.jl.

All concrete types that specify the order of matrix processing steps—such as denoising, detoning, and algorithm application—should subtype AbstractMatrixProcessingOrder. This enables flexible configuration of the sequence in which matrix processing operations are applied within the matrix processing pipeline.

Related Types

source
PortfolioOptimisers.DenoiseDetoneAlg Type
julia
struct DenoiseDetoneAlg <: AbstractMatrixProcessingOrder end

Matrix processing order: Denoising → Detoning → Custom Algorithm.

DenoiseDetoneAlg specifies that matrix processing should be performed in the order of denoising, then detoning, followed by any custom algorithm. This type is used to configure the sequence of operations in matrix processing pipelines.

Related

source
PortfolioOptimisers.DenoiseAlgDetone Type
julia
struct DenoiseAlgDetone <: AbstractMatrixProcessingOrder end

Matrix processing order: Denoising → Custom Algorithm → Detoning.

DenoiseAlgDetone specifies that matrix processing should be performed in the order of denoising, then applying a custom algorithm, followed by detoning. This type is used to configure the sequence of operations in matrix processing pipelines.

Related

source
PortfolioOptimisers.DetoneDenoiseAlg Type
julia
struct DetoneDenoiseAlg <: AbstractMatrixProcessingOrder end

Matrix processing order: Detoning → Denoising → Custom Algorithm.

DetoneDenoiseAlg specifies that matrix processing should be performed in the order of detoning, then denoising, followed by any custom algorithm. This type is used to configure the sequence of operations in matrix processing pipelines.

Related

source
PortfolioOptimisers.DetoneAlgDenoise Type
julia
struct DetoneAlgDenoise <: AbstractMatrixProcessingOrder end

Matrix processing order: Detoning → Custom Algorithm → Denoising.

DetoneAlgDenoise specifies that matrix processing should be performed in the order of detoning, then applying a custom algorithm, followed by denoising. This type is used to configure the sequence of operations in matrix processing pipelines.

Related

source
PortfolioOptimisers.AlgDenoiseDetone Type
julia
struct AlgDenoiseDetone <: AbstractMatrixProcessingOrder end

Matrix processing order: Custom Algorithm → Denoising → Detoning.

AlgDenoiseDetone specifies that matrix processing should be performed in the order of applying a custom algorithm, then denoising, followed by detoning. This type is used to configure the sequence of operations in matrix processing pipelines.

Related

source
PortfolioOptimisers.AlgDetoneDenoise Type
julia
struct AlgDetoneDenoise <: AbstractMatrixProcessingOrder end

Matrix processing order: Custom Algorithm → Detoning → Denoising.

AlgDetoneDenoise specifies that matrix processing should be performed in the order of applying a custom algorithm, then detoning, followed by denoising. This type is used to configure the sequence of operations in matrix processing pipelines.

Related

source
PortfolioOptimisers.DenoiseDetoneAlgMatrixProcessing Type
julia
struct DenoiseDetoneAlgMatrixProcessing{T1, T2, T3, T4, T5} <: AbstractMatrixProcessingEstimator
    pdm::T1
    denoise::T2
    detone::T3
    alg::T4
    order::T5
end

A flexible container type for configuring and applying matrix processing routines in PortfolioOptimisers.jl.

DenoiseDetoneAlgMatrixProcessing encapsulates all steps required for processing covariance or correlation matrices, including positive definiteness enforcement, denoising, detoning, and optional custom matrix processing algorithms via matrix_processing! and matrix_processing. This estimator allows users to build complex matrix processing pipelines tailored to their specific needs.

Fields

  • pdm: Positive definite matrix estimator, or nothing to skip.

  • denoise: Denoising estimator, or nothing to skip.

  • detone: Detoning estimator, or nothing to skip.

  • alg: Optional custom matrix processing algorithm, or nothing to skip.

  • order: Specifies the order in which denoising, detoning, and custom algorithm steps are applied.

Constructor

julia
DenoiseDetoneAlgMatrixProcessing(; pdm::Option{<:Posdef} = Posdef(),
                        denoise::Option{<:Denoise} = nothing,
                        detone::Option{<:Detone} = nothing,
                        alg::Option{<:AbstractMatrixProcessingAlgorithm} = nothing,
                        order::AbstractMatrixProcessingOrder = DenoiseDetoneAlg())

Keyword arguments correspond to the fields above.

Examples

julia
julia> DenoiseDetoneAlgMatrixProcessing()
DenoiseDetoneAlgMatrixProcessing
      pdm ┼ Posdef
          │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton
          │   kwargs ┴ @NamedTuple{}: NamedTuple()
  denoise ┼ nothing
   detone ┼ nothing
      alg ┼ nothing
    order ┴ DenoiseDetoneAlg()

julia> DenoiseDetoneAlgMatrixProcessing(; denoise = Denoise(), detone = Detone(; n = 2))
DenoiseDetoneAlgMatrixProcessing
      pdm ┼ Posdef
          │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton
          │   kwargs ┴ @NamedTuple{}: NamedTuple()
  denoise ┼ Denoise
          │      alg ┼ ShrunkDenoise
          │          │   alpha ┴ Float64: 0.0
          │     args ┼ Tuple{}: ()
          │   kwargs ┼ @NamedTuple{}: NamedTuple()
          │   kernel ┼ typeof(AverageShiftedHistograms.Kernels.gaussian): AverageShiftedHistograms.Kernels.gaussian
          │        m ┼ Int64: 10
          │        n ┼ Int64: 1000
          │      pdm ┼ Posdef
          │          │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton
          │          │   kwargs ┴ @NamedTuple{}: NamedTuple()
   detone ┼ Detone
          │     n ┼ Int64: 2
          │   pdm ┼ Posdef
          │       │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton
          │       │   kwargs ┴ @NamedTuple{}: NamedTuple()
      alg ┼ nothing
    order ┴ DenoiseDetoneAlg()

Related

References

  • [1] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Chapter 2.

  • [2] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967).

source
PortfolioOptimisers.matrix_processing! Function
julia
matrix_processing!(mp::AbstractMatrixProcessingEstimator, sigma::MatNum, X::MatNum, args...;
                   kwargs...)
matrix_processing!(::Nothing, args...; kwargs...)

In-place processing of a covariance or correlation matrix.

Arguments

  • mp::AbstractMatrixProcessingEstimator: Matrix processing estimator specifying the pipeline.

  • sigma: Covariance or correlation matrix to be processed (modified in-place).

  • X: Data matrix (observations × assets) used for denoising and detoning.

  • args...: Additional positional arguments passed to custom algorithms.

  • kwargs...: Additional keyword arguments passed to custom algorithms.

Returns

  • nothing. The input matrix sigma is modified in-place.

Related

source
julia
matrix_processing!(mp::DenoiseDetoneAlgMatrixProcessing, sigma::MatNum, X::MatNum, args...;
                   kwargs...)

In-place matrix processing pipeline using the DenoiseDetoneAlg order.

This method applies a sequence of matrix processing steps to the input covariance or correlation matrix sigma, modifying it in-place. The steps are performed in the following order: positive definiteness enforcement, denoising, detoning, and an optional custom algorithm. The order is determined by the DenoiseDetoneAlg type.

Arguments

  • mp: Matrix processing estimator specifying the pipeline and order.

  • sigma: Covariance or correlation matrix to be processed (modified in-place).

  • X: Data matrix (observations × assets) used for denoising and detoning.

  • args...: Additional positional arguments passed to custom algorithms.

  • kwargs...: Additional keyword arguments passed to custom algorithms.

Returns

  • nothing. The input matrix sigma is modified in-place.

Details

  • Applies positive definiteness enforcement using mp.pdm.

  • Applies denoising using mp.denoise and the ratio T / N from X.

  • Applies detoning using mp.detone.

  • Applies an optional custom matrix processing algorithm using mp.alg.

  • The order of operations depends on the specific type of mp.order.

Examples

julia
julia> using StableRNGs, Statistics

julia> rng = StableRNG(123456789);

julia> X = rand(rng, 10, 5);

julia> sigma = cov(X)
5×5 Matrix{Float64}:
  0.132026     0.0022567   0.0198243    0.00359832  -0.00743829
  0.0022567    0.0514194  -0.0131242    0.004123     0.0312379
  0.0198243   -0.0131242   0.0843837   -0.0325342   -0.00609624
  0.00359832   0.004123   -0.0325342    0.0424332    0.0152574
 -0.00743829   0.0312379  -0.00609624   0.0152574    0.0926441

julia> matrix_processing!(DenoiseDetoneAlgMatrixProcessing(; denoise = Denoise()), sigma, X)

julia> sigma
5×5 Matrix{Float64}:
 0.132026  0.0        0.0        0.0        0.0
 0.0       0.0514194  0.0        0.0        0.0
 0.0       0.0        0.0843837  0.0        0.0
 0.0       0.0        0.0        0.0424332  0.0
 0.0       0.0        0.0        0.0        0.0926441

julia> sigma = cov(X)
5×5 Matrix{Float64}:
  0.132026     0.0022567   0.0198243    0.00359832  -0.00743829
  0.0022567    0.0514194  -0.0131242    0.004123     0.0312379
  0.0198243   -0.0131242   0.0843837   -0.0325342   -0.00609624
  0.00359832   0.004123   -0.0325342    0.0424332    0.0152574
 -0.00743829   0.0312379  -0.00609624   0.0152574    0.0926441

julia> matrix_processing!(DenoiseDetoneAlgMatrixProcessing(; detone = Detone()), sigma, X)

julia> sigma
5×5 Matrix{Float64}:
 0.132026    0.0124802   0.0117303    0.0176194    0.0042142
 0.0124802   0.0514194   0.0273105   -0.0290864    0.0088165
 0.0117303   0.0273105   0.0843837   -0.00279296   0.0619156
 0.0176194  -0.0290864  -0.00279296   0.0424332   -0.0242252
 0.0042142   0.0088165   0.0619156   -0.0242252    0.0926441

Related

References

  • [1] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Chapter 2.

  • [2] V. A. Marčenko and L. A. Pastur. Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik 1, 457 (1967).

source
PortfolioOptimisers.matrix_processing Function
julia
matrix_processing(mp::AbstractMatrixProcessingEstimator, sigma::MatNum, X::MatNum, args...;
                  kwargs...)
matrix_processing(::Nothing, args...; kwargs...)

Out-of-place version of matrix_processing!.

Related

source
PortfolioOptimisers.matrix_processing_algorithm! Method
julia
matrix_processing_algorithm!(::Nothing, args...; kwargs...)

No-op fallback for matrix processing algorithm routines.

These methods are called internally when no matrix processing algorithm is specified (i.e., when the algorithm argument is nothing). They perform no operation and return nothing, ensuring that the matrix processing pipeline can safely skip optional algorithmic steps.

Arguments

  • ::Nothing: Indicates that no algorithm is provided.

  • args...: Additional positional arguments (ignored).

  • kwargs...: Additional keyword arguments (ignored).

Returns

  • nothing.

Related

source
PortfolioOptimisers.matrix_processing_algorithm Method
julia
matrix_processing_algorithm(::Nothing, args...; kwargs...)

Same as matrix_processing_algorithm!, but meant for returning a new matrix instead of modifying it in-place.

Related

source