Detone

Financial data is often responds to broad market conditions. This market-wide behaviour can obscure specific correlation signals. By removing the largest n eigenvalues, the idiosyncratic relationships between assets are allowed to shine through [1].

Detoned matrices may be non-positive definite, so they can be unsuitable for traditional optimisations, but they can be quite effective for clustering ones.

PortfolioOptimisers.AbstractDetoneEstimator Type

abstract type AbstractDetoneEstimator <: AbstractEstimator

Abstract supertype for all detoning estimators in PortfolioOptimisers.jl.

All concrete and/or abstract types representing detoning estimators should be subtypes of AbstractDetoneEstimator.

Interfaces

In order to implement a new detoning estimator which will work seamlessly with the library, subtype AbstractDetoneEstimator with all necessary parameters as part of the struct, and implement the following methods:

• detone!(dt::AbstractDetoneEstimator, X::MatNum) -> MatNum: In-place detoning.

• detone(dt::AbstractDetoneEstimator, X::MatNum) -> MatNum: Optional out-of-place detoning.

Arguments

• dt: Matrix detoning estimator.

• X: Covariance-like or correlation-like matrix features × features.

Returns

• X::MatNum: The detoned input matrix X.

Examples

We can create a dummy detoning estimator as follows:

julia> struct MyDetoneEstimator <: PortfolioOptimisers.AbstractDetoneEstimator end

julia> function PortfolioOptimisers.detone!(dt::MyDetoneEstimator, X::PortfolioOptimisers.MatNum)
           # Implement your in-place detoning estimator here.
           println("Detoning matrix in-place...")
           return X
       end

julia> function PortfolioOptimisers.detone(dt::MyDetoneEstimator, X::PortfolioOptimisers.MatNum)
           X = copy(X)
           println("Copy X...")
           detone!(dt, X)
           return X
       end

julia> detone!(MyDetoneEstimator(), [1.0 2.0; 2.0 1.0])
Detoning matrix in-place...
2×2 Matrix{Float64}:
 1.0  2.0
 2.0  1.0

julia> detone(MyDetoneEstimator(), [1.0 2.0; 2.0 1.0])
Copy X...
Detoning matrix in-place...
2×2 Matrix{Float64}:
 1.0  2.0
 2.0  1.0

Related

• Detone

• detone!

• detone

source

PortfolioOptimisers.Detone Type

struct Detone{__T_pdm, __T_n} <: AbstractDetoneEstimator

A concrete detoning estimator for removing the largest n principal components (market modes) from a covariance or correlation matrix in detone! and detone.

For financial data, the leading principal components often represent market-wide movements that can obscure asset-specific signals. The Detone estimator allows users to specify the number of these leading components to remove, thereby enhancing the focus on idiosyncratic relationships between market members [1].

Detoned matrices may not be suitable for non-clustering optimisations because it can make the matrix non-positive definite. However, they can be quite effective for clustering optimsations.

Fields

• pdm: Optional positive definite matrix estimator.

• n: Number of leading principal components to remove.

Constructors

Detone(;
    pdm::Option{<:Posdef} = Posdef(),
    n::Integer = 1,
) -> Detone

Keywords correspond to the struct's fields.

Validation

• n > 0.

Examples

julia> Detone(; n = 2)
Detone
  pdm ┼ Posdef
      │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton
      │   kwargs ┴ @NamedTuple{}: NamedTuple()
    n ┴ Int64: 2

Related

• detone!

• detone

References

• [1] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Chapter 2.

source

PortfolioOptimisers.detone! Function

detone!(dt::Option{<:Detone}, X::MatNum) -> MatNum

In-place removal of the top n principal components (market modes) from a covariance or correlation matrix.

For matrices without unit diagonal, the function converts them into correlation matrices i.e. matrices with unit diagonal, applies the algorithm, and rescales them back.

Mathematical definition

The detoned matrix removes the $n$ largest eigenmodes:

$$\begin{array}{rl}\tilde{\mathbf{X}}& =\mathbf{X}-\sum _{k=N-n+1}^N{\lambda }_k{\mathit{v}}_k{\mathit{v}}_k^{⊺}.\end{array}$$

Where:

• $\tilde{\mathbf{X}}$: Detoned matrix.

• $\mathbf{X}$: Original correlation or covariance matrix.

• ${\lambda }_k$: $k$-th largest eigenvalue of $\mathbf{X}$.

• ${\mathit{v}}_k$: $k$-th largest eigenvector of $\mathbf{X}$.

• $n$: Number of eigenmodes (market modes) to remove.

• $N$: Number of assets.

Arguments

• dt: Optional matrix detoning estimator.

  • ::Detone: The top n principal components are removed from X in-place.

  • ::Nothing: No-op.

• X: Covariance-like or correlation-like matrix features × features.

Validation

• 0 < dt.n <= size(X, 2).

Returns

• X::MatNum: The input matrix X is modified in-place.

Details

• Asserts the number of elements to remove is within the valid range.

• If X is not a correlation matrix, it is converted to one before applying the algorithm.

• Performs an eigenvector decomposition of X.

• Removes the top n principal components (market modes) from the eigenvalues and eigenvectors of X.

• Reconstructs the correlation matrix X in-place from the modified eigenvalues vals and eigenvectors vecs.

• If X was not originally a correlation matrix, it is converted back.

• Returns X.

Examples

julia> using StableRNGs

julia> rng = StableRNG(123456789);

julia> X = rand(rng, 10, 5);

julia> X = X' * X
5×5 Matrix{Float64}:
 3.29494  2.0765   1.73334  2.01524  1.77493
 2.0765   2.46967  1.39953  1.97242  2.07886
 1.73334  1.39953  1.90712  1.17071  1.30459
 2.01524  1.97242  1.17071  2.24818  1.87091
 1.77493  2.07886  1.30459  1.87091  2.44414

julia> detone!(Detone(), X)
5×5 Matrix{Float64}:
  3.29494    -1.14673     0.0868439  -0.502106   -1.71581
 -1.14673     2.46967    -0.876289   -0.0864304   0.274663
  0.0868439  -0.876289    1.90712    -1.18851    -0.750345
 -0.502106   -0.0864304  -1.18851     2.24818    -0.0774753
 -1.71581     0.274663   -0.750345   -0.0774753   2.44414

Related

• detone

• Detone

• MatNum

• Option

• Posdef

References

• [1] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Chapter 2.

source

PortfolioOptimisers.detone Function

detone(dt::Option{<:Detone}, X::MatNum) -> MatNum

Out-of-place version of detone!.

Arguments

• dt: Optional matrix detoning estimator.

  • ::Detone: The top n principal components are removed from a copy of X.

  • ::Nothing: No-op, returns X unchanged.

• X: Covariance-like or correlation-like matrix features × features.

Returns

• X::MatNum: A new matrix equal to the detoned version of the input.

Examples

julia> using StableRNGs

julia> rng = StableRNG(123456789);

julia> X = rand(rng, 10, 5);
       X = X' * X;

julia> Xd = detone(Detone(), X);

julia> size(Xd)
(5, 5)

Related

• detone!

• Detone

• MatNum

• Option

• Posdef

References

• [1] M. M. De Prado. Machine learning for asset managers (Cambridge University Press, 2020). Chapter 2.

source