GitHub

Positive definite matrix projection

All co-moment matrices are supposed to be positive definite. However, non-positive definite matrices can arise due to collinearity, having fewer observations than observables, and floating point inacuracies. Non-positive definite matrices have zero or negative eigenvalues.

  • Zero eigenvalues imply collinearity between assets, which means the system is underdetermined and therefore a linear or quadratic system involving the matrix does not have a unique solution.
  • Negative eigenvalues imply numerical stability issues, which usually arise from poorly conditioned systems, which for covariance and correlation matrices arise from high collinearity.

In order to obtain unique results and improve numerical stability, these non-positive definite matrices can be projected to the nearest positive definite matrix. This keeps the underlying relationships as intact as possible, while ensuring they have the appropriate numerical characteristics.

These types and functions let us do so.

PortfolioOptimisers.AbstractPosdefEstimator — Type
abstract type AbstractPosdefEstimator <: AbstractEstimator

Abstract supertype for all positive definite matrix estimator types.

All concrete and/or abstract types that implement positive definite matrix projection or estimation should be subtypes of AbstractPosdefEstimator.

Interfaces

In order to implement a new positive definite matrix estimator which will work seamlessly with the library, subtype AbstractPosdefEstimator with all necessary parameters as part of the struct, and implement the following methods:

  • posdef!(pdm::AbstractPosdefEstimator, X::MatNum) -> MatNum: In-place projection of a matrix to the nearest positive definite matrix.
  • posdef(pdm::AbstractPosdefEstimator, X::MatNum) -> MatNum: Optional out-of-place projection of a matrix to the nearest positive definite matrix. A fallback method copies X and calls posdef!, so it is only needed if the copy can be avoided.

Arguments

  • pdm: Positive definite matrix estimator.
  • X: Covariance-like or correlation-like matrix assets × assets.

Returns

  • X::MatNum: The projected input matrix X.

Examples

We can create a dummy positive definite estimator as follows:

julia> struct MyPosdefEstimator <: PortfolioOptimisers.AbstractPosdefEstimator endjulia> function PortfolioOptimisers.posdef!(pdm::MyPosdefEstimator, X::PortfolioOptimisers.MatNum)           # Implement your in-place PD projection logic here.           println("Projecting to positive definite matrix in-place...")           return X       endjulia> function PortfolioOptimisers.posdef(pdm::MyPosdefEstimator, X::PortfolioOptimisers.MatNum)           X = copy(X)           println("Copy X...")           posdef!(pdm, X)           return X       endjulia> posdef!(MyPosdefEstimator(), [1.0 2.0; 2.0 1.0])Projecting to positive definite matrix in-place...2×2 Matrix{Float64}: 1.0  2.0 2.0  1.0julia> posdef(MyPosdefEstimator(), [1.0 2.0; 2.0 1.0])Copy X...Projecting to positive definite matrix in-place...2×2 Matrix{Float64}: 1.0  2.0 2.0  1.0

Related

source
PortfolioOptimisers.Posdef — Type
struct Posdef{__T_alg, __T_kwargs} <: AbstractPosdefEstimator

Projects a matrix to the nearest positive definite matrix, typically used for co-moment matrices.

Posdef encapsulates all parameters required for positive definite matrix projection in posdef! and posdef to perform the nearest positive definite projection according to the estimator.

Fields

  • alg: The algorithm used for the nearest correlation matrix projection.
  • kwargs: A named tuple of keyword arguments to be passed to the algorithm.

Constructors

Posdef(;    alg::Any = NearestCorrelationMatrix.Newton,    kwargs::NamedTuple = (;),) -> Posdef

Keywords correspond to the struct's fields.

Examples

julia> Posdef()Posdef     alg ┼ UnionAll: NearestCorrelationMatrix.Newton  kwargs ┴ @NamedTuple{}: NamedTuple()

Related

References

  • [7] N. J. Higham. Computing the nearest correlation matrix—a problem from finance. IMA Journal of Numerical Analysis 22, 329–343 (2002).
  • [8] H. Qi and D. Sun. A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM Journal on Matrix Analysis and Applications 28, 360–385 (2006).
source
PortfolioOptimisers.posdef — Function
posdef(pdm::Option{<:AbstractPosdefEstimator}, X::MatNum) -> MatNum

Out-of-place version of posdef!.

Algorithm

  1. Copy X.
  2. Apply posdef! to the copy, and return it. The input is never modified.

Arguments

  • pdm: Optional positive definite matrix estimator.
  • X: Covariance-like or correlation-like matrix assets × assets.

Returns

  • X::MatNum: A new matrix equal to the nearest positive definite projection of the input.

Examples

julia> using LinearAlgebrajulia> X = [1.0 2.0; 2.0 1.0];julia> Xpd = posdef(Posdef(), X);julia> LinearAlgebra.isposdef(Xpd)true

Related

source
PortfolioOptimisers.posdef! — Function
posdef!(pdm::Option{<:AbstractPosdefEstimator}, X::MatNum) -> MatNum

In-place projection of a matrix to the nearest positive definite matrix using the specified estimator.

For matrices without unit diagonal, the function converts them into correlation matrices i.e. matrices with unit diagonal, applies the algorithm, and rescales them back.

Mathematical definition

Solves the nearest correlation matrix problem:

\[\begin{align} \hat{\mathbf{C}} &= \underset{\mathbf{Y} \succeq 0,\; Y_{ii} = 1}{\arg\min} \lVert \mathbf{C} - \mathbf{Y} \rVert_F\,. \end{align}\]

Where:

  • $\hat{\mathbf{C}}$: Nearest positive semidefinite correlation matrix.
  • $\mathbf{C}$: Input correlation matrix.
  • $\mathbf{Y}$: Feasible correlation matrix (positive semidefinite, unit diagonal).
  • $\lVert \cdot \rVert_F$: Frobenius norm.

For covariance matrices, first standardise $\mathbf{C} = \mathrm{diag}(\mathbf{\Sigma})^{-1/2} \mathbf{\Sigma}\, \mathrm{diag}(\mathbf{\Sigma})^{-1/2}$, project, then rescale back.

Algorithm

  1. Return X unchanged when it is already positive definite. The projection has nothing to do, and the check runs before every other step.
  2. Check that X is square.
  3. Read the diagonal of X into s. When any entry of s is not one, X is a covariance matrix: replace s with its square roots and convert X to a correlation matrix with StatsBase.cov2cor!. The test is any(!isone, s), so it is the value of the diagonal that decides, never the type of X.
  4. Project X onto the nearest correlation matrix with NearestCorrelationMatrix.nearest_cor!, under the algorithm pdm.alg and the keyword arguments pdm.kwargs.
  5. Warn when the projected X is still not positive definite. X is returned either way, so the caller must check the result when it cannot tolerate an unrepaired matrix.
  6. When step 3 converted a covariance matrix, convert X back with StatsBase.cor2cov!. The standard deviations are the ones read in step 3, so the original diagonal returns exactly.

Arguments

  • pdm: Optional positive definite matrix estimator.

    • ::Posdef: The algorithm specified in pdm.alg is used to project X to the nearest PD matrix. If X is already positive definite, it is left unchanged.
    • ::Nothing: No-op.

  • X: Covariance-like or correlation-like matrix assets × assets.

Validation

  • X is validated with assert_matrix_issquare. The check runs after the early return, so it is reached only when X is not already positive definite.

Returns

  • X::MatNum: The input matrix X modified in-place.

Examples

julia> using LinearAlgebrajulia> est = Posdef();julia> X = [1.0 0.9; 0.9 1.0];julia> X[1, 2] = 2.0;  # Not PDjulia> posdef!(est, X)2×2 Matrix{Float64}: 1.0  1.0 1.0  1.0julia> LinearAlgebra.isposdef(X)true

Related

source

References

[7]
N. J. Higham. Computing the nearest correlation matrix—a problem from finance. IMA Journal of Numerical Analysis 22, 329–343 (2002).
[8]
H. Qi and D. Sun. A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM Journal on Matrix Analysis and Applications 28, 360–385 (2006).