Base moments

Abstract moment types and fallbacks

Some optimisations and constraints make use of summary statistics. These types and functions form the base for moment estimation in PortfolioOptimisers.jl.

They also provide generic fallbacks for the various functionality in the library.

PortfolioOptimisers.AbstractExpectedReturnsEstimator Type

abstract type AbstractExpectedReturnsEstimator <: AbstractEstimator

Abstract supertype for all expected returns estimator types in PortfolioOptimisers.jl.

All concrete and/or abstract types that implement expected returns estimation should be subtypes of AbstractExpectedReturnsEstimator.

Interfaces

In order to implement a new expected returns estimator which will work seamlessly with the library, subtype AbstractExpectedReturnsEstimator with all necessary parameters–-including observation weights–-as part of the struct, and implement the following methods:

Expected returns

• Statistics.mean(me::AbstractExpectedReturnsEstimator, X::MatNum; kwargs...) -> ArrNum: Expected returns estimation.

Arguments

• me: Expected returns estimator.

• X: Data matrix observations × features if the dims keyword does not exist or dims = 1, features × observations when dims = 2.

• kwargs...: Additional keyword arguments passed to the mean estimator.

Returns

• mu::ArrNum: Expected returns vector features x 1 if the dims keyword does not exist or dims = 2, 1 x features if dims = 1.

Factory

• PortfolioOptimisers.factory(me::AbstractExpectedReturnsEstimator, w::PortfolioOptimisers.ObsWeights) -> AbstractExpectedReturnsEstimator: Factory method for creating instances of the estimator with new observation weights.

Arguments

• me: Expected returns estimator.

• w: Observation weights vector observations × 1.

Returns

• me: New expected returns estimator of the same type as the argument, with the appropriate weights applied.

Examples

julia> struct MyExpectedReturnsEstimator{T1} <:
              PortfolioOptimisers.AbstractExpectedReturnsEstimator
           w::T1
           function MyExpectedReturnsEstimator(w::PortfolioOptimisers.Option{<:PortfolioOptimisers.ObsWeights})
               PortfolioOptimisers.assert_nonempty_nonneg_finite_val(w, :w)
               return new{typeof(w)}(w)
           end
       end

julia> function MyExpectedReturnsEstimator(;
                                           w::PortfolioOptimisers.Option{<:PortfolioOptimisers.ObsWeights} = nothing)
           return MyExpectedReturnsEstimator(w)
       end
MyExpectedReturnsEstimator

julia> function PortfolioOptimisers.factory(::MyExpectedReturnsEstimator,
                                            w::PortfolioOptimisers.ObsWeights)
           return MyExpectedReturnsEstimator(; w = w)
       end

julia> function Statistics.mean(est::MyExpectedReturnsEstimator, X::PortfolioOptimisers.MatNum;
                                dims::Int = 1, kwargs...)
           if !(dims in (1, 2))
               throw(DomainError(dims, "dims must be either 1 or 2"))
           end
           if dims == 2
               X = X'
           end
           w = ifelse(isnothing(est.w), fill(one(eltype(X)), size(X, 1)), est.w)
           X = X .* w
           mu = sum(X; dims = 1) / sum(w)
           return isone(dims) ? reshape(mu, 1, :) : reshape(mu, :, 1)
       end

julia> mean(MyExpectedReturnsEstimator(), [1.0 2.0; 0.3 0.7; 0.5 1.1]; dims = 2)
3×1 Matrix{Float64}:
 1.5
 0.5
 0.8

julia> PortfolioOptimisers.factory(MyExpectedReturnsEstimator(), StatsBase.Weights([1, 2, 3]))
MyExpectedReturnsEstimator
  w ┴ StatsBase.Weights{Int64, Int64, Vector{Int64}}: [1, 2, 3]

Related

• AbstractEstimator

• AbstractExpectedReturnsAlgorithm

source

PortfolioOptimisers.AbstractExpectedReturnsAlgorithm Type

abstract type AbstractExpectedReturnsAlgorithm <: AbstractAlgorithm

Abstract supertype for all expected returns algorithm types in PortfolioOptimisers.jl.

All concrete and/or abstract types that implement a specific algorithm used by an expected returns estimator should be subtypes of AbstractExpectedReturnsAlgorithm.

Interfaces

Given that these are meant to be used by expected returns estimators, there are no specific methods that need to be implemented for this abstract type. However, it serves as a marker for dispatching and organising different expected returns algorithms within the library. The interfaces should be defined at the level of the expected returns estimator that utilises these algorithms.

Related

• AbstractAlgorithm

• AbstractExpectedReturnsEstimator

source

PortfolioOptimisers.AbstractMomentAlgorithm Type

abstract type AbstractMomentAlgorithm <: AbstractAlgorithm

Abstract supertype for all moment algorithm types in PortfolioOptimisers.jl.

All concrete and/or abstract types that implement a specific algorithm for moment estimation should be subtypes of AbstractMomentAlgorithm.

Interfaces

Given that these are meant to be used by covariance estimators, there are no specific methods that need to be implemented for this abstract type. However, it serves as a marker for dispatching and organising different moment algorithms within the library. The interfaces should be defined at the level of the covariance estimator that utilises these algorithms.

Related

• AbstractAlgorithm

• AbstractCovarianceEstimator

source

PortfolioOptimisers.AbstractCovarianceEstimator Type

abstract type AbstractCovarianceEstimator <: CovarianceEstimator

Abstract supertype for all covariance estimator types in PortfolioOptimisers.jl.

All concrete and/or abstract types that implement covariance estimation should be subtypes of AbstractCovarianceEstimator.

Interfaces

In order to implement a new covariance estimator which will work seamlessly with the library, subtype AbstractCovarianceEstimator with all necessary parameters–-including observation weights–-as part of the struct, and implement the following methods:

Covariance and correlation

• Statistics.cov(ce::AbstractCovarianceEstimator, X::MatNum; kwargs...) -> MatNum: Covariance matrix estimation.

• Statistics.cor(ce::AbstractCovarianceEstimator, X::MatNum; kwargs...) -> MatNum: Correlation matrix estimation.

Arguments

• ce: Covariance estimator.

• X: Data matrix observations × features if the dims keyword does not exist or dims = 1, features × observations when dims = 2.

• kwargs...: Additional keyword arguments passed to the underlying covariance estimator.

Returns

• sigrho::MatNum: Covariance/correlation matrix features x features.

Factory

• PortfolioOptimisers.factory(ce::AbstractCovarianceEstimator, w::PortfolioOptimisers.ObsWeights) -> AbstractCovarianceEstimator: Factory method for creating instances of the estimator with new observation weights.

Arguments

• ce: Covariance estimator.

• w: Observation weights vector observations × 1.

Returns

• ce: New covariance estimator of the same type as the argument, with the new weights applied.

Examples

We can create a dummy covariance estimator as follows:

julia> struct MyCovarianceEstimator{T1} <: PortfolioOptimisers.AbstractCovarianceEstimator
           w::T1
           function MyCovarianceEstimator(w::PortfolioOptimisers.Option{<:PortfolioOptimisers.ObsWeights})
               PortfolioOptimisers.assert_nonempty_nonneg_finite_val(w, :w)
               return new{typeof(w)}(w)
           end
       end

julia> function MyCovarianceEstimator(;
                                      w::PortfolioOptimisers.Option{<:PortfolioOptimisers.ObsWeights} = nothing)
           return MyCovarianceEstimator(w)
       end
MyCovarianceEstimator

julia> function PortfolioOptimisers.factory(::MyCovarianceEstimator,
                                            w::PortfolioOptimisers.ObsWeights)
           return MyCovarianceEstimator(; w = w)
       end

julia> function Statistics.cov(est::MyCovarianceEstimator, X::PortfolioOptimisers.MatNum;
                               dims::Int = 1, kwargs...)
           if !(dims in (1, 2))
               throw(DomainError(dims, "dims must be either 1 or 2"))
           end
           if dims == 2
               X = X'
           end
           w = ifelse(isnothing(est.w), StatsBase.fweights(fill(1.0, size(X, 1))), est.w)
           X = X .* w
           sigma = X * X'
           return sigma
       end

julia> function Statistics.cor(est::MyCovarianceEstimator, X::PortfolioOptimisers.MatNum;
                               dims::Int = 1, kwargs...)
           if !(dims in (1, 2))
               throw(DomainError(dims, "dims must be either 1 or 2"))
           end
           if dims == 2
               X = X'
           end
           w = ifelse(isnothing(est.w), StatsBase.fweights(fill(1.0, size(X, 1))), est.w)
           X = X .* w
           sigma = X * X'
           d = LinearAlgebra.diag(sigma)
           StatsBase.cov2cor!(sigma, sqrt.(d))
           return sigma
       end

julia> cov(MyCovarianceEstimator(), [1.0 2.0; 0.3 0.7; 0.5 1.1])
3×3 Matrix{Float64}:
 5.0  1.7   2.7
 1.7  0.58  0.92
 2.7  0.92  1.46

julia> cor(MyCovarianceEstimator(), [1.0 2.0; 0.3 0.7; 0.5 1.1])
3×3 Matrix{Float64}:
 1.0       0.998274  0.999315
 0.998274  1.0       0.999764
 0.999315  0.999764  1.0

julia> PortfolioOptimisers.factory(MyCovarianceEstimator(), StatsBase.Weights([1, 2, 3]))
MyCovarianceEstimator
  w ┴ StatsBase.Weights{Int64, Int64, Vector{Int64}}: [1, 2, 3]

Related

• StatsBase.CovarianceEstimator

• AbstractMomentAlgorithm

source

PortfolioOptimisers.AbstractVarianceEstimator Type

abstract type AbstractVarianceEstimator <: AbstractCovarianceEstimator

Abstract supertype for all variance estimator types in PortfolioOptimisers.jl.

All concrete and/or abstract types that implement variance estimation should be subtypes of AbstractVarianceEstimator.

Interfaces

In order to implement a new covariance estimator which will work seamlessly with the library, subtype AbstractVarianceEstimator with all necessary parameters–-including observation weights–-as part of the struct, and implement the following methods:

Variance and standard deviation

• Statistics.var(ve::AbstractVarianceEstimator, X::MatNum; kwargs...) -> ArrNum: Variance estimation.

• Statistics.std(ve::AbstractVarianceEstimator, X::MatNum; kwargs...) -> ArrNum: Standard deviation estimation.

• Statistics.var(ve::AbstractVarianceEstimator, X::VecNum; kwargs...) -> Num: Variance estimation.

• Statistics.std(ve::AbstractVarianceEstimator, X::VecNum; kwargs...) -> Num: Standard deviation estimation.

Arguments

• ve: Variance estimator.

• X

  • X: Data matrix observations × features if the dims keyword does not exist or dims = 1, features × observations when dims = 2.

  • X: Data vector observations × 1.

• kwargs...: Additional keyword arguments passed to the mean estimator.

Returns

• X: Data matrix observations × features if the dims keyword does not exist or dims = 1, features × observations when dims = 2.

  • res::ArrNum: Variance or standard deviation vector of X, reshaped to be consistent with the dimension along which the value is computed.

• X: Data vector observations × 1.

  • res::Number: Variance or standard deviation X

Factory

• PortfolioOptimisers.factory(ve::AbstractVarianceEstimator, w::PortfolioOptimisers.ObsWeights) -> AbstractVarianceEstimator: Factory method for creating instances of the estimator with new observation weights.

Arguments

• ve: Variance estimator.

• w: Observation weights vector observations × 1.

Returns

• ve: New variance estimator of the same type as the argument, with the new weights applied.

Examples

We can create a dummy variance estimator as follows:

julia> struct MyVarianceEstimator{T1} <: PortfolioOptimisers.AbstractVarianceEstimator
           w::T1
           function MyVarianceEstimator(w::PortfolioOptimisers.Option{<:PortfolioOptimisers.ObsWeights})
               PortfolioOptimisers.assert_nonempty_nonneg_finite_val(w, :w)
               return new{typeof(w)}(w)
           end
       end

julia> function MyVarianceEstimator(;
                                    w::PortfolioOptimisers.Option{<:PortfolioOptimisers.ObsWeights} = nothing)
           return MyVarianceEstimator(w)
       end
MyVarianceEstimator

julia> function PortfolioOptimisers.factory(::MyVarianceEstimator,
                                            w::PortfolioOptimisers.ObsWeights)
           return MyVarianceEstimator(; w = w)
       end

julia> function Statistics.var(est::MyVarianceEstimator, X::PortfolioOptimisers.MatNum;
                               dims::Int = 1, kwargs...)
           if !(dims in (1, 2))
               throw(DomainError(dims, "dims must be either 1 or 2"))
           end
           if dims == 2
               X = X'
           end
           w = ifelse(isnothing(est.w), StatsBase.fweights(fill(1.0, size(X, 1))), est.w)
           X = X .* w
           sigma = LinearAlgebra.diag(X * X')
           return isone(dims) ? reshape(sigma, 1, :) : reshape(sigma, :, 2)
       end

julia> function Statistics.std(est::MyVarianceEstimator, X::PortfolioOptimisers.MatNum;
                               dims::Int = 1, kwargs...)
           if !(dims in (1, 2))
               throw(DomainError(dims, "dims must be either 1 or 2"))
           end
           if dims == 2
               X = X'
           end
           w = ifelse(isnothing(est.w), StatsBase.fweights(fill(1.0, size(X, 1))), est.w)
           X = X .* w
           sigma = sqrt.(LinearAlgebra.diag(X * X'))
           return isone(dims) ? reshape(sigma, 1, :) : reshape(sigma, :, 1)
       end

julia> function Statistics.var(est::MyVarianceEstimator, X::PortfolioOptimisers.VecNum; kwargs...)
           w = ifelse(isnothing(est.w), StatsBase.fweights(fill(1.0, size(X, 1))), est.w)
           X = X .* w
           return mean(LinearAlgebra.diag(X' * X))
       end

julia> function Statistics.std(est::MyVarianceEstimator, X::PortfolioOptimisers.VecNum; kwargs...)
           w = ifelse(isnothing(est.w), StatsBase.fweights(fill(1.0, size(X, 1))), est.w)
           X = X .* w
           return sqrt(mean(LinearAlgebra.diag(X' * X)))
       end

julia> var(MyVarianceEstimator(), [1.0 2.0; 0.3 0.7; 0.5 1.1])
1×3 Matrix{Float64}:
 5.0  0.58  1.46

julia> std(MyVarianceEstimator(), [1.0 2.0; 0.3 0.7; 0.5 1.1])
1×3 Matrix{Float64}:
 2.23607  0.761577  1.2083

julia> PortfolioOptimisers.factory(MyVarianceEstimator(), StatsBase.Weights([1, 2, 3]))
MyVarianceEstimator
  w ┴ StatsBase.Weights{Int64, Int64, Vector{Int64}}: [1, 2, 3]

Related

• AbstractCovarianceEstimator

source

PortfolioOptimisers.factory Method

factory(
    ce::StatsBase.CovarianceEstimator,
    args...;
    kwargs...
) -> StatsBase.CovarianceEstimator

Fallback for covariance estimator factory methods.

Arguments

• ce: Covariance estimator.

• args...: Optional arguments (ignored).

• kwargs...: Optional keyword arguments (ignored).

Returns

• ce::StatsBase.CovarianceEstimator: The original covariance estimator.

Related

• factory

• AbstractCovarianceEstimator

• StatsBase.CovarianceEstimator

source

PortfolioOptimisers.robust_cov Function

robust_cov(
    ce::StatsBase.CovarianceEstimator,
    X::MatNum,
    [w::StatsBase.AbstractWeights];
    dims::Int = 1,
    mean = nothing,
    kwargs...
) -> MatNum

Compute the covariance matrix robustly using the specified covariance estimator ce, data matrix X, and optional weights vector w.

Arguments

• ce: Covariance estimator.

• X: Data matrix observations × features if the dims keyword does not exist or dims = 1, features × observations when dims = 2.

• w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.

• dims: Dimension along which to perform the computation.

• mean: Optional mean value to use for centering.

• kwargs...: Additional keyword arguments passed to cov.

Returns

• sigma::MatNum: Covariance matrix features x features.

Details

• This function attempts to compute the optionally weighted covariance matrix using the provided estimator and keyword arguments.

• If an error occurs (e.g., due to unsupported keyword arguments), it retries with a reduced set of arguments for compatibility.

Related

• MatNum

• robust_cor

• Statistics.cov

source

PortfolioOptimisers.robust_cor Function

robust_cor(
    ce::StatsBase.CovarianceEstimator,
    X::MatNum,
    [w::StatsBase.AbstractWeights];
    dims::Int = 1,
    mean = nothing,
    kwargs...
) -> MatNum

Compute the correlation matrix robustly using the specified covariance estimator ce, data matrix X, and optional weights vector w.

Arguments

• ce: Covariance estimator.

• X: Data matrix observations × features if the dims keyword does not exist or dims = 1, features × observations when dims = 2.

• w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.

• dims: Dimension along which to perform the computation.

• mean: Optional mean value to use for centering.

• kwargs...: Additional keyword arguments passed to cor.

Returns

• rho::MatNum: Correlation matrix features x features.

Details

• This function attempts to compute the optionally weighted correlation matrix using the provided estimator and keyword arguments.

• If an error occurs (e.g., due to unsupported keyword arguments), it retries with a reduced set of arguments for compatibility.

• If that also errors, it tries again with robust_cov and converts the result to a correlation matrix.

Related

• MatNum

• robust_cov

• Statistics.cor

source

PortfolioOptimisers.moment_window_and_weights Function

moment_window_and_weights(
    X::VecNum_MatNum,
    w::Option{<:ObsWeights},
    args...;
    dims = dims,
    kwargs...
) -> (VecNum_MatNum, Option{<:StatsBase.AbstractWeights})
moment_window_and_weights(
    X::VecNum_MatNum,
    w::Option{<:ObsWeights},
    window::VecInt;
    dims = dims,
    kwargs...
) -> (VecNum_MatNum, Option{<:StatsBase.AbstractWeights})

Apply the observation window and resolve weights for moment estimation.

Slices X to the last window observations (if provided) and resolves the observation weights, returning the windowed data and finalised weights.

Arguments

• X: Data matrix or vector.

• w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.

• Either:

  • args: Additional positional arguments (ignored).

  • window: Observation window.

• dims: Dimension along which to perform the computation. Ignored if X is a vector.

• kwargs: Additional keyword arguments (ignored).

Returns

• X::VecNum_MatNum: Appropriately windowed data matrix.

• w::Option{<:StatsBase.AbstractWeights}: Resolved and appropriately windowed weights.

Details

• If window is provided:

  • Gets the appropriate view of X given its type and the value of dims.

  • Calls nothing_scalar_array_getindex on w to resolve the windowed weights.

• If no window is provided:

  • Calls get_observation_weights on w to resolve the weights.

• Returns the appropriate X and w.

Related

• get_window

• get_observation_weights

source

Full and semi moments

Moments other than the expected return can be estimated using the entire spectrum of deviations (full), or only the deviations below a target (semi/downside). These types allow us to provide such functionality.

PortfolioOptimisers.Full Type

struct Full <: AbstractMomentAlgorithm

Full is used to indicate that all deviations are included in the moment estimation process.

When computing the full moments, the expression of deviation used is the following:

$$\begin{array}{rl}\mathit{D}& =\mathit{X}-t\end{array}$$

Where:

• $\mathit{X}$: Data vector observations × 1.

• $t$: Target value, usually the unweighted (or weighted) expected value $E[\mathit{X}]$.

Related

• AbstractMomentAlgorithm

• Semi

source

PortfolioOptimisers.Semi Type

struct Semi <: AbstractMomentAlgorithm

Semi is used for semi-moment estimators, where only observations below the a target are considered.

$$\begin{array}{rl}\mathit{D}& =min(\mathit{X}-t,0)\end{array}$$

Where:

• $\mathit{X}$: Data vector observations × 1.

• $t$: Target value, usually the unweighted (or weighted) expected value $E[\mathit{X}]$.

Related

• AbstractMomentAlgorithm

• Full

source