Covariance: private API

The weights of a portfolio and the covariance matrix of its assets give the portfolio variance, which the classic Markowitz portfolio [32] uses as its measure of risk. The entries below support the two sample estimators of the covariance and the correlation, GeneralCovariance and Covariance.

General covariance

GeneralCovariance calls a StatsBase covariance estimator through the functions below. They retry a call without the keywords that the estimator does not take, and they make a lazy observation matrix dense before a weighted call.

PortfolioOptimisers.show_fields — Method
show_fields(_::GeneralCovariance) -> Tuple{Symbol, Symbol}

Renders every field of a GeneralCovariance except cache.

A cache holds the state of an incremental fit. That state is not part of the configuration, and it prints under the estimator at each site that renders the estimator. Set set_show_nothing_fields!(:GeneralCovariance, true) to render it.

Arguments

  • ::GeneralCovariance: The covariance estimator. The method reads only its type.

Returns

  • fields::Tuple: The field names to render, which is (:ce, :w).

Related

source
PortfolioOptimisers.densify — Function
densify(X::MatNum) -> MatNum

Materialise a lazy or sparse observation matrix as a dense Matrix.

The weighted moment methods of StatsBase take a DenseMatrix. A Transpose, an Adjoint, a SubArray or a sparse matrix does not match them. Without a mean, the call raises a MethodError, and the caller can recover from it. With a mean, the caller cannot recover. cov(::SimpleCovariance, X, w; mean = mu) forwards four positional arguments as covm(X, mu, w, dims). When the DenseMatrix method does not match, that call resolves to Statistics.covm(x, xmean, y, ymean, vardim), which is the cross-covariance of X and the weight vector. It returns an N × 1 matrix in place of an N × N matrix, and raises no error. robust_cov and robust_cor densify X before every weighted call, so neither failure can happen.

Algorithm

  1. When X is a DenseMatrix of numbers or of JuMP scalars, return it unchanged.
  2. Otherwise return Matrix(X), a dense copy.

Arguments

  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.

Returns

  • X::MatNum: X itself when it is already a dense Matrix, and Matrix(X) otherwise.

Related

source
PortfolioOptimisers.robust_cov — Function
robust_cov(
    ce::StatsBase.CovarianceEstimator,
    X::MatNum,
    [w::StatsBase.AbstractWeights];
    dims::Int = 1,
    mean = nothing,
    kwargs...
) -> MatNum

Computes the weighted or unweighted covariance with compat_cov on dense observations. The unweighted method retries once with a densified Matrix after a MethodError. The weighted method calls densify before the estimator.

Algorithm

  1. Without w, call compat_cov on X as the caller gave it. When that call raises a MethodError, call compat_cov once more on Matrix(X). Any other error propagates to the caller.
  2. With w, densify X with densify, then call compat_cov once. The method does no retry, because densify already removes the only failure that the retry handles.
  3. Return the matrix that compat_cov returned.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for compat_cov.

Validation

  • dims in (1, 2).

Returns

  • sigma::MatNum: Covariance matrix assets x assets.

Related

source
PortfolioOptimisers.robust_cor — Function
robust_cor(
    ce::StatsBase.CovarianceEstimator,
    X::MatNum,
    [w::StatsBase.AbstractWeights];
    dims::Int = 1,
    mean = nothing,
    kwargs...
) -> MatNum

Computes the weighted or unweighted correlation with compat_cor on dense observations. The unweighted method retries once with a densified Matrix after a MethodError. The weighted method calls densify before the estimator.

Algorithm

  1. Without w, call compat_cor on X as the caller gave it. When that call raises a MethodError, call compat_cor once more on Matrix(X). Any other error propagates to the caller.
  2. With w, densify X with densify, then call compat_cor once. The method does no retry, because densify already removes the only failure that the retry handles.
  3. Return the matrix that compat_cor returned.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for compat_cor.

Validation

  • dims in (1, 2).

Returns

  • rho::MatNum: Correlation matrix assets x assets.

Related

source
PortfolioOptimisers.compat_cov — Function
compat_cov(
    ce::StatsBase.CovarianceEstimator,
    X::MatNum,
    [w::StatsBase.AbstractWeights];
    dims::Int = 1,
    mean = nothing,
    kwargs...
) -> MatNum

Compute the covariance matrix with the estimator ce, and pass the extra keyword arguments only when the method of ce accepts them.

Algorithm

  1. When the caller passed extra keyword arguments, and hasmethod reports that Statistics.cov takes dims, mean and those keys for this estimator and these arguments, call Statistics.cov with all of them and return the result.
  2. When step 1 raises a MethodError, drop the extra keyword arguments and continue. A method whose signature ends in a kwargs... slurp satisfies hasmethod and can still reject a key further down its call chain. Any other error propagates to the caller.
  3. Call Statistics.cov(ce, X, args...; dims = dims, mean = mean), and return the result.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for cov.

Returns

  • sigma::MatNum: Covariance matrix assets x assets.

Related

source
PortfolioOptimisers.compat_cor — Function
compat_cor(
    ce::StatsBase.CovarianceEstimator,
    X::MatNum,
    [w::StatsBase.AbstractWeights];
    dims::Int = 1,
    mean = nothing,
    kwargs...
) -> MatNum

Compute the correlation matrix with the estimator ce, and fall back to its covariance matrix when ce has no cor method for these arguments.

Algorithm

  1. When hasmethod reports that Statistics.cor takes dims and mean for this estimator and these arguments, take steps 2 and 3. Otherwise take step 4.
  2. When the caller passed extra keyword arguments, and hasmethod reports that Statistics.cor takes those keys too, call Statistics.cor with all of them and return the result. When that call raises a MethodError, drop the extra keyword arguments and continue. A method whose signature ends in a kwargs... slurp satisfies hasmethod and can still reject a key further down its call chain.
  3. Call Statistics.cor(ce, X, args...; dims = dims, mean = mean), and return the result. When that call raises a MethodError, continue to step 4. Any other error propagates to the caller.
  4. The estimator has no cor method for these arguments, so compute the covariance matrix sigma with robust_cov.
  5. When sigma is mutable, convert it to a correlation matrix in place with StatsBase.cov2cor! and the square roots of its own diagonal. Otherwise convert a dense copy with StatsBase.cov2cor.
  6. Return sigma.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for cor.

Returns

  • rho::MatNum: Correlation matrix assets x assets.

Related

source
PortfolioOptimisers.library_covariance_estimator — Function
library_covariance_estimator(ce::AbstractCovarianceEstimator) -> AbstractCovarianceEstimator
library_covariance_estimator(ce::StatsBase.CovarianceEstimator) -> GeneralCovariance

Returns a nested covariance estimator in a form that accepts all the calls that the library makes to a nested estimator.

A field bound to StatsBase.CovarianceEstimator accepts an estimator that the library does not own, such as StatsBase.SimpleCovariance(). The library calls a nested estimator in these three ways, and StatsBase has no method for any of them:

  • With its own keywords, such as iv, ivpa and active_mask.
  • With an Asset Panel argument.
  • Through Statistics.var and Statistics.std.

Thus a verb that passes a call to a nested estimator calls it through this function. The two methods select one branch each:

  • The function returns an AbstractCovarianceEstimator without change, so it still gets all the keywords.
  • The function wraps all other StatsBase.CovarianceEstimator types in a GeneralCovariance without weights. The wrapper is an AbstractCovarianceEstimator, so the Asset Panel methods and the diagonal methods of var and std accept it. Its cov and cor go through robust_cov and robust_cor. These call the wrapped estimator with only dims and mean when its method does not take the other keywords. The wrapper returns what the bare estimator returns for the same dims and mean.

Arguments

  • ce: Covariance estimator.

Returns

  • ce: ce without change, or GeneralCovariance(; ce = ce).

Related

source
PortfolioOptimisers.weighted_comoment_guard — Function
weighted_comoment_guard(f::typeof(Statistics.cov), ce::StatsBase.SimpleCovariance,
                        sigma::AbstractMatrix, w::StatsBase.AbstractWeights) -> AbstractMatrix
weighted_comoment_guard(f::typeof(Statistics.cor), ce::StatsBase.SimpleCovariance,
                        rho::AbstractMatrix, w::StatsBase.AbstractWeights) -> AbstractMatrix
weighted_comoment_guard(f, ce::StatsBase.CovarianceEstimator, val::AbstractMatrix,
                        w::StatsBase.AbstractWeights) -> AbstractMatrix

Return the weighted covariance or correlation matrix that ce computed, or NaN where the sample gives the moment no value.

The weighted mean of one observation can differ from it by one unit in the last place, so the deviations of a weighted fit on one observation are not zero. The unweighted fit on one observation has deviations of exactly zero, and its moments are NaN where a division by zero makes them undefined. The function gives the weighted fit the same values.

  • Covariance. The method returns NaN in each entry when the divisor of the weights is not positive, with weighted_moment_guard. Without the guard, a corrected fit on one observation of positive weight is Inf.
  • Correlation. A correlation needs two observations of positive weight. With fewer, the method writes NaN at every entry, as the unweighted fit on one observation does: its variances are zero, so each correlation is 0 / 0. Without the guard, the rounding gives an entry of 1 or -1.
  • Other estimators. The library does not know the divisor of a StatsBase.CovarianceEstimator other than StatsBase.SimpleCovariance, so the fallback returns val without change.

Arguments

  • f: Statistics.cov or Statistics.cor.
  • ce: The StatsBase estimator that computed the matrix.
  • val: The matrix that ce computed with w. The StatsBase.SimpleCovariance methods write into it.
  • w: The resolved observation weights of the fit.

Returns

  • val::AbstractMatrix: The matrix, with NaN in each entry that has no value.

Related

source

Covariance

PortfolioOptimisers.show_fields — Method
show_fields(
    ce::Covariance
) -> Union{NTuple{4, Symbol}, NTuple{5, Symbol}}

Renders every field of a Covariance except cache, and cvg only where a policy is set.

A cache holds the state of an incremental fit. That state is not part of the configuration, and it prints under the estimator at each site that renders the estimator. Set set_show_nothing_fields!(:Covariance, true) to render it. The method reads cvg from the instance and not from the type, as the method for SimpleExpectedReturns does. That method states the reason.

Arguments

  • ce: The covariance estimator. The method reads only its cvg field.

Returns

  • fields::Tuple: The field names to render, which is (:me, :ce, :alg, :w) with no policy and (:me, :ce, :alg, :w, :cvg) with one.

Related

source
PortfolioOptimisers.covariance_centre_and_estimator — Function
covariance_centre_and_estimator(
    ce::Covariance,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    dims,
    mean,
    kwargs...
) -> Tuple{Any, AbstractCovarianceEstimator}

Get the centring vector and the inner covariance estimator for a Covariance method.

The four methods of Statistics.cov and Statistics.cor that take a Covariance get one centre and one inner estimator from this function, so one rule applies ce.w to the centre and to the deviations.

Algorithm

  1. Get the centre mu from ce.me and ce.w with weighted_centre. It uses mean if the caller gave one. ce.w gets to ce.me through factory, so the centre has the weights of the deviations.
  2. If ce.w is nothing, take ce.ce without change.
  3. If ce.w is not nothing, send ce.ce through factory with ce.w. An estimator of the library takes the weights. A StatsBase.CovarianceEstimator that the library does not own does not change, because no verb of the library reads its weights.
  4. Send the estimator of step 2 or 3 through library_covariance_estimator. It wraps an estimator that the library does not own in a GeneralCovariance, so that the keywords of the caller, such as iv and ivpa, do not get to a StatsBase method that refuses them.

Step 2 is for performance and is not a second contract. ce.w is a field, so its type decides the branch. The step stops a windowed loop from a rebuild of the estimator tree of ce in each window. A ce.ce with its own weights keeps them when ce.w is nothing, and ce.w replaces them when it is not. factory does the same on all other paths.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for the mean estimator.

Returns

  • mu::Union{Number, ArrNum}: The centring vector.
  • cel::AbstractCovarianceEstimator: The inner covariance estimator. It has the weights ce.w when ce.w is not nothing. It is in a GeneralCovariance wrapper when the library does not own it.

Related

source
PortfolioOptimisers.covariance_inner_estimator — Function
covariance_inner_estimator(
    ce::Covariance
) -> AbstractCovarianceEstimator

Return the inner covariance estimator of a Covariance, with the weights of ce and in a form that takes all the calls of the library.

This is steps 2 to 4 of covariance_centre_and_estimator. variance_count reads the same estimator, so the count and the fit read one rule for the weights.

Arguments

  • ce: Covariance estimator.

Returns

Related

source

Incremental fit

The full-moment sample covariance can also take the observations one at a time. Both estimators use one state type, because they update the same three quantities: the observation count, the mean, and the sum of the outer products of the deviations from the mean. partial_fit! returns a new estimator whose cache field holds that state, and cov(ce) computes the estimate from it, with no data.

PortfolioOptimisers.CovarianceState — Type
struct CovarianceState{__T_n, __T_mu, __T_M, __T_cvg} <: AbstractPartialFitState

Holds the observation count, the running mean and the co-moment accumulator of an incremental covariance fit.

It is the state of GeneralCovariance and of Covariance{<:Any, <:Any, <:FullMoment} under partial_fit!. One struct serves both, because the two estimators run the same recursion over the same three quantities. M is the accumulator $\sum_t (\boldsymbol{r}_t - \hat{\boldsymbol{\mu}}) (\boldsymbol{r}_t - \hat{\boldsymbol{\mu}})^{\intercal}$ and not the covariance, so cov(ce::GeneralCovariance, state::CovarianceState) divides it by the count, or by the count less one when the inner StatsBase.SimpleCovariance is corrected.

Fields

  • n: Number of observations folded into the state.
  • mu: Running mean of the observations folded into the state, assets × 1.
  • M: Running second-moment accumulator of the observations folded into the state, about mu.
  • cvg: Optional CoverageCounts, the per-cell denominators and the per-asset records of an available-case fold. It is nothing when the estimator holds no CoveragePolicy, so the plain state costs nothing.

Constructors

CovarianceState(;    n::Integer = 0,    mu::VecNum,    M::MatNum = zeros(eltype(mu), length(mu), length(mu)),    cvg::Option{<:CoverageCounts} = nothing) -> CovarianceState

Keywords correspond to the struct's fields. A state seeded for N assets is CovarianceState(; mu = zeros(N)), which partial_fit! builds when the cache field of the estimator holds nothing.

Validation

  • n >= 0. A DomainError is thrown otherwise.
  • !isempty(mu). An IsEmptyError is thrown otherwise.
  • Every entry of mu and of M is finite. An IsNonFiniteError is thrown otherwise.
  • size(M) == (length(mu), length(mu)). A DimensionMismatch is thrown otherwise.

The positional constructor runs each check except the scan for finite values, see assert_partial_fit_state, so a fold and a lens write run them too. The keyword constructor alone runs the scan, see assert_partial_fit_values, because it costs as much as the fold of one observation.

View parameters

port_opt_view takes the view of its fields at the selected assets:

Examples

julia> PortfolioOptimisers.CovarianceState(; mu = [0.0, 0.0])PortfolioOptimisers.CovarianceState    n ┼ Int64: 0   mu ┼ Vector{Float64}: [0.0, 0.0]    M ┼ 2×2 Matrix{Float64}  cvg ┴ nothing

Related

source
Base.copy — Method
copy(x::CovarianceState) -> CovarianceState

Copies a CovarianceState, so the copy shares no array with the original.

This method is the copy method of the AbstractPartialFitState interface, which partial_fit calls before it folds. The count is a scalar, and the method uses it without change. The method copies the running mean, the co-moment accumulator and the coverage counts.

Arguments

  • x: The state to copy.

Returns

  • state::CovarianceState: A new state, equal to x, whose mu and M are new arrays.

Related

source
PortfolioOptimisers.covariance_state_seed — Function
covariance_state_seed(cache::Nothing, x::VecNum,
                      cvg::Option{<:CoveragePolicy} = nothing) -> CovarianceState
covariance_state_seed(cache::CovarianceState, x::VecNum,
                      cvg::Option{<:CoveragePolicy} = nothing) -> CovarianceState

Returns the CovarianceState that an incremental covariance fit folds into. If the estimator holds no state, the function seeds a state of zeros.

The two covariance estimators, GeneralCovariance and Covariance, seed the same state from the same observation, so the seed is in this one function and not in each partial_fit! method.

Arguments

  • cache: The state that the estimator holds, or nothing.
  • x: One observation, assets × 1. The function reads only its length and its element type.
  • cvg: The policy that the estimator holds. It decides if the seed holds per-pair counts.

Returns

  • state::CovarianceState: The state that cache holds, or a state of zeros over length(x) assets.

Related

source
PortfolioOptimisers.partial_fit_corrected — Method
partial_fit_corrected(ce::SimpleCovariance) -> Bool

Returns the bias correction of a covariance estimator that an incremental fit can reproduce, and refuses all other estimators.

A CovarianceState accumulates the full-moment sum of outer products about the running mean. StatsBase.SimpleCovariance divides that sum by the count, or by the count less one. All other covariance estimators read the sample in a way that the accumulator cannot reproduce, so the function refuses them and does not give a wrong result. The function goes through the wrappers, so it reads the flag through a GeneralCovariance and through a Covariance whose algorithm is FullMoment.

Arguments

  • ce: Covariance estimator.

Validation

  • ce holds no observation weights at each level that the function goes through. An ArgumentError is thrown otherwise.
  • The innermost estimator is a StatsBase.SimpleCovariance. An ArgumentError is thrown otherwise.

Returns

  • corrected::Bool: The bias correction of the innermost StatsBase.SimpleCovariance.

Related

source

Available-case fit

With a CoveragePolicy in its cvg field, the estimator computes the covariance of each pair of assets from the rows that the pair shares, where both returns are finite and both assets are active. Without a policy, it computes the ordinary sample covariance over the assets that have no gap in the window. PortfolioOptimisers.coverage_covariance selects one of the two fits from the type of the cvg field.

PortfolioOptimisers.coverage_covariance — Function
coverage_covariance(f, ce, cvg, X; dims::Int = 1, mean = nothing,
                    active_mask = nothing, kwargs...) -> MatNum

Routes a covariance or correlation fit to the Coverage Universe arm or to the available-case arm.

The cvg field of the estimator is the third argument, so dispatch on the policy selects the arm, not a branch on its value. coverage_mean does the same for a sample mean. f is Statistics.cov or Statistics.cor. The moment algorithm of ce selects the family of arms. The FullMoment arms fold. The SemiMoment arms have no recursion, and they make two passes over the block.

Arguments

  • f: Statistics.cov or Statistics.cor.
  • ce: Covariance estimator.
  • cvg: The policy that the estimator holds. It selects the arm.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which the computation runs.
  • mean: A precomputed centre, or nothing.
  • active_mask: The active mask of the Asset Panel, observations × assets, or nothing. The Coverage Universe arms refuse an inactive cell, as they refuse a return that is not finite.
  • kwargs...: More keyword arguments for the centring estimator and for the inner covariance estimator.

Returns

  • sigma::MatNum: The covariance or correlation matrix.

Related

source
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
    f,
    ce::Covariance{<:Any, <:Any, <:FullMoment},
    ::Nothing,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    kwargs...
) -> Any

The Nothing method of the FullMoment arm of coverage_covariance is the Coverage Universe arm. It gets the centre and the inner estimator with covariance_centre_and_estimator, and then calls the inner estimator. This is the body that the verb had before the cvg field existed. It refuses a gap and an inactive cell of active_mask with assert_finite_sample, because a plain estimator reads an inactive cell as a return that is not finite.

Related

source
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
    f,
    ce::Covariance{<:Any, <:Any, <:FullMoment, Nothing},
    cvg::CoveragePolicy,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    strict,
    kwargs...
) -> Any

The CoveragePolicy method of the FullMoment arm of coverage_covariance is the available-case arm. It fits each pair on the observations at which its two assets are finite and active. The batch result is the incremental result, because the arm folds the block through partial_fit! and then reads the state.

The centre of a pair is the mean of that pair, which a pairwise Welford recursion holds, so the diagonal is the ordinary available-case variance of each asset. An off-diagonal entry is centred on the observations that the pair shares. The arm refuses a mean from the caller. partial_fit_corrected refuses a ce.ce, a ce.me or a ce.w that an incremental fit cannot reproduce.

Algorithm

  1. Refuse a mean from the caller.
  2. Read the Bessel correction, and refuse a configuration that no incremental fit can reproduce, with partial_fit_corrected.
  3. Fold each row of X into a new state with partial_fit!, together with its row of the active mask.
  4. Read the state with cov(ce::Covariance, state::CovarianceState). Rescale the result to a unit diagonal when the verb is Statistics.cor.

Related

source
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
    f,
    ce::Covariance{<:Any, <:Any, <:SemiMoment},
    ::Nothing,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    kwargs...
) -> Any

The Nothing method of the SemiMoment arm of coverage_covariance is the Coverage Universe arm. It clips the de-meaned returns at zero before it computes the covariance or the correlation matrix, so the matrix holds only the downside co-movements. This is the body that the verb had before the cvg field existed. It refuses a gap and an inactive cell of active_mask with assert_finite_sample, because a plain estimator reads an inactive cell as a return that is not finite.

Algorithm

  1. Refuse a gap and an inactive cell with assert_finite_sample.
  2. Get the centring vector mu and the inner estimator cel with covariance_centre_and_estimator.
  3. Replace X with min.(X .- mu, 0), the de-meaned returns clipped at zero.
  4. Call f(cel, X; dims = dims, mean = 0, kwargs...). The zero mean stops a second centring of the clipped returns.

Related

source
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
    f,
    ce::Covariance{<:Any, <:Any, <:SemiMoment, Nothing},
    cvg::CoveragePolicy,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    strict,
    kwargs...
) -> Any

The CoveragePolicy method of the SemiMoment arm of coverage_covariance is the available-case arm. It makes two passes over the block and does not fold, because a semi-covariance has no incremental recursion. The clip is about a centre that the whole window sets, so a new observation changes all the past terms.

The centre is the available-case mean of each asset, over only the finite and active observations of that asset. It is not the mean of the pair, which the FullMoment arm uses. The plain path has the same asymmetry. It takes the clip about a centre, and the co-moment about zero.

Algorithm

  1. Check dims. Refuse a ce.me or a ce.w that holds observation weights, with assert_partial_fittable. Refuse a mean from the caller.
  2. Read the valid entries of the block with coverage_valid_block.
  3. Centre each asset on its available-case mean. Clip the deviations at zero, and set the invalid entries to zero.
  4. Take the numerator as the cross product of the clipped block, and the denominator as the cross product of the valid mask, which counts the observations of each pair.
  5. Admit the assets with coverage_admission. Then peel the assets of the undetermined pairs with coverage_peel, which gives a warning, or refuses them under strict.
  6. Divide and put the result in a frame with coverage_divide. Rescale the result to a unit diagonal when the verb is Statistics.cor.

Related

source
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
    ce::Union{Covariance{<:Any, <:Any, <:FullMoment}, GeneralCovariance},
    ::Nothing,
    state::CovarianceState;
    kwargs...
) -> Any

This is the Nothing method of coverage_covariance for a state. cov(ce) with no data calls it. All the pairs share one count, so the result is all NaN until the count is more than the Bessel correction.

GeneralCovariance has no cvg field and reads the same state, so it always gets to this arm.

Related

source
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
    ce::Covariance{<:Any, <:Any, <:FullMoment},
    cvg::CoveragePolicy,
    state::CovarianceState;
    strict
) -> Any

This is the CoveragePolicy method of coverage_covariance for a state. cov(ce) with no data calls it. The method divides the accumulator of each pair by the count of that pair less the Bessel correction. An asset that the policy refuses is NaN in all of its row and column.

A pair of two admitted assets that share too few observations has no covariance, because a covariance of no observations is not a number. coverage_peel removes the assets of such pairs from the admitted set with the peel rule of the policy. It reports the peel as strict sets. Under peel = nothing the cell stays NaN, and the matrix repair refuses it.

Related

source
PortfolioOptimisers.coverage_correlation — Function
coverage_correlation(f::typeof(Statistics.cov), sigma::MatNum) -> MatNum
coverage_correlation(f::typeof(Statistics.cor), sigma::MatNum) -> MatNum

Changes an available-case covariance to the result that the verb of the caller asks for.

An available-case fit makes a covariance for both verbs, because the state holds only a co-moment accumulator. The correlation is that covariance, rescaled by the square roots of its diagonal. The method dispatches on the verb and does not compare it, so the compiler makes the choice.

Arguments

  • f: Statistics.cov or Statistics.cor.
  • sigma: The available-case covariance.

Returns

  • val::MatNum: sigma without change, or sigma rescaled to a unit diagonal.

Related

source
PortfolioOptimisers.coverage_policy — Function
coverage_policy(ce) -> Nothing
coverage_policy(ce::Covariance) -> Option{<:CoveragePolicy}

Reads the CoveragePolicy of a covariance estimator. The estimator can have no cvg field.

GeneralCovariance and Covariance share one state, and one method that makes the estimate from it, cov(ce) with no data. Only Covariance has a cvg field, so that method gets the policy through this function and not from the field. Dispatch still selects the arm. This function is a query, so it owns the answer "none": every estimator other than Covariance returns nothing. The Asset Panel form of forecast_location reads the policy of any covariance estimator through it.

Arguments

  • ce: The covariance estimator, of any type.

Returns

  • cvg::Option{<:CoveragePolicy}: The policy that the estimator holds, or nothing.

Related

source
PortfolioOptimisers.gap_fill_value — Method
gap_fill_value(ce::Covariance{<:Any, <:Any, <:Any, <:Any, <:CoveragePolicy}) -> Float64

Return NaN, so that a gapped sample gets to the available-case arm of a Covariance with its gaps.

A Covariance with a CoveragePolicy fits each pair on the observations at which its two assets are finite and active. coverage_covariance states this. A finite fill value makes a gapped cell into an observation, and the pair then reads a return that the asset did not have. The consumer passes the sample without change, together with the active mask that explains each gap. A Covariance with no policy refuses a gap itself, so it keeps the fallback of zero.

A wrapper that passes the sample on, such as PortfolioOptimisersCovariance, returns the same value as this method, so the wrapper reads a gapped sample when the Covariance in it holds a policy.

Arguments

Returns

  • fv::Float64: NaN.

Related

source

References

[32]
H. Markowitz. Portfolio Selection. The Journal of Finance 7, 77–91 (1952).