Weighted available case: private API
With a CoveragePolicy and observation weights, an estimator fits each asset, each pair of assets and each cell of a higher-order co-moment over its own valid observations, with the weights of those observations. A weighted fit does not fold, because the weights describe the whole sample, so these arms make one pass over the block. Each fit drops its other observations with their weights, through PortfolioOptimisers.observed_sample. The unweighted arms fold, and they stay in the file of each estimator.
PortfolioOptimisers.weighted_coverage_centres — Function
weighted_coverage_centres(
Xo::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
msk::AbstractMatrix{Bool},
w::Union{DynamicAbstractWeights, AbstractWeights}
) -> Any
Returns the weighted available-case mean of each asset, over the valid observations of that asset.
An available-case fit under observation weights reads each asset over its own valid observations. Each observation that the fit drops takes its weight with it, through observed_sample. The kept weights keep their type and are not scaled, and a DynamicAbstractWeights resolves on the observations of the asset. An asset with no observation of positive weight has no mean, so its entry is NaN, as has_positive_weight states.
Arguments
Xo: The oriented block,observations × assets.msk: The valid entries of the block, after the fold-time rule of the policy, fromcoverage_valid_block.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.
Returns
mu::VecNum: The weighted mean of each asset over its valid observations, orNaN.
Related
PortfolioOptimisers.coverage_mean — Method
coverage_mean(
me::SimpleExpectedReturns{<:Union{DynamicAbstractWeights, AbstractWeights}},
cvg::CoveragePolicy,
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
dims,
active_mask,
kwargs...
) -> Any
The weighted method of the available-case arm of coverage_mean. An estimator with observation weights and a CoveragePolicy calls it.
A weighted mean does not fold, because the weights of a sample describe the whole sample. So this arm makes one pass over the block, in place of the fold of the unweighted arm. It reads the same valid entries and the same admission as that arm.
Algorithm
- Read the valid entries of the block and the bookkeeping of each asset with
coverage_valid_block. - Take the weighted mean of each asset over its valid observations with
weighted_coverage_centres. - Admit the assets with
coverage_admission, against the observation count of each asset. - Put the means in a frame with
coverage_frame, and orient the result as thedimsof the caller sets.
Related
PortfolioOptimisers.weighted_observed_variance — Function
weighted_observed_variance(
x::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
w::Union{DynamicAbstractWeights, AbstractWeights},
obs::AbstractVector{Bool},
corrected::Bool
) -> Any
Returns the weighted variance of one asset over its observed sample, or NaN when the sample gives it no value.
The function reads the observations that obs marks, with their weights, through observed_sample. StatsBase then reads the bias correction of the type of the weights on that sample. A sample with no observation of positive weight is NaN, and so is a sample whose divisor is not positive, through weighted_moment_guard.
Arguments
x: The returns of the asset.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.obs:trueat each observation that the fit reads.corrected: The bias correction of the estimator.
Returns
sigma2::Number: The weighted variance, orNaN.
Related
PortfolioOptimisers.coverage_variance — Method
coverage_variance(
f,
ve::SimpleVariance,
cvg::CoveragePolicy,
::AbstractExpectedReturnsEstimator,
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
dims,
mean,
active_mask,
kwargs...
) -> Any
The CoveragePolicy method of coverage_variance is the available-case arm. It fits the dispersion of each asset on the finite and active observations of that asset. The batch result is the incremental result, because the arm folds the block through partial_fit! and then reads the state.
The centre is the running mean of the fold, so the arm refuses a mean from the caller, and it does not ignore that mean without a message. An available-case fit centres each asset on the observations of that asset, and a centre fitted over the whole window is different.
Algorithm
- Fold each row of
Xinto a new state withpartial_fit!, together with its row of the active mask. - Read the state with
var(ve::SimpleVariance, state::SimpleVarianceState). - Change the variance to the result that
fasks for withcoverage_moment_map. Orient the result as thedimsof the caller sets.
Related
coverage_variance(
f,
ve::SimpleVariance{<:Any, <:Union{DynamicAbstractWeights, AbstractWeights}},
cvg::CoveragePolicy,
::AbstractExpectedReturnsEstimator,
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
dims,
mean,
active_mask,
kwargs...
) -> Any
The weighted method of the available-case arm of coverage_variance. A SimpleVariance with observation weights and a CoveragePolicy calls it.
Each asset is centred on its weighted mean over its own valid observations, as StatsBase centres a weighted variance. The arm refuses a mean from the caller, as the unweighted arm does.
Algorithm
- Refuse a
meanfrom the caller. - Read the valid entries of the block and the bookkeeping of each asset with
coverage_valid_block. - Take the weighted variance of each asset over its valid observations with
weighted_observed_variance. - Admit the assets with
coverage_admission, and put the variances in a frame withcoverage_frame. - Change the variance to the result that
fasks for withcoverage_moment_map. Orient the result as thedimsof the caller sets.
Related
PortfolioOptimisers.coverage_variance — Method
coverage_variance(
f,
ve::SimpleVariance,
cvg::CoveragePolicy,
X::AbstractVector{<:Union{Number, AbstractJuMPScalar}};
mean
) -> Any
This is the CoveragePolicy method of the vector arm of coverage_variance. One asset has no pair, so the available-case fit over its series is the ordinary fit over its finite entries. The method compares the coverage floor with the fraction of finite entries in the series. It returns NaN if too few finite entries remain, or if the policy does not admit the asset. It refuses a mean from the caller, as the matrix method does.
Related
coverage_variance(
f,
ve::SimpleVariance{<:Any, <:Union{DynamicAbstractWeights, AbstractWeights}},
cvg::CoveragePolicy,
X::AbstractVector{<:Union{Number, AbstractJuMPScalar}};
mean
) -> Any
The weighted method of the vector arm of coverage_variance. A SimpleVariance with observation weights and a CoveragePolicy calls it on the series of one asset.
The fit reads the finite entries of the series with their weights, through weighted_observed_variance. The method compares the coverage floor with the fraction of finite entries, and it returns NaN when the policy does not admit the asset. It refuses a mean from the caller, as the matrix method does.
Related
PortfolioOptimisers.weighted_correlation_estimator — Function
weighted_correlation_estimator(ce::StatsBase.CovarianceEstimator,
w::StatsBase.AbstractWeights)
-> StatsBase.CovarianceEstimator
weighted_correlation_estimator(ce::StatsBase.SimpleCovariance,
w::StatsBase.AbstractWeights) -> StatsBase.SimpleCovarianceReturns the estimator that a weighted correlation of GeneralCovariance calls.
A correlation does not read the divisor of the weights, because the divisor cancels between the covariance and the two standard deviations. But a divisor that is not positive gives a variance at or below zero, and the square root of the diagonal then has no value, or throws. Frequency weights that sum to one or less give such a divisor under the correction. So the StatsBase.SimpleCovariance method returns an uncorrected estimator when the corrected divisor is not positive. The divisor of an uncorrected fit is the sum of the weights, which is positive. A positive divisor keeps ce, so a valid correlation does not change. A sample with fewer than two observations of positive weight then has no correlation off the diagonal, as weighted_comoment_guard states.
The fallback returns ce without change, because the library does not know the divisor of any other estimator.
Arguments
ce: Covariance estimator.w: The resolved observation weights of the fit.
Returns
ce: The estimator of the correlation.
Related
PortfolioOptimisers.coverage_inner_corrected — Function
coverage_inner_corrected(w::Option{<:ObsWeights}, ce::StatsBase.CovarianceEstimator) -> Bool
coverage_inner_corrected(w::Nothing, ce::GeneralCovariance) -> Bool
coverage_inner_corrected(w::ObsWeights, ce::GeneralCovariance) -> BoolRefuses the inner estimator of a Covariance that its available-case arm cannot fit, and returns its bias correction.
The type of the weights of the Covariance selects the arm. With no weights the arm folds, through partial_fit_corrected. The available-case arm reads the weights of the Covariance alone, so an inner GeneralCovariance with weights of its own is refused, and the message tells the caller to set the weights on the Covariance. With weights the arm makes one pass. The weights of the Covariance replace the weights of a GeneralCovariance, as covariance_inner_estimator states, so the function passes through a GeneralCovariance without reading its weights. In each arm, partial_fit_corrected reads the flag of the StatsBase.SimpleCovariance, and it refuses every other estimator.
assert_coverage_fittable calls the function at construction, and the weighted available-case arm of coverage_covariance calls it for the correction of its pairs.
Arguments
w: The observation weights of theCovariance, ornothing.ce: Covariance estimator.
Returns
corrected::Bool: The bias correction of the innermostStatsBase.SimpleCovariance.
Related
PortfolioOptimisers.weighted_pair_sample — Function
weighted_pair_sample(alg::FullMoment, Xo::MatNum, msk::AbstractMatrix{Bool},
w::ObsWeights) -> Tuple
weighted_pair_sample(alg::SemiMoment, Xo::MatNum, msk::AbstractMatrix{Bool},
w::ObsWeights) -> TupleReturns the sample that each pair of a weighted available-case covariance reads, and the centre of the pair.
The two moment algorithms take the centre at a different level, as the unweighted arms do. FullMoment centres each pair on the weighted mean of the observations that the pair shares, so the pair reads the block itself and StatsBase centres it. SemiMoment clips each asset about its own weighted available-case mean, with weighted_coverage_centres, and it takes the co-moment about zero.
Arguments
alg: The moment algorithm of the estimator.Xo: The oriented block,observations × assets.msk: The valid entries of the block.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.
Returns
(D, mean)::Tuple: The matrix that each pair reads, and themeankeyword of the pair fit.
Related
PortfolioOptimisers.weighted_pair_covariance — Function
weighted_pair_covariance(
D::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
Xo::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
msk::AbstractMatrix{Bool},
w::Union{DynamicAbstractWeights, AbstractWeights},
i::Integer,
j::Integer,
corrected::Bool,
mean
) -> Any
Returns the weighted covariance of the pair of assets i and j over the observations that both share, or NaN when the sample gives it no value.
The pair reads the rows at which both assets are valid, with their weights, through observed_sample. A DynamicAbstractWeights resolves on those rows of Xo. The covariance goes through robust_cov with a StatsBase.SimpleCovariance, the call of the weighted path of GeneralCovariance, and weighted_comoment_guard writes NaN when the divisor of the weights is not positive. A pair with no observation of positive weight is NaN.
Arguments
D: The matrix that the pair reads, fromweighted_pair_sample.Xo: The oriented block, on which a dynamic weight resolves.msk: The valid entries of the block.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.i,j: The two assets.corrected: The bias correction of the covariance.mean: The centre of the pair, ornothingfor its weighted mean.
Returns
sigma_ij::Number: The weighted covariance of the pair, orNaN.
Related
PortfolioOptimisers.coverage_covariance — Method
coverage_covariance(
f,
ce::Covariance{<:Any, <:Any, <:Union{FullMoment, SemiMoment}, <:Union{DynamicAbstractWeights, AbstractWeights}},
cvg::CoveragePolicy,
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
dims,
mean,
active_mask,
strict,
kwargs...
) -> Any
The weighted FullMoment and SemiMoment method of the available-case arm of coverage_covariance. It computes the weighted available-case covariance of a Covariance with observation weights and a CoveragePolicy. The unweighted methods take nothing in the slot of the weights, so the two arms do not overlap.
Each pair reads the observations that its two assets share, with their weights, through weighted_pair_covariance. The arm reads the valid entries, the admission and the peel of the unweighted arms, so the two arms read one universe. The counts of the admission are counts of observations, and the weights do not change them. A pair whose weighted sample gives no covariance is NaN, also when it shares observations: every shared observation can have a weight of zero, or the divisor of the weights can be not positive. The peel counts no observation for such a pair, so the pair is undetermined, and the peel removes an asset of it or refuses it under strict, as it does for a pair with no shared observation.
Algorithm
- Refuse a
meanfrom the caller. Read the bias correction of the inner estimator withcoverage_inner_corrected. - Read the valid entries of the block and the bookkeeping of each asset with
coverage_valid_block. - Take the sample and the centre of each pair with
weighted_pair_sample. - Fill each entry of the upper triangle with
weighted_pair_covariance, and mirror it. - Admit the assets with
coverage_admission, and peel the assets of the undetermined pairs withcoverage_peel. The peel reads a count of zero at each pair whose entry isNaN. - Put the matrix in a frame with
coverage_frame. Rescale it to a unit diagonal when the verb isStatistics.cor, withcoverage_correlation.
Arguments
f:Statistics.covorStatistics.cor.ce: The covariance estimator.cvg: The policy of the estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.mean: Must benothing.active_mask: The active mask of the Asset Panel, ornothing.strict: Whether a peel refuses rather than warns.kwargs...: Ignored. The unweighted arms take them too.
Returns
val::MatNum: The covariance or the correlation matrix.
Related
PortfolioOptimisers.weighted_comoment_cell — Function
weighted_comoment_cell(
Y::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
Xo::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
msk::AbstractMatrix{Bool},
w::Union{DynamicAbstractWeights, AbstractWeights},
idx::Union{Tuple{Vararg{Integer}}, AbstractVector{<:Integer}}
) -> Any
Returns the weighted co-moment of one cell of a higher-order available-case fit, or NaN when the cell has no observation of positive weight.
The cell reads the observations at which every asset of idx is valid, with their weights, through observed_sample. A DynamicAbstractWeights resolves on those rows of Xo. The co-moment is the weighted mean of the product of the deviations of the assets of the cell. A higher-order co-moment takes no bias correction, so the divisor is the sum of the kept weights, as the count of observations is the divisor of the unweighted arm.
Arguments
Y: The deviations of each asset from its weighted available-case centre,observations × assets.Xo: The oriented block, on which a dynamic weight resolves.msk: The valid entries of the block.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.idx: The assets of the cell, as a vector or a tuple of indices.
Returns
m::Number: The weighted co-moment of the cell, orNaN.
Related
PortfolioOptimisers.weighted_comoment — Function
weighted_comoment(
Y::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
Xo::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
msk::AbstractMatrix{Bool},
w::Union{DynamicAbstractWeights, AbstractWeights},
_::Val{K}
) -> Any
Builds the weighted available-case co-moment of order K, as an array of K axes of one entry for each asset.
The co-moment is symmetric in its K indices, so the function computes each cell once, with weighted_comoment_cell. It walks the cells in column-major order. The permutation of a cell whose indices descend has the smallest linear index of all the permutations of that cell, so the walk reaches it first. The function computes the value there, and every other permutation copies it. The dense layouts of the library are reshapes of this array: the coskewness is N × N² and the cokurtosis is N² × N².
A cell reads the rows of its own assets, and a dynamic weight resolves on each cell, so the function cannot take the products of the expansions that the unweighted arm takes.
Arguments
Y: The deviations of each asset from its weighted available-case centre.Xo: The oriented block, on which a dynamic weight resolves.msk: The valid entries of the block.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.::Val{K}: The order of the co-moment.
Returns
A::Array: The co-moment, withKaxes of lengthN.
Related
PortfolioOptimisers.weighted_comoment_block — Function
weighted_comoment_block(
alg::AbstractMomentAlgorithm,
cvg::CoveragePolicy,
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
active_mask::Union{Nothing, AbstractMatrix{<:Bool}},
w::Union{DynamicAbstractWeights, AbstractWeights},
dims::Int64
) -> Tuple{Any, Any, Any, Union{Nothing, BitVector}}
Reads the deviations, the valid entries and the admitted assets that a weighted higher-order available-case fit needs.
It is the weighted form of coverage_comoment_block. Each asset is centred on its weighted mean over its own valid observations, with weighted_coverage_centres, and the moment algorithm then clips the deviations, with coverage_comoment_deviations. The admission reads the observation count of each asset, as in the unweighted arm.
Arguments
alg: The moment algorithm of the estimator.cvg: The policy of the estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.active_mask: The active mask of the Asset Panel, ornothing.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.dims: Dimension along which the computation runs.
Returns
(Xo, Y, msk, cmsk)::Tuple: The oriented block, the deviations, the valid entries, and the admitted assets ornothing.
Related