Simple variance and standard deviation: private API

The variance is used throughout the library, it can be used as part of the expected return, covariance estimation, performance analysis, and constraint generation. It is trivial to compute the standard deviation from the variance, so we provide those too.

PortfolioOptimisers.show_fields — Method

show_fields(
    ve::SimpleVariance
) -> Union{Tuple{Symbol, Symbol, Symbol}, NTuple{4, Symbol}}

Renders every field of a SimpleVariance except cache, and cvg only where a policy is set.

The state a cache holds is the running detail of an incremental fit, not the configuration a reader looks the type up for, and it prints under the estimator at every site that renders one. Set set_show_nothing_fields!(:SimpleVariance, true) to render it. cvg is read from the instance rather than from the type, as it is for SimpleExpectedReturns, which states the reason.

Arguments

ve: Variance estimator, read for its cvg field.

Returns

fields::Tuple: The field names to render, which is (:me, :w, :corrected) with no policy and (:me, :w, :corrected, :cvg) with one.

Related

source

PortfolioOptimisers.simple_variance_kernel — Function

simple_variance_kernel(
    f,
    ve::SimpleVariance,
    me::AbstractExpectedReturnsEstimator,
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    kwargs...
) -> Any

Dispersion kernel shared by the SimpleVariance methods of Statistics.std and Statistics.var.

Algorithm

The matrix method:

Check that dims is 1 or 2.
Resolve the centring vector mu from me and ve.w with weighted_centre, which reads mean when the caller gave one.
Resolve the observation weights from ve.w against X with get_observation_weights, giving w.
When w is nothing, call f(X; dims = dims, corrected = ve.corrected, mean = mu).
Otherwise call f(X, w, dims; corrected = ve.corrected, mean = mu).

The vector method:

Resolve the observation weights from ve.w against X, giving w.
When w is nothing, call f(X; corrected = ve.corrected, mean = mean).
Otherwise call f(X, w; corrected = ve.corrected, mean = mean).

The two methods reach one centre by two routes. The matrix method resolves a centre before it calls f, and weighted_centre takes that centre from me after factory writes ve.w into it. The vector method passes mean through, so a mean of nothing leaves f to centre on the weighted mean of X. One SimpleVariance therefore answers a one-column matrix and the matching vector with one number. ve.w wins over the weights that me carries, which is what factory does on every other path, and mean takes any other centre.

weighted_centre calls factory only when ve.w is not nothing. That test is a performance guard and not a second contract: ve.w is a field, so its type decides the branch, and the guard keeps a windowed loop from rebuilding the estimator tree of me once per window.

Arguments

f: Dispersion function to apply, either Statistics.std or Statistics.var.
ve::SimpleVariance: Variance estimator. Supplies the observation weights and the corrected flag.
me::AbstractExpectedReturnsEstimator: Expected returns estimator used when no mean is provided. Matrix methods only.
X::VecNum_MatNum: Data matrix or vector.
dims::Int = 1: Dimension along which to operate. Matrix methods only.
mean = nothing: Precomputed mean.
kwargs...: Forwarded to the mean and weight resolution. Matrix methods only.

Validation

dims in (1, 2). Matrix methods only.

Returns

sigma::Union{<:Number, <:ArrNum}: Dispersion of X computed by f.

Related

source

Incremental fit

The sample variance folds one observation at a time, so a long history need not be held or re-read. partial_fit! returns a new estimator whose cache field carries the state, and var reads the fit off the estimator alone.

PortfolioOptimisers.SimpleVarianceState — Type

struct SimpleVarianceState{__T_n, __T_mu, __T_M, __T_cvg} <: AbstractPartialFitState

Carries the running observation count, mean and per-asset second-moment accumulator of an incremental variance fit.

The state of SimpleVariance under partial_fit!. M is the accumulator $\sum_t (r_{tj} - \hat{\mu}_j)^2$ and not the variance, so var(ve::SimpleVariance, state::SimpleVarianceState) divides it by the count, or by the count less one when ve.corrected holds.

Fields

n: Number of observations folded into the state.

mu: Running mean of the observations folded into the state, assets × 1.

M: Running second-moment accumulator of the observations folded into the state, about mu.

cvg: Optional CoverageCounts, the per-cell denominators and per-asset bookkeeping of an available-case fold. It is nothing when the estimator carries no CoveragePolicy, so the plain state costs nothing.

Constructors

SimpleVarianceState(;    n::Integer = 0,    mu::VecNum,    M::VecNum = zeros(eltype(mu), length(mu)),    cvg::Option{<:CoverageCounts} = nothing) -> SimpleVarianceState

Keywords correspond to the struct's fields. A state seeded for N assets is SimpleVarianceState(; mu = zeros(N)), which partial_fit! builds when the cache field of the estimator holds nothing.

Validation

n >= 0. A DomainError is thrown otherwise.
!isempty(mu). An IsEmptyError is thrown otherwise.
Every entry of mu and of M is finite. An IsNonFiniteError is thrown otherwise.
length(M) == length(mu). A DimensionMismatch is thrown otherwise.

View parameters

When port_opt_view is called on this type, its fields are subset to the selected assets:

mu: Sliced to the selected indices via port_opt_view.
M: Sliced to the selected indices via port_opt_view.

Examples

julia> PortfolioOptimisers.SimpleVarianceState(; mu = [0.0, 0.0])PortfolioOptimisers.SimpleVarianceState    n ┼ Int64: 0   mu ┼ Vector{Float64}: [0.0, 0.0]    M ┼ Vector{Float64}: [0.0, 0.0]  cvg ┴ nothing

Related

source

PortfolioOptimisers.variance_state_seed — Function

variance_state_seed(
    cache::Union{Nothing, SimpleVarianceState},
    x::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}
) -> Any
variance_state_seed(
    cache::Union{Nothing, SimpleVarianceState},
    x::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    cvg::Union{Nothing, CoveragePolicy}
) -> Any

Returns the SimpleVarianceState an incremental variance fit folds into, seeding one of zeros when the estimator carries none.

The seed is written here rather than inside partial_fit!, so the fold reads as one line and the branch that reads the cache field has one home.

Arguments

cache: The state the estimator carries, or nothing.
x: One observation, assets × 1, read for its length and its element type.

Returns

state::SimpleVarianceState: The state cache holds, or a state of zeros over length(x) assets.

Related

SimpleVarianceState
partial_fit!

source

Base.copy — Method

copy(x::SimpleVarianceState) -> SimpleVarianceState

Copies a SimpleVarianceState, so the copy shares no array with the original.

The copy method of the AbstractPartialFitState interface, which partial_fit calls before it folds. The count is a scalar and passes through, and the running mean and the per-asset accumulator are copied.

Arguments

x: The state to copy.

Returns

state::SimpleVarianceState: A fresh state, equal to x, whose mu and M are fresh vectors.

Related

source

Available-case fit

With a CoveragePolicy in its cvg field the estimator fits each asset on that asset's own finite and active observations, and PortfolioOptimisers.coverage_variance routes between that arm and the Coverage Universe one.

PortfolioOptimisers.coverage_variance — Function

coverage_variance(f, ve, cvg, me, X; dims::Int = 1, mean = nothing,
                  active_mask = nothing, kwargs...) -> ArrNum
coverage_variance(f, ve, cvg, X::VecNum; mean = nothing) -> Number

Routes a dispersion fit to the Coverage Universe arm or to the available-case arm.

The cvg field of the estimator is passed as the third argument, so the arm is chosen by dispatch on the policy rather than by a branch on its value, exactly as coverage_mean does for a sample mean. f is Statistics.var or Statistics.std, and the available-case arm maps one onto the other through coverage_moment_map.

Arguments

f: Statistics.var or Statistics.std.
ve: Variance estimator.
cvg: The policy the estimator carries, which selects the arm.
me: The estimator that centres the fit.
X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
dims: Dimension along which to perform the computation.
mean: A precomputed centre, or nothing.
active_mask: The active mask of the Asset Panel, observations × assets, or nothing. The Coverage Universe arm ignores it.
kwargs...: Additional keyword arguments passed to the centring estimator and the weights.

Returns

val: The dispersion, of the shape f gives.

Related

source

PortfolioOptimisers.coverage_variance — Method

coverage_variance(
    f,
    ve::SimpleVariance,
    ::Nothing,
    me::AbstractExpectedReturnsEstimator,
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    kwargs...
) -> Any

Nothing method of coverage_variance. The Coverage Universe arm, which is the body simple_variance_kernel has always had: it refuses a gapped sample with assert_finite_sample and ignores the active mask.

Related

coverage_variance
simple_variance_kernel

source

PortfolioOptimisers.coverage_variance — Method

coverage_variance(
    f,
    ve::SimpleVariance,
    cvg::CoveragePolicy,
    ::AbstractExpectedReturnsEstimator,
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}};
    dims,
    mean,
    active_mask,
    kwargs...
) -> Any

CoveragePolicy method of coverage_variance. The available-case arm: each asset's dispersion is fitted on that asset's own finite and active observations, and the batch answer is the incremental one, because the arm folds the block through partial_fit! and reads the state out.

The centre is the fold's own running mean, so a mean a caller passes is refused rather than silently dropped: an available-case fit centres each asset on that asset's own observations, and a centre fitted over the whole window is not that.

Algorithm

Fold every row of X into a fresh state with partial_fit!, carrying the active mask.
Read the state out with var(ve::SimpleVariance, state::SimpleVarianceState).
Map the variance onto the answer f asks for with coverage_moment_map, and orient it as the caller's dims asks.

Related

source

PortfolioOptimisers.coverage_variance — Method

coverage_variance(
    f,
    ve::SimpleVariance,
    ::Nothing,
    X::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}};
    mean
) -> Any

Nothing method of the vector arm of coverage_variance. The Coverage Universe arm over one asset's series, which refuses a gapped series with assert_finite_sample.

Related

coverage_variance
simple_variance_kernel

source

PortfolioOptimisers.coverage_variance — Method

coverage_variance(
    f,
    ve::SimpleVariance,
    cvg::CoveragePolicy,
    X::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}};
    mean
) -> Any

CoveragePolicy method of the vector arm of coverage_variance. One asset has no pair, so available-case estimation over its series is the ordinary fit over the finite entries of it, and the coverage floor is read against the length of the series.

Related

source

PortfolioOptimisers.coverage_variance — Method

coverage_variance(
    ve::SimpleVariance,
    _::Nothing,
    state::SimpleVarianceState
) -> Any

Nothing method of the read-out arm of coverage_variance. Every asset shares one count, so the whole answer is NaN until the count passes the Bessel correction.

Related

coverage_variance
SimpleVarianceState

source

PortfolioOptimisers.coverage_variance — Method

coverage_variance(
    ve::SimpleVariance,
    cvg::CoveragePolicy,
    state::SimpleVarianceState
) -> Any

CoveragePolicy method of the read-out arm of coverage_variance. Each asset's accumulator is divided by that asset's own count less the Bessel correction, and an asset the policy refuses is NaN.

Related

source

PortfolioOptimisers.coverage_moment_map — Function

coverage_moment_map(f::typeof(Statistics.var), v::VecNum) -> VecNum
coverage_moment_map(f::typeof(Statistics.std), v::VecNum) -> VecNum

Maps an available-case variance onto the answer the caller's verb asks for.

coverage_variance fits a variance whichever verb called it, because the state carries a second-moment accumulator and nothing else, so the standard deviation is its square root. Dispatching on the verb rather than comparing it keeps the choice at compile time.

Arguments

f: Statistics.var or Statistics.std.
v: The available-case variance.

Returns

val::VecNum: v itself, or its entrywise square root.

Related

source