GitHub

Exponentially Weighted Variance

Types

PortfolioOptimisers.ExpWeightedVariance — Type
struct ExpWeightedVariance{__T_decay, __T_min_obs, __T_centred, __T_cache} <: AbstractVarianceEstimator

Estimates per-asset variance by an exponentially weighted recursion that freezes on a holiday and resets on an inactive period.

The recursion is seeded at zero, so a newly listed asset starts from a cold state and the output divides out the damping that the cold start costs. An asset below min_obs valid observations is NaN, and so is an asset that the active mask leaves inactive at the last observation.

Keeping a young asset investable has a cost the prior pays for it. A prior fitted with this estimator zero-fills the rows the asset was missing through scenario_fill, because every consumer of a Prior Result reads its returns matrix; a scenario-based measure then reads a zero return where the asset had none and understates that asset's risk over those rows, while the variance stays the estimate this recursion made from the rows it saw. The fill is silent at or below the fitting prior's own fill_limit field, a share of that asset's own observations, warns above it, and refuses any fill under strict; fill_limit defaults to nothing, and this family carries no CoveragePolicy to derive a limit from, so every fill is named.

Fields

  • decay: Exponential decay factor for the exponentially weighted estimator.
  • min_obs: Minimum number of observations required before the estimator produces a valid result.
  • centred: Whether to treat the returns as pre-centred (mean zero). If false, the location is estimated online.
  • cache: Running state of an incremental fit, or nothing before the first call to partial_fit!. It is the one Result this estimator holds, and its type bound is the enforcement of that exception. A fit over a matrix ignores it.

Constructors

ExpWeightedVariance(;    decay::Number = exp2(-inv(40.0)),    min_obs::Integer = round(Int, max(1, inv(log2(inv(decay))))),    centred::Bool = false,    cache::Option{<:AbstractPartialFitState} = nothing) -> ExpWeightedVariance

Keywords correspond to the struct's fields.

Validation

  • decay > 0.
  • min_obs > 0.

Mathematical definition

The internal state of asset $i$ after $n_i$ valid observations is

\[S_i = (1 - \lambda) \sum_{k=0}^{n_i - 1} \lambda^{k} e_{i, n_i - k}^{2},\]

and the reported variance divides out the weights the cold start never accumulated,

\[\hat{\sigma}^{2}_i = \frac{S_i}{1 - \lambda^{n_i}}.\]

Where:

  • $\lambda$: decay.
  • $e_{i, t}$: the return of asset $i$ at the valid observation $t$, less the running location where centred is false, and the return itself where it is true.
  • $n_i$: the count of valid observations of asset $i$.

Examples

julia> ce = ExpWeightedVariance();julia> ce.decay ≈ exp2(-inv(40.0))truejulia> ce.min_obs40

Related

source

Functions

Statistics.var — Method
Statistics.var(
    ce::ExpWeightedVariance,
    X::MatNum;
    dims::Int = 1,
    active_mask::Option{<:AbstractMatrix{<:Bool}} = nothing,
    kwargs...
) -> Vector{<:Number}

Compute the exponentially weighted variance of each asset.

Iterates over the observation dimension of X, updating an online variance cache at each step, then applies the cold-start bias correction and blanks every asset that is not ready.

Arguments

  • ce: Exponentially weighted variance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which to perform the computation.
  • active_mask: Optional boolean matrix with the same size as X. An asset whose entry is false is inactive at that observation: its state is reset and its answer is NaN while it stays inactive. With nothing every asset is active, so a non-finite return reads as a holiday and the variance freezes.
  • kwargs: Additional keyword arguments (ignored).

Validation

  • dims in (1, 2).
  • If active_mask is not nothing, size(X) == size(active_mask).

Returns

  • var::Vector{<:Number}: Per-asset variance vector of length assets. An asset with fewer than ce.min_obs valid observations is NaN.

Examples

julia> X = [0.01 -0.02; -0.015 0.03; 0.02 -0.01; -0.005 0.012];julia> ce = ExpWeightedVariance(; decay = 0.9, min_obs = 2);julia> length(var(ce, X))2

Related

source
Statistics.std — Method
Statistics.std(
    ce::ExpWeightedVariance,
    X::MatNum;
    dims::Int = 1,
    active_mask::Option{<:AbstractMatrix{<:Bool}} = nothing,
    kwargs...
) -> Vector{<:Number}

Compute the exponentially weighted volatility of each asset.

This is the square root of the variance of the same call.

Arguments

  • ce: Exponentially weighted variance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which to perform the computation.
  • active_mask: Optional boolean matrix with the same size as X.
  • kwargs: Additional keyword arguments (ignored).

Returns

  • std::Vector{<:Number}: Per-asset volatility vector of length assets.

Related

source
Statistics.var — Method
Statistics.var(
    ce::ExpWeightedVariance,
    X::MatNum,
    pnl::Option{<:AssetPanel};
    dims::Int = 1,
    kwargs...
) -> Vector{<:Number}

Compute the exponentially weighted variance from a window of an Asset Panel.

This estimator is mask-aware, so it overrides the reduce-and-expand root of the verb and reads the panel's active mask itself. The answer therefore lives on the whole universe rather than on the Coverage Universe: a young asset that lists inside the window is answered from the observations it has, and it is NaN only while it stays below ce.min_obs.

Arguments

  • ce: Exponentially weighted variance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • pnl: Optional AssetPanel, whose active mask the Coverage Universe of the fit is derived from. nothing makes the rule finiteness alone.
  • dims: Dimension along which to perform the computation.
  • kwargs: Additional keyword arguments (ignored).

Returns

  • var::Vector{<:Number}: Per-asset variance vector of length assets.

Related

source
Statistics.std — Method
Statistics.std(
    ce::ExpWeightedVariance,
    X::MatNum,
    pnl::Option{<:AssetPanel};
    dims::Int = 1,
    kwargs...
) -> Vector{<:Number}

Compute the exponentially weighted volatility from a window of an Asset Panel.

This is the square root of the variance of the same call, and it reads the panel's active mask through the same override.

Arguments

  • ce: Exponentially weighted variance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • pnl: Optional AssetPanel, whose active mask the Coverage Universe of the fit is derived from. nothing makes the rule finiteness alone.
  • dims: Dimension along which to perform the computation.
  • kwargs: Additional keyword arguments (ignored).

Returns

  • std::Vector{<:Number}: Per-asset volatility vector of length assets.

Related

source
PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    est::Union{AbstractEstimator, CovarianceEstimator},
    X::Union{AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}};
    dims,
    active_mask,
    estimation_mask
) -> Any

Folds observations into the sample buffer an estimator carries.

The buffering arm of partial_fit!, and the method every estimator carrying a SampleBufferState reaches. A family that folds exactly writes methods of its own, and each of them narrows the cache type parameter of its own estimator to the state that fold reads, so a buffer never meets them and this method is what remains. The state's type is therefore the whole route, and nothing refuses the step.

It is one method over both arms of the interface rather than two, because the families that refuse the step declare one method over both arms too, and a pair of narrower methods here would be ambiguous against each of them. So the arm is chosen by the type of X inside the body, which is statically resolved at every call site.

A buffer carries the per-observation masks beside the observations, so a CoveragePolicy mask threads through the wrapper as it does through an estimator's own accumulator, and the read-out hands it back to the batch verb. A wrapped estimator folded under a policy therefore answers what a batch fit over the same window under the same policy answers, and the unwrapped and wrapped paths agree.

Algorithm

  1. Read the buffer out of the cache field with assert_sample_buffer, which refuses an estimator that was never wrapped in Online.
  2. Fold a matrix and its masks through the block arm of partial_fit!, and a vector and its masks through the single-observation arm.
  3. Rebind est.cache with Accessors.@reset, and return the estimator.

Arguments

  • est: Estimator whose buffer is folded forward.
  • X: Observations to fold. A matrix holds one observation per row when dims == 1, and one per column when dims == 2. A vector is a single observation across the assets, and dims is ignored.
  • dims: Dimension along which to perform the computation.
  • active_mask: The active mask of the block, of the shape of X, or of one entry per asset when X is one observation, or nothing.
  • estimation_mask: The estimation mask, on the same terms as active_mask.

Validation

  • est carries a SampleBufferState. An ArgumentError is thrown otherwise.
  • The masks, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.
  • A buffer holding observations is given the masks it already records. An ArgumentError is thrown otherwise.
  • dims in (1, 2).

Returns

  • est: The estimator, with its cache field rebound to the buffer after the last observation.

Related

source
PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    est::Union{AbstractEstimator, CovarianceEstimator},
    X::Union{AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}};
    dims,
    active_mask,
    estimation_mask
) -> Any

Folds observations into the sample buffer an estimator carries.

The buffering arm of partial_fit!, and the method every estimator carrying a SampleBufferState reaches. A family that folds exactly writes methods of its own, and each of them narrows the cache type parameter of its own estimator to the state that fold reads, so a buffer never meets them and this method is what remains. The state's type is therefore the whole route, and nothing refuses the step.

It is one method over both arms of the interface rather than two, because the families that refuse the step declare one method over both arms too, and a pair of narrower methods here would be ambiguous against each of them. So the arm is chosen by the type of X inside the body, which is statically resolved at every call site.

A buffer carries the per-observation masks beside the observations, so a CoveragePolicy mask threads through the wrapper as it does through an estimator's own accumulator, and the read-out hands it back to the batch verb. A wrapped estimator folded under a policy therefore answers what a batch fit over the same window under the same policy answers, and the unwrapped and wrapped paths agree.

Algorithm

  1. Read the buffer out of the cache field with assert_sample_buffer, which refuses an estimator that was never wrapped in Online.
  2. Fold a matrix and its masks through the block arm of partial_fit!, and a vector and its masks through the single-observation arm.
  3. Rebind est.cache with Accessors.@reset, and return the estimator.

Arguments

  • est: Estimator whose buffer is folded forward.
  • X: Observations to fold. A matrix holds one observation per row when dims == 1, and one per column when dims == 2. A vector is a single observation across the assets, and dims is ignored.
  • dims: Dimension along which to perform the computation.
  • active_mask: The active mask of the block, of the shape of X, or of one entry per asset when X is one observation, or nothing.
  • estimation_mask: The estimation mask, on the same terms as active_mask.

Validation

  • est carries a SampleBufferState. An ArgumentError is thrown otherwise.
  • The masks, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.
  • A buffer holding observations is given the masks it already records. An ArgumentError is thrown otherwise.
  • dims in (1, 2).

Returns

  • est: The estimator, with its cache field rebound to the buffer after the last observation.

Related

source
Statistics.var — Method
Statistics.var(ce::ExpWeightedVariance; kwargs...) -> Vector{<:Number}

Read the exponentially weighted variance out of the estimator's own state.

The one-argument form is what an incremental fit answers: partial_fit! leaves the state in the cache field, and this verb turns it into the ordinary answer. An estimator that has been given no observation carries no state, so the call is refused rather than answered with a zero.

Arguments

  • ce: Exponentially weighted variance estimator carrying a state.
  • kwargs: Additional keyword arguments (ignored).

Validation

  • ce.cache is not nothing. An ArgumentError is thrown otherwise.

Returns

  • var::Vector{<:Number}: Per-asset variance vector of length assets.

Examples

julia> X = [0.01 -0.02; -0.015 0.03; 0.02 -0.01; -0.005 0.012];julia> ce = partial_fit!(ExpWeightedVariance(; decay = 0.9, min_obs = 2), X);julia> length(var(ce))2julia> var(ExpWeightedVariance())ERROR: ArgumentError: `ce` holds no partial-fit state, so there is nothing to read. Call `partial_fit!(ce, X)` first, or `var(ce, X)` for a fit over a whole sample.[...]

Related

source
Statistics.std — Method
Statistics.std(ce::ExpWeightedVariance, state::ExpWeightedVarianceState; kwargs...) -> Vector{<:Number}

Read the exponentially weighted volatility out of a state the caller holds.

Arguments

  • ce: Exponentially weighted variance estimator.
  • state::ExpWeightedVarianceState: The state to read.
  • kwargs: Additional keyword arguments (ignored).

Returns

  • std::Vector{<:Number}: Per-asset volatility vector.

Related

source
Statistics.std — Method
Statistics.std(ce::ExpWeightedVariance; kwargs...) -> Vector{<:Number}

Read the exponentially weighted volatility out of the estimator's own state.

Arguments

  • ce: Exponentially weighted variance estimator carrying a state.
  • kwargs: Additional keyword arguments (ignored).

Validation

  • ce.cache is not nothing. An ArgumentError is thrown otherwise.

Returns

  • std::Vector{<:Number}: Per-asset volatility vector of length assets.

Related

source
PortfolioOptimisers.merge_states — Method
merge_states(
    a::ExpWeightedVarianceState,
    b::ExpWeightedVarianceState
)

Refuses to merge two ExpWeightedVarianceState.

An exponentially weighted state folds forward exactly, S = λ^{n_b} S_a + S_b, but only while no asset resets inside the second block. The state records the count that a reset zeroed and not the reset itself, so the two cases are indistinguishable after the fact and a merge would silently keep a history the reset discarded. Fold the second block into the first with partial_fit! instead.

Arguments

  • a: State of the first block.
  • b: State of the second block.

Validation

  • The pair is refused with an ArgumentError.

Related

source