GitHub

Simple expected returns

The most basic moment is the simple expected return. These types and functions implement it.

PortfolioOptimisers.SimpleExpectedReturns — Type
struct SimpleExpectedReturns{__T_w, __T_cvg, __T_cache} <: AbstractExpectedReturnsEstimator

Computes the expected returns as the sample mean of the asset returns.

w carries optional observation weights. If w is nothing, the mean is unweighted. This is the default expected returns estimator throughout the library.

Fields

  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • cvg: Optional CoveragePolicy. nothing is the reduce-and-expand path of the Coverage Universe, in which an asset that is non-finite or inactive at any observation of the window is NaN throughout the answer. A policy replaces it by available-case estimation: every cell is fitted on the observations at which the assets of that cell are all finite and active, each cell carries its own denominator, and an asset reaches the answer where admits says so.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one, and the estimator's read-out verb reads it when the caller gives no data matrix. Each propagation channel does one thing with it: factory carries it unchanged, because a factory call resolves configuration rather than the sample; port_opt_view slices it to the selected assets by index copy, so the viewed estimator answers over those assets alone; and obs_weights_view drops it, because no slice of a state exists on the observation axis. A family whose state has no exact asset slice drops it on both axes and names the reason.

Constructors

SimpleExpectedReturns(;    w::Option{<:ObsWeights} = nothing,    cvg::Option{<:CoveragePolicy} = nothing,    cache::Option{<:AbstractPartialFitState} = nothing) -> SimpleExpectedReturns

Keywords correspond to the struct's fields.

Validation

  • If w is not nothing, !isempty(w).

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

View parameters

When port_opt_view is called on this type, the following @vprop-tagged fields are automatically subset to the selected indices:

Observation weight parameters

When obs_weights_view is called on this type, the following fields are automatically indexed to the selected observations:

Examples

julia> SimpleExpectedReturns()SimpleExpectedReturns  w ┴ nothingjulia> SimpleExpectedReturns(; w = StatsBase.Weights([0.5, 0.5]))SimpleExpectedReturns  w ┴ StatsBase.Weights{Float64, Float64, Vector{Float64}}: [0.5, 0.5]

Related

source
Statistics.mean — Method
Statistics.mean(
    me::SimpleExpectedReturns,
    X::MatNum;
    dims::Int = 1,
    kwargs...
) -> ArrNum

Compute the mean of asset returns using a SimpleExpectedReturns estimator.

This method computes the expected returns as the sample mean of the input data X according to me.

Mathematical definition

Unweighted:

\[\begin{align} \hat{\mu}_j &= \frac{1}{T} \sum_{t=1}^{T} r_{tj}\,. \end{align}\]

Weighted:

\[\begin{align} \hat{\mu}_j &= \frac{\sum_{t=1}^{T} w_t \, r_{tj}}{\sum_{t=1}^{T} w_t}\,. \end{align}\]

Where:

  • $\hat{\boldsymbol{\mu}}$: $N \times 1$ vector of estimated expected returns, whose $j$-th entry is $\hat{\mu}_j$.
  • $\hat{\mu}_j$: Estimated mean of asset $j$.
  • $r_{tj}$: Return of asset $j$ at time $t$.
  • $T$: Number of observations.
  • $w_{t}$: Observation weight of observation $t$.

Algorithm

  1. Check that dims is 1 or 2.
  2. Resolve the observation weights from me.w against X, giving w.
  3. When w is nothing, take the unweighted mean of X along dims.
  4. Otherwise take the mean of X weighted by w along dims.

Arguments

  • me: Expected returns estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which to perform the computation.
  • kwargs...: Additional keyword arguments passed to Statistics.mean.

Validation

  • dims in (1, 2).

Returns

  • mu::ArrNum: Expected returns vector assets x 1 if the dims keyword does not exist or dims = 2, 1 x assets if dims = 1.

Examples

julia> X = [0.01 0.02; 0.03 0.04];julia> ser = SimpleExpectedReturns()SimpleExpectedReturns  w ┴ nothingjulia> mean(ser, X)1×2 Matrix{Float64}: 0.02  0.03julia> serw = SimpleExpectedReturns(; w = StatsBase.Weights([0.2, 0.8]))SimpleExpectedReturns  w ┴ StatsBase.Weights{Float64, Float64, Vector{Float64}}: [0.2, 0.8]julia> mean(serw, X)1×2 Matrix{Float64}: 0.026  0.036

Related

source

Incremental fit

The sample mean folds one observation at a time, so a long history need not be held or re-read. partial_fit! returns a new estimator whose cache field carries the state, and mean reads the fit off the estimator alone.

PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    state::SimpleExpectedReturnsState,
    x::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}
) -> Any

SimpleExpectedReturnsState method of partial_fit!. Folds one observation into the running count and mean.

Mathematical definition

\[\begin{align} n &\leftarrow n + 1\\ \boldsymbol{d} &= \boldsymbol{x} - \boldsymbol{\mu}\\ \boldsymbol{\mu} &\leftarrow \boldsymbol{\mu} + \frac{\boldsymbol{d}}{n}\, . \end{align}\]

Where:

  • $n$: observation count.
  • $\boldsymbol{x}$: the observation.
  • $\boldsymbol{\mu}$: the running mean.
  • $\boldsymbol{d}$: deviation of the observation from the mean before the fold.

Algorithm

  1. Refuse an observation whose length is not the number of assets the state describes.
  2. Add one to the count.
  3. Move mu in place along the deviation, by the reciprocal of the new count.
  4. Rebind the count with Accessors.@reset, and return the state.
source
PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    est::Union{AbstractEstimator, CovarianceEstimator},
    X::Union{AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}};
    dims,
    active_mask,
    estimation_mask
) -> Any

Folds observations into the sample buffer an estimator carries.

The buffering arm of partial_fit!, and the method every estimator carrying a SampleBufferState reaches. A family that folds exactly writes methods of its own, and each of them narrows the cache type parameter of its own estimator to the state that fold reads, so a buffer never meets them and this method is what remains. The state's type is therefore the whole route, and nothing refuses the step.

It is one method over both arms of the interface rather than two, because the families that refuse the step declare one method over both arms too, and a pair of narrower methods here would be ambiguous against each of them. So the arm is chosen by the type of X inside the body, which is statically resolved at every call site.

A buffer carries the per-observation masks beside the observations, so a CoveragePolicy mask threads through the wrapper as it does through an estimator's own accumulator, and the read-out hands it back to the batch verb. A wrapped estimator folded under a policy therefore answers what a batch fit over the same window under the same policy answers, and the unwrapped and wrapped paths agree.

Algorithm

  1. Read the buffer out of the cache field with assert_sample_buffer, which refuses an estimator that was never wrapped in Online.
  2. Fold a matrix and its masks through the block arm of partial_fit!, and a vector and its masks through the single-observation arm.
  3. Rebind est.cache with Accessors.@reset, and return the estimator.

Arguments

  • est: Estimator whose buffer is folded forward.
  • X: Observations to fold. A matrix holds one observation per row when dims == 1, and one per column when dims == 2. A vector is a single observation across the assets, and dims is ignored.
  • dims: Dimension along which to perform the computation.
  • active_mask: The active mask of the block, of the shape of X, or of one entry per asset when X is one observation, or nothing.
  • estimation_mask: The estimation mask, on the same terms as active_mask.

Validation

  • est carries a SampleBufferState. An ArgumentError is thrown otherwise.
  • The masks, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.
  • A buffer holding observations is given the masks it already records. An ArgumentError is thrown otherwise.
  • dims in (1, 2).

Returns

  • est: The estimator, with its cache field rebound to the buffer after the last observation.

Related

source
PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    est::Union{AbstractEstimator, CovarianceEstimator},
    X::Union{AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}};
    dims,
    active_mask,
    estimation_mask
) -> Any

Folds observations into the sample buffer an estimator carries.

The buffering arm of partial_fit!, and the method every estimator carrying a SampleBufferState reaches. A family that folds exactly writes methods of its own, and each of them narrows the cache type parameter of its own estimator to the state that fold reads, so a buffer never meets them and this method is what remains. The state's type is therefore the whole route, and nothing refuses the step.

It is one method over both arms of the interface rather than two, because the families that refuse the step declare one method over both arms too, and a pair of narrower methods here would be ambiguous against each of them. So the arm is chosen by the type of X inside the body, which is statically resolved at every call site.

A buffer carries the per-observation masks beside the observations, so a CoveragePolicy mask threads through the wrapper as it does through an estimator's own accumulator, and the read-out hands it back to the batch verb. A wrapped estimator folded under a policy therefore answers what a batch fit over the same window under the same policy answers, and the unwrapped and wrapped paths agree.

Algorithm

  1. Read the buffer out of the cache field with assert_sample_buffer, which refuses an estimator that was never wrapped in Online.
  2. Fold a matrix and its masks through the block arm of partial_fit!, and a vector and its masks through the single-observation arm.
  3. Rebind est.cache with Accessors.@reset, and return the estimator.

Arguments

  • est: Estimator whose buffer is folded forward.
  • X: Observations to fold. A matrix holds one observation per row when dims == 1, and one per column when dims == 2. A vector is a single observation across the assets, and dims is ignored.
  • dims: Dimension along which to perform the computation.
  • active_mask: The active mask of the block, of the shape of X, or of one entry per asset when X is one observation, or nothing.
  • estimation_mask: The estimation mask, on the same terms as active_mask.

Validation

  • est carries a SampleBufferState. An ArgumentError is thrown otherwise.
  • The masks, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.
  • A buffer holding observations is given the masks it already records. An ArgumentError is thrown otherwise.
  • dims in (1, 2).

Returns

  • est: The estimator, with its cache field rebound to the buffer after the last observation.

Related

source
Statistics.mean — Method
Statistics.mean(
    me::SimpleExpectedReturns,
    state::SimpleExpectedReturnsState
) -> VecNum
Statistics.mean(
    me::SimpleExpectedReturns
) -> VecNum

Read the mean of an incremental fit out of a SimpleExpectedReturnsState.

The two-argument method reads a state the caller holds, and the one-argument method reads the state the cache field of me carries. Both return the running mean as a vector, assets × 1, where the batch method over a matrix returns a row when dims = 1.

Algorithm

  1. Refuse a configuration no incremental fit reproduces, with assert_partial_fittable.
  2. Return a vector of NaN when the state holds no observation, in the way min_obs reads an asset with too few observations.
  3. Otherwise return the running mean.

Arguments

  • me: Expected returns estimator.
  • state: The state to read.

Validation

  • me carries no observation weights. An ArgumentError is thrown otherwise.
  • me.cache is not nothing, for the one-argument method. An ArgumentError is thrown otherwise.

Returns

  • mu::VecNum: Running mean of the fit, assets × 1, or NaN where the state holds no observation.

Examples

julia> me = foldl(partial_fit!, eachrow([1.0 2.0; 3.0 4.0]); init = SimpleExpectedReturns());julia> mean(me)2-element Vector{Float64}: 2.0 3.0

Related

source
PortfolioOptimisers.port_opt_view — Method
port_opt_view(
    x::SimpleExpectedReturnsState,
    i,
    args...
) -> Union{SimpleExpectedReturnsState{_A, _B, Nothing} where {_A, _B}, SimpleExpectedReturnsState{_A, _B, __T_cvg} where {_A, _B, __T_cvg<:CoverageCounts}}

Slices a SimpleExpectedReturnsState to the selected assets.

The Welford mean of one asset reads that asset's observations alone, so the slice of the state is the state of the sliced universe, entry for entry, and the count is shared by every asset and passes through. The slice copies by index and does not view: a later partial_fit! on the viewed estimator would otherwise write through into the arrays of the estimator the view was taken from.

Arguments

  • x: The state to slice.
  • i: Index or indices of the assets to keep.
  • args...: Additional positional arguments (ignored).

Returns

  • state::SimpleExpectedReturnsState: The state of the same sample over the selected assets.

Related

source
PortfolioOptimisers.merge_states — Method
merge_states(
    a::SimpleExpectedReturnsState,
    b::SimpleExpectedReturnsState
) -> Union{SimpleExpectedReturnsState{_A, _B, Nothing} where {_A, _B}, SimpleExpectedReturnsState{_A, _B, __T_cvg} where {_A, _B, __T_cvg<:(CoverageCounts{_A, Nothing} where _A)}}

Folds two SimpleExpectedReturnsState fitted on disjoint blocks into the state of the concatenated block.

Algorithm

  1. Refuse the pair with assert_mergeable_states.
  2. Fold the counts and the means with chan_merge, whose accumulator argument is false, the zero of a state that carries no accumulator. The accumulator it returns is discarded.

Arguments

  • a: The state of the first block of observations.
  • b: The state of the second block of observations.

Validation

Returns

  • state::SimpleExpectedReturnsState: The state the two blocks give when they are fitted as one block.

Related

source

Available-case fit

With a CoveragePolicy in its cvg field the estimator fits each asset on that asset's own finite and active observations, and PortfolioOptimisers.coverage_mean routes between that arm and the Coverage Universe one.

PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    state::SimpleExpectedReturnsState,
    x::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    _::Nothing,
    _::Union{Nothing, AbstractVector{<:Bool}}
) -> Any

Nothing method of the coverage arm of partial_fit!. An estimator that carries no CoveragePolicy folds through partial_fit!(state::SimpleExpectedReturnsState, x::VecNum), and the active mask is ignored: a plain state carries no per-cell count for the mask to gate, and its universe is the Coverage Universe the read-out already reduces to.

Related

source
PortfolioOptimisers.partial_fit! — Method
partial_fit!(
    state::SimpleExpectedReturnsState,
    x::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    cvg::CoveragePolicy,
    active_mask::Union{Nothing, AbstractVector{<:Bool}}
) -> Any

CoveragePolicy method of the coverage arm of partial_fit!. Folds one observation into the running per-asset count and mean, reading each asset's own observations alone.

Mathematical definition

\[\begin{align} \nu_j &\leftarrow \nu_j + 1\\ \mu_j &\leftarrow \mu_j + \frac{r_{tj} - \mu_j}{\nu_j}\, , \end{align}\]

for every asset $j$ that is finite and active at observation $t$, and neither line for an asset that is not. Where:

  • $\nu_j$: the number of observations at which asset $j$ was finite and active.
  • $r_{tj}$: Return of asset $j$ at time $t$.
  • $\mu_j$: the running mean of asset $j$.

An asset with no observation keeps $\mu_j = 0$ and $\nu_j = 0$, and the read-out answers NaN for it, so the zero is never read as an estimate. This is Welford's recursion per asset, so a mean folded observation by observation is the mean of the same rows fitted as a block.

Algorithm

  1. Refuse an observation whose length is not the number of assets the state describes.
  2. Read the valid assets and the newly inactive ones with coverage_valid.
  3. Apply the algorithm's fold-time rule with fold_inactive!.
  4. Fold each valid asset's return into its own count and mean.
  5. Move the per-asset bookkeeping on with coverage_step!, add one to the observation count, and return the state.

Arguments

  • state: The state to fold into, mutated in place.
  • x: One observation, one entry per asset.
  • cvg: The policy the estimator carries.
  • active_mask: The active mask of the Asset Panel at this observation, or nothing.

Validation

  • length(x) is the number of assets the state describes. A DimensionMismatch is thrown otherwise.

Returns

  • state::SimpleExpectedReturnsState: The state after the observation.

Related

source
PortfolioOptimisers.fold_inactive! — Method
fold_inactive!(
    _::ResetCoverage,
    state::SimpleExpectedReturnsState,
    ni::AbstractVector{<:Bool}
) -> SimpleExpectedReturnsState

SimpleExpectedReturnsState method of fold_inactive! under ResetCoverage. Zeroes the count, the centre and the running mean of every asset that has just gone inactive, so that a relisting starts the asset cold. The centre of a per-asset state is nothing, because its mu is already the cell's centre, and coverage_reset! passes that through.

Related

source