Partial fit: private API

PortfolioOptimisers.assert_partial_fit_state — Function
assert_partial_fit_state(
    n::Integer,
    mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
)
assert_partial_fit_state(
    n::Integer,
    mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    M::Union{Nothing, AbstractArray{<:Union{Number, AbstractJuMPScalar}}}
)

Refuses a partial-fit state whose count, mean or accumulator cannot describe a sample.

The states of the sample mean, variance and covariance hold the same three quantities, so one check applies to the three. The count must not be negative, and the mean must not be empty. Each axis of the accumulator must have the length of the mean. The accumulator holds one entry for each asset in a variance state, and it is a co-moment matrix in a covariance state. The axis rule reads both.

The inner constructor of each state calls this function, so a fold, a rebuild and a lens write run it. Each check costs O(1). The scan for finite values costs O(N²) for a covariance state, which is as much as the fold of one observation, so the keyword constructor alone runs it, see assert_partial_fit_values.

Algorithm

  1. Refuse a negative n.
  2. Refuse an empty mu.
  3. Return when M is nothing, which is the mean-only state.
  4. Refuse an M whose any axis does not have the length of mu.

Arguments

  • n: Number of observations folded into the state.
  • mu: Running mean of the observations folded into the state, assets × 1.
  • M: The second-moment accumulator, or nothing for a state that holds a mean alone.

Validation

  • n >= 0. The function throws a DomainError otherwise.
  • !isempty(mu). The function throws an IsEmptyError otherwise.
  • Every axis of M has the length of mu. The function throws a DimensionMismatch otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_partial_fit_values — Function
assert_partial_fit_values(
    mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
)
assert_partial_fit_values(
    mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    M::Union{Nothing, AbstractArray{<:Union{Number, AbstractJuMPScalar}}}
)

Refuses a partial-fit state whose mean or accumulator holds a value that is not finite.

The keyword constructor of the states of the sample mean, variance and covariance calls this function on the values that a caller gives. The inner constructor does not, because the fold of one observation builds the state through it, and the scan of M costs as much as that fold. assert_partial_fit_state holds the checks that each route runs.

Arguments

  • mu: Running mean of the observations folded into the state, assets × 1.
  • M: The second-moment accumulator, or nothing for a state that holds a mean alone.

Validation

  • Every entry of mu is finite. The function throws an IsNonFiniteError otherwise.
  • Every entry of M is finite. The function throws an IsNonFiniteError otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.partial_fit_cache — Function
partial_fit_cache(
    est::Union{AbstractEstimator, CovarianceEstimator}
) -> Any

Returns the partial-fit state that an estimator holds, and refuses an estimator that holds none.

A batch verb with the estimator alone, and no data, reads the state from the cache field. That field holds nothing until the first fold. The refusal names the verb that fills the field, so the caller gets an instruction and not a MethodError. An estimator with no cache field gets the same refusal.

Arguments

  • est: The estimator whose cache field holds the state.

Validation

  • est.cache is not nothing. The function throws an ArgumentError otherwise.

Returns

  • state::AbstractPartialFitState: The state that the estimator holds.

Related

source
PortfolioOptimisers.observation_count — Function
observation_count(est::Union{AbstractEstimator, StatsBase.CovarianceEstimator}) -> Integer
observation_count(state::AbstractPartialFitState) -> Integer

Returns the number of observations that an estimator folded into its partial-fit state.

The states of the sample moments and the sample buffer record this count in an n field. A call with no data that needs the sample size, and not the sample, reads the count from the state. Statistics.cov of a PortfolioOptimisersCovariance is one such call. Its matrix processing reads size(X, 1) alone in a batch fit, and the folded count is that number, with the rows that hold a NaN.

The estimator method reads the state from est.cache, and the state method reads the count. A state that keeps the count in another field adds its own state method. The two exponentially weighted covariance states do this. A composite that folds through an inner estimator, and keeps no state of its own, adds an estimator method that reads the inner estimator.

Arguments

  • est: The estimator whose state holds the count.
  • state: The state that holds the count.

Validation

  • est.cache is not nothing. The function throws an ArgumentError otherwise.
  • The state has an n field, or a method of its own. The function throws an ArgumentError otherwise.

Returns

  • n::Integer: The number of observations folded so far.

Related

source
observation_count(
    ce::PortfolioOptimisersCovariance{<:Any, <:Any, Nothing}
) -> Any

PortfolioOptimisersCovariance method of observation_count, for a composite whose own cache is nothing.

The composite then folds through ce.ce and keeps no state of its own, so the method reads the count of ce.ce. A composite whose cache holds a SampleBufferState reads the count of its buffer through the generic method instead.

Arguments

  • ce: Covariance estimator.

Validation

  • ce.ce holds a partial-fit state with a count, as observation_count states. An ArgumentError is thrown otherwise.

Returns

  • n::Integer: The number of observations that ce.ce folded.

Related

source
observation_count(
    est::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> Any

SeedWindowed method of observation_count.

A windowed estimator under a SeedWindow keeps no state of its own, and its inner estimator holds the state of the fold. So the method reads the count of the inner estimator. That count is the number of rows that the inner estimator folded: the rows of the first block that the window kept, and every later row.

Arguments

  • est: Windowed estimator under a seed window.

Validation

  • The inner estimator holds a partial-fit state with a count, as observation_count states. An ArgumentError is thrown otherwise.

Returns

  • n::Integer: The number of observations that the inner estimator folded.

Related

source
observation_count(state::ExpWeightedCovarianceState) -> Any

ExpWeightedCovarianceState method of observation_count.

The state counts the observations of each asset in obs_count, and a reset sets such a count to zero. So the method reads rows, the number of folded rows, which is the size(X, 1) of a batch fit over the same rows.

Arguments

  • state: The state that holds the count.

Returns

  • n::Integer: The number of observations folded so far.

Related

source
observation_count(
    state::RegimeAdjustedCovarianceState
) -> Any

RegimeAdjustedCovarianceState method of observation_count.

The state counts the observations of each asset in obs_count, and a reset sets such a count to zero. n_regime_obs counts only the observations that the regime statistic scored. So the method reads rows, the number of folded rows, which is the size(X, 1) of a batch fit over the same rows.

Arguments

  • state: The state that holds the count.

Returns

  • n::Integer: The number of observations folded so far.

Related

source
observation_count(
    ce::GeodesicShrinkageCovariance{<:Any, <:Any, <:Any, <:Any, Nothing}
) -> Any

GeodesicShrinkageCovariance method of observation_count, for an estimator whose own cache is nothing.

The estimator then folds through ce.ce and keeps no state of its own, so the method reads the count of ce.ce. An estimator whose cache holds a SampleBufferState reads the count of its buffer through the generic method instead.

Arguments

  • ce: The geodesic shrinkage covariance estimator.

Validation

  • ce.ce holds a partial-fit state with a count, as observation_count states. An ArgumentError is thrown otherwise.

Returns

  • n::Integer: The number of observations that ce.ce folded.

Related

source
PortfolioOptimisers.shared_series_member_rows — Function
shared_series_member_rows(X::MatNum, gaps::Nothing; kwargs...)
shared_series_member_rows(X::MatNum, gaps::AbstractVector{Bool}; active_mask = nothing,
                          kwargs...)

Returns the observations of a carry step that its members fold, and the keyword arguments of their fold.

A carry fold adds each observation to running moments. A batch fit drops an observation with a Shared Series gap, see shared_series_view, so the running moments must not get it. The carried rows keep it, because the Fold Context of an optimiser holds one row for each observation. So the members fold the unmarked rows of X and the same rows of the active mask. The carried rows keep every row and its mark.

Algorithm

The method that Julia selects is the algorithm.

  1. When gaps is nothing, return X and the keyword arguments with no change.
  2. Otherwise, find the rows that gaps does not mark. Return the views of X and of active_mask at those rows, and the other keyword arguments.

Arguments

  • X: The observations of the step, observations × assets.
  • gaps: One mark for each observation, true at an observation with a Shared Series gap, or nothing.
  • active_mask: The active mask of the block, of the shape of X, or nothing. A buffer records it for every observation it holds or for none of them.
  • kwargs...: The other keyword arguments of the fold of a member, unchanged.

Returns

  • (X, kwargs)::Tuple: The observations that the members fold, and the keyword arguments of their fold.

Related

source