Partial fit: private API
PortfolioOptimisers.assert_partial_fit_state — Function
assert_partial_fit_state(
n::Integer,
mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
)
assert_partial_fit_state(
n::Integer,
mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
M::Union{Nothing, AbstractArray{<:Union{Number, AbstractJuMPScalar}}}
)
Refuses a partial-fit state whose count, mean or accumulator cannot describe a sample.
The states of the sample mean, variance and covariance hold the same three quantities, so one check applies to the three. The count must not be negative, and the mean must not be empty. Each axis of the accumulator must have the length of the mean. The accumulator holds one entry for each asset in a variance state, and it is a co-moment matrix in a covariance state. The axis rule reads both.
The inner constructor of each state calls this function, so a fold, a rebuild and a lens write run it. Each check costs O(1). The scan for finite values costs O(N²) for a covariance state, which is as much as the fold of one observation, so the keyword constructor alone runs it, see assert_partial_fit_values.
Algorithm
- Refuse a negative
n. - Refuse an empty
mu. - Return when
Misnothing, which is the mean-only state. - Refuse an
Mwhose any axis does not have the length ofmu.
Arguments
n: Number of observations folded into the state.mu: Running mean of the observations folded into the state,assets × 1.M: The second-moment accumulator, ornothingfor a state that holds a mean alone.
Validation
n >= 0. The function throws aDomainErrorotherwise.!isempty(mu). The function throws anIsEmptyErrorotherwise.- Every axis of
Mhas the length ofmu. The function throws aDimensionMismatchotherwise.
Returns
nothing.
Related
PortfolioOptimisers.assert_partial_fit_values — Function
assert_partial_fit_values(
mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
)
assert_partial_fit_values(
mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
M::Union{Nothing, AbstractArray{<:Union{Number, AbstractJuMPScalar}}}
)
Refuses a partial-fit state whose mean or accumulator holds a value that is not finite.
The keyword constructor of the states of the sample mean, variance and covariance calls this function on the values that a caller gives. The inner constructor does not, because the fold of one observation builds the state through it, and the scan of M costs as much as that fold. assert_partial_fit_state holds the checks that each route runs.
Arguments
mu: Running mean of the observations folded into the state,assets × 1.M: The second-moment accumulator, ornothingfor a state that holds a mean alone.
Validation
- Every entry of
muis finite. The function throws anIsNonFiniteErrorotherwise. - Every entry of
Mis finite. The function throws anIsNonFiniteErrorotherwise.
Returns
nothing.
Related
PortfolioOptimisers.partial_fit_cache — Function
partial_fit_cache(
est::Union{AbstractEstimator, CovarianceEstimator}
) -> Any
Returns the partial-fit state that an estimator holds, and refuses an estimator that holds none.
A batch verb with the estimator alone, and no data, reads the state from the cache field. That field holds nothing until the first fold. The refusal names the verb that fills the field, so the caller gets an instruction and not a MethodError. An estimator with no cache field gets the same refusal.
Arguments
est: The estimator whosecachefield holds the state.
Validation
est.cacheis notnothing. The function throws anArgumentErrorotherwise.
Returns
state::AbstractPartialFitState: The state that the estimator holds.
Related
PortfolioOptimisers.observation_count — Function
observation_count(est::Union{AbstractEstimator, StatsBase.CovarianceEstimator}) -> Integer
observation_count(state::AbstractPartialFitState) -> IntegerReturns the number of observations that an estimator folded into its partial-fit state.
The states of the sample moments and the sample buffer record this count in an n field. A call with no data that needs the sample size, and not the sample, reads the count from the state. Statistics.cov of a PortfolioOptimisersCovariance is one such call. Its matrix processing reads size(X, 1) alone in a batch fit, and the folded count is that number, with the rows that hold a NaN.
The estimator method reads the state from est.cache, and the state method reads the count. A state that keeps the count in another field adds its own state method. The two exponentially weighted covariance states do this. A composite that folds through an inner estimator, and keeps no state of its own, adds an estimator method that reads the inner estimator.
Arguments
est: The estimator whose state holds the count.state: The state that holds the count.
Validation
est.cacheis notnothing. The function throws anArgumentErrorotherwise.- The state has an
nfield, or a method of its own. The function throws anArgumentErrorotherwise.
Returns
n::Integer: The number of observations folded so far.
Related
observation_count(
ce::PortfolioOptimisersCovariance{<:Any, <:Any, Nothing}
) -> Any
PortfolioOptimisersCovariance method of observation_count, for a composite whose own cache is nothing.
The composite then folds through ce.ce and keeps no state of its own, so the method reads the count of ce.ce. A composite whose cache holds a SampleBufferState reads the count of its buffer through the generic method instead.
Arguments
ce: Covariance estimator.
Validation
ce.ceholds a partial-fit state with a count, asobservation_countstates. AnArgumentErroris thrown otherwise.
Returns
n::Integer: The number of observations thatce.cefolded.
Related
observation_count(
est::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> Any
SeedWindowed method of observation_count.
A windowed estimator under a SeedWindow keeps no state of its own, and its inner estimator holds the state of the fold. So the method reads the count of the inner estimator. That count is the number of rows that the inner estimator folded: the rows of the first block that the window kept, and every later row.
Arguments
est: Windowed estimator under a seed window.
Validation
- The inner estimator holds a partial-fit state with a count, as
observation_countstates. AnArgumentErroris thrown otherwise.
Returns
n::Integer: The number of observations that the inner estimator folded.
Related
observation_count(state::ExpWeightedCovarianceState) -> Any
ExpWeightedCovarianceState method of observation_count.
The state counts the observations of each asset in obs_count, and a reset sets such a count to zero. So the method reads rows, the number of folded rows, which is the size(X, 1) of a batch fit over the same rows.
Arguments
state: The state that holds the count.
Returns
n::Integer: The number of observations folded so far.
Related
observation_count(
state::RegimeAdjustedCovarianceState
) -> Any
RegimeAdjustedCovarianceState method of observation_count.
The state counts the observations of each asset in obs_count, and a reset sets such a count to zero. n_regime_obs counts only the observations that the regime statistic scored. So the method reads rows, the number of folded rows, which is the size(X, 1) of a batch fit over the same rows.
Arguments
state: The state that holds the count.
Returns
n::Integer: The number of observations folded so far.
Related
observation_count(
ce::GeodesicShrinkageCovariance{<:Any, <:Any, <:Any, <:Any, Nothing}
) -> Any
GeodesicShrinkageCovariance method of observation_count, for an estimator whose own cache is nothing.
The estimator then folds through ce.ce and keeps no state of its own, so the method reads the count of ce.ce. An estimator whose cache holds a SampleBufferState reads the count of its buffer through the generic method instead.
Arguments
ce: The geodesic shrinkage covariance estimator.
Validation
ce.ceholds a partial-fit state with a count, asobservation_countstates. AnArgumentErroris thrown otherwise.
Returns
n::Integer: The number of observations thatce.cefolded.
Related
PortfolioOptimisers.shared_series_member_rows — Function
shared_series_member_rows(X::MatNum, gaps::Nothing; kwargs...)
shared_series_member_rows(X::MatNum, gaps::AbstractVector{Bool}; active_mask = nothing,
kwargs...)Returns the observations of a carry step that its members fold, and the keyword arguments of their fold.
A carry fold adds each observation to running moments. A batch fit drops an observation with a Shared Series gap, see shared_series_view, so the running moments must not get it. The carried rows keep it, because the Fold Context of an optimiser holds one row for each observation. So the members fold the unmarked rows of X and the same rows of the active mask. The carried rows keep every row and its mark.
Algorithm
The method that Julia selects is the algorithm.
- When
gapsisnothing, returnXand the keyword arguments with no change. - Otherwise, find the rows that
gapsdoes not mark. Return the views ofXand ofactive_maskat those rows, and the other keyword arguments.
Arguments
X: The observations of the step,observations × assets.gaps: One mark for each observation,trueat an observation with a Shared Series gap, ornothing.active_mask: The active mask of the block, of the shape ofX, ornothing. A buffer records it for every observation it holds or for none of them.kwargs...: The other keyword arguments of the fold of a member, unchanged.
Returns
(X, kwargs)::Tuple: The observations that the members fold, and the keyword arguments of their fold.
Related