The prior family on the partial-fit seam

A prior takes the online step by one of two routes, and the type of the state it carries is the route.

A prior that folds its moments exactly carries a PortfolioOptimisers.PriorCarryState: the moments come off its members' own folds and the rows are kept only because a LowOrderPrior carries X for the scenario risk measures, so a read-out never reads them. A prior that has no recursion carries a PortfolioOptimisers.SampleBufferState, which Online seeds, and its read-out is the batch verb over the rows the buffer kept. The factor observations ride inside that same buffer, and whether the fold records them is decided by the estimator tree through PortfolioOptimisers.needs_factor_returns, so the fold mirrors the batch verb's arity.

A host that carries the observations folds every member that folds and runs the batch verb over its own rows for every member that does not, so a caller writes the estimator they would write in batch. ADR 0136 records the decision.

PortfolioOptimisers.partial_fit! — Method

partial_fit!(state::PriorCarryState, x::VecNum)
partial_fit!(state::PriorCarryState, X::MatNum; dims::Int = 1)

Folds observations into the buffer a PriorCarryState carries.

The buffer's own fold, forwarded, keyword arguments included: a CoveragePolicy mask travels into the buffer beside the rows it explains, exactly as it does for a SampleBufferState an Online seeded. No factor observation reaches the buffer, because the one carrying prior, EmpiricalPrior, never reads one and drops it before the carry. The named-asset set is untouched: a fold reads no moment and names nothing, and only a read-out can find an asset to name.

Arguments

state: The carry state to fold into.
x: One observation, whose entries are the assets.
X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
dims: Dimension along which to perform the computation.

Returns

state::PriorCarryState: The state after the last observation.

Related

source

PortfolioOptimisers.port_opt_view — Method

port_opt_view(
    x::PriorCarryState,
    i,
    args...
) -> PriorCarryState{_A, Set{Int64}} where _A

Slices a PriorCarryState to the selected assets.

The buffer is sliced as it always is, and the named-asset set is remapped rather than dropped or carried: its entries are asset indices, and the slice renumbers the assets. An asset the state had already named keeps its silence at its new index, and an asset the slice removed is forgotten with it.

Arguments

x: The state to slice.
i: Index or indices of the assets to keep.
args...: Additional positional arguments, forwarded to the buffer.

Returns

state::PriorCarryState: The state of the same observations over the selected assets.

Related

source

PortfolioOptimisers.merge_states — Method

merge_states(
    a::PriorCarryState,
    b::PriorCarryState
) -> PriorCarryState

Folds two PriorCarryState fitted on disjoint blocks into the state of the concatenated block.

The buffer's merge, which is concatenation, and the union of the two named-asset sets: an asset either half has already told the caller about does not need telling again.

Arguments

a: The state of the first block of observations.
b: The state of the second block of observations.

Returns

state::PriorCarryState: The state the two blocks give when they are folded as one block.

Related

source

PortfolioOptimisers.prior — Method

prior(pe::AbstractPriorEstimator; kwargs...)

Reads a prior out of the state its estimator carries, with no data matrix.

The read-out arm of prior, and the entry point of every prior that has taken the online step. It reads the state and dispatches on its type, which is the whole of the routing:

A SampleBufferState is a refit. The estimator has no recursion of its own, so its estimate is the batch verb over the observations the buffer kept — every one folded so far when the buffer is uncapped, and the last max_history of them when Online capped it. The batch verb is given the factor rows the buffer recorded, through factor_buffer, and nothing where it recorded none, which is the arity the estimator tree fixed at the fold.
A PriorCarryState is a fold and a carry, and the families that take it write read-outs of their own.

An estimator that has folded nothing meets partial_fit_cache's refusal, which names the verb that fills the field.

Arguments

pe: Prior estimator carrying a state.
kwargs...: Additional keyword arguments, forwarded to the batch verb.

Validation

pe.cache is not nothing. An ArgumentError is thrown otherwise.

Returns

pr::AbstractPriorResult: The prior the estimator's state answers.

Related

source

PortfolioOptimisers.partial_fit! — Function

partial_fit!(
    pe::AbstractPriorEstimator,
    X::Union{AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}};
    ...
) -> HighOrderPriorEstimator
partial_fit!(
    pe::AbstractPriorEstimator,
    X::Union{AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}},
    F::Union{Nothing, AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}};
    dims,
    active_mask,
    estimation_mask
) -> HighOrderPriorEstimator

Folds observations, and the factor observations beside them, into the sample buffer a prior carries.

The refit route of the prior family, and the arm every prior carrying a SampleBufferState reaches: the buffering arm of partial_fit! with the arity of the prior's own batch verb, prior(pe, X, F). What the fold does with F is decided by the estimator tree through needs_factor_returns, and it mirrors what the batch verb does with the same argument:

true: the tree holds a factor leaf, so a fold given no F is refused by name with the door's own refusal, before any row is appended, and one given F records it.
false: the tree never reads F, so it is dropped, as prior(EmpiricalPrior(), X, F) drops it, and the buffer records rows alone.
nothing: the tree does not say, so the buffer records F when it is given and not otherwise, which is what the batch verb of an optional-argument prior does.

The buffer then fixes what it records at its first append and refuses a mixture, so a run that gives F at one step and not at the next is refused there by name.

Algorithm

Answer needs_factor_returns for the tree, refuse a missing F where the answer is true with assert_factor_returns, and drop F where it is false, through fold_factor_argument.
Read the buffer out of the cache field with assert_sample_buffer, which refuses a prior that was never wrapped in Online.
Fold a matrix, its factor block and its masks through the block arm of partial_fit!, and a vector, its factor observation and its masks through the single-observation arm.
Rebuild the prior with its cache rebound, and return it.

Arguments

pe: The prior whose buffer is folded forward.
X: Observations to fold. A matrix holds one observation per row when dims == 1, and one per column when dims == 2. A vector is a single observation across the assets, and dims is ignored.
F: The factor observations, oriented as X is, or nothing.
dims: Dimension along which to perform the computation.
active_mask: The active mask of the block, of the shape of X, or of one entry per asset when X is one observation, or nothing.
estimation_mask: The estimation mask, on the same terms as active_mask.

Validation

F is not nothing when needs_factor_returns(pe) === true. An IsNothingError is thrown otherwise.
pe carries a SampleBufferState. An ArgumentError is thrown otherwise.
Everything the buffer's own fold refuses.

Returns

pe: The prior, with its cache field rebound to the buffer after the last observation.

Related

source

PortfolioOptimisers.partial_fit! — Function

partial_fit!(pe::EmpiricalPrior{<:Any, <:Any, Nothing, <:Any, <:Any,
                                <:Option{<:PriorCarryState}}, X, F = nothing; kwargs...)
partial_fit!(pe::EmpiricalPrior{<:Any, <:Any, <:Number, <:Any, <:Any,
                                <:Option{<:PriorCarryState}}, X, F = nothing; kwargs...)

Folds observations into an EmpiricalPrior, which folds its moments and carries its rows.

pe.me and pe.ce fold exactly, so the prior's own step is theirs plus an append: it is quadratic in the number of assets and independent of the number of observations folded, where a refit from a buffer is linear in that number as well. That difference is the whole point of the seam at this layer, and it is why prior(pe) = prior(pe, sample_buffer(pe)) is the shape to refuse in review — it passes every parity test and throws both exact folds away.

The two arms differ in one line. The no-horizon arm folds the observation as it stands. The horizon arm folds log1p of it, because the horizon method fits its moments in log space, and carries the observation itself, because the LowOrderPrior it returns carries the arithmetic returns the caller handed in. Buffering the log rows would silently change the matrix every scenario measure reads.

A member that does not fold — a SemiMoment covariance, a composite whose mp.alg reads the sample — is left alone by fold_member and refitted over the carried rows at the read-out. So a caller writes the estimator they would write in batch, and no member carries a second copy of the sample.

A factor observation is taken and dropped, because prior(EmpiricalPrior(), X, F) declares F and never reads it: the fold mirrors the batch verb's arity so that a host forwarding F down its tree meets no MethodError here, and needs_factor_returns answers false for this estimator so that nothing above it keeps a factor row on its behalf.

Algorithm

Fold pe.me and pe.ce through fold_member, on the observation under the horizon arm's transform.
Append the arithmetic observation to the carry state through fold_carry.

Arguments

pe: Empirical prior estimator.
X: Observations to fold. A matrix holds one observation per row when dims == 1 and one per column when dims == 2; a vector is one observation across the assets.
F: The factor observations beside X, or nothing. Dropped, as the batch verb drops them.
dims: Dimension along which to perform the computation.
kwargs...: Additional keyword arguments, forwarded to the two arms.

Returns

pe: The estimator, with its arms folded and its cache field rebound.

Related

source

PortfolioOptimisers.prior — Method

prior(pe::EmpiricalPrior{<:Any, <:Any, Nothing, <:Any, <:Any,
                         <:Option{<:PriorCarryState}}; strict::Bool = false, kwargs...)
prior(pe::EmpiricalPrior{<:Any, <:Any, <:Number, <:Any, <:Any,
                         <:Option{<:PriorCarryState}}; strict::Bool = false, kwargs...)

Reads an EmpiricalPrior out of its fold, with no data matrix.

mu comes off pe.me, sigma comes off pe.ce, and X is the matrix the carry state already holds; the rows are read, never refitted. The horizon arm then takes the same four steps of the batch method — scale by pe.horizon, exponentiate, build the covariance on the $\hat{\mu}_i + 1$ factors, subtract the one — through horizon_moments!, which is the one place that algebra lives.

pe.max_scenarios cuts the rows the Result carries, exactly as it does in batch, and the fill runs over the window that cut leaves. When the cap cuts, the Result states the number of observations folded in ens, through scenario_ens, so a count reader prices t and not the w rows carried, exactly as in batch. The fill's notice is narrowed by the carry state's named-asset set, so a walk-forward names an asset at the step it lists and not at every step afterwards.

No AssetPanel is read. A panel is fold context rather than sample, and the Coverage Universe of this read-out agrees with a batch fit's by construction, because coverage_mask is a pure function of the rows and the carry state holds exactly the rows a batch fit would have seen.

Algorithm

Read the carry state, which refuses an estimator that has folded nothing.
Resolve the fill limit against the arms' coverage floor, as the batch method does.
Read mu and sigma through read_member, which refits a member that did not fold over the carried rows.
Under the horizon arm, apply horizon_moments!.
Cut the carried rows to pe.max_scenarios with scenario_window, and materialise them, because the buffer's own storage moves under the next fold.
Fill and return a LowOrderPrior, with the ens of scenario_ens read off the rows before the cut.

Arguments

pe: Empirical prior estimator carrying a PriorCarryState.
strict: Whether a zero-filled scenario raises rather than warns.
kwargs...: Additional keyword arguments, forwarded to the arms.

Validation

pe.cache is not nothing. An ArgumentError is thrown otherwise.

Returns

pr::LowOrderPrior: Result object containing asset returns, mean vector, and covariance matrix.

Related

source

PortfolioOptimisers.partial_fit! — Function

partial_fit!(pe::HighOrderPriorEstimator, X, F = nothing; kwargs...)

Folds observations into a HighOrderPriorEstimator, by forwarding them to its members.

The host owns no buffer. It builds HighOrderPrior(; pr = pr, ...) around whatever its embedded prior returned, so the observations it needs at read-out are the ones that prior already carries, and a buffer here would be a second copy of the sample. The fold is therefore a forward and nothing else.

pe.pe is forwarded unconditionally, because it is the member that carries the rows: an embedded prior that cannot fold refuses here, naming the wrapper that would give it somewhere to buffer, and that refusal is right — the host has no rows of its own to offer it. The factor observations travel with it, as the batch verb hands F to the embedded prior, and the embedded prior's tree decides what becomes of them. pe.kte and pe.ske go through fold_member instead, because the host does have rows for them by then, and a co-moment that cannot fold is refitted over pr.X at the read-out. So HighOrderPriorEstimator(; ske = Coskewness(; alg = SemiMoment())) needs no wrapper at the call site and keeps one copy of the sample.

Arguments

pe: High order prior estimator.
X: Observations to fold, a matrix of rows or a single observation.
F: The factor observations beside X, or nothing. Forwarded to the embedded prior.
kwargs...: Additional keyword arguments, forwarded to the members.

Returns

pe: The estimator, with its members folded.

Related

source

PortfolioOptimisers.prior — Method

prior(pe::HighOrderPriorEstimator; kwargs...)

Reads a HighOrderPriorEstimator out of its fold, with no data matrix.

The embedded prior answers first, and the co-moments are read off their own folds where they folded and refitted over pr.X where they did not. assemble_high_order_prior then does exactly what it does in batch, so the two routes cannot drift.

A co-moment refitted here reads pr.X, which is the matrix the embedded prior carries after its own max_scenarios. So a Scenario Cap on the embedded prior also caps the rows a non-folding co-moment is fitted over, while a folding one is untouched, because it folded every observation as it arrived. A caller who wants the two windows to agree either leaves the cap unset or gives the co-moments a FullMoment algorithm.

Arguments

pe: High order prior estimator whose members carry states.
kwargs...: Additional keyword arguments, forwarded to the members.

Returns

hop::HighOrderPrior: Result object carrying the low order result and the co-moments.

Related

source

PortfolioOptimisers.partial_fit! — Function

partial_fit!(pe::BlackLittermanPrior, X, F = nothing; kwargs...)

Folds observations into a BlackLittermanPrior, by forwarding them to its embedded prior.

The host owns no buffer, for the reason HighOrderPriorEstimator owns none: it returns forward_prior of the result its embedded prior answered, so the rows it needs are already carried one level down. The factor observations are forwarded with the rows, as the batch verb hands F down. The views are configuration and fold nothing.

Arguments

pe: Black-Litterman prior estimator.
X: Observations to fold, a matrix of rows or a single observation.
F: The factor observations beside X, or nothing. Forwarded to the embedded prior.
kwargs...: Additional keyword arguments, forwarded to the embedded prior.

Returns

pe: The estimator, with its embedded prior folded.

Related

source

PortfolioOptimisers.prior — Method

prior(pe::BlackLittermanPrior; strict::Bool = false, kwargs...)

Reads a BlackLittermanPrior out of its fold, with no data matrix.

The embedded prior answers first, and bl_posterior is the batch body unchanged: every number it reads — the number of observations behind tau, the asset axis, the matrix the processing runs over — comes off the result rather than off a returns matrix, so the fold and the batch fit reach one body.

Arguments

pe: Black-Litterman prior estimator whose embedded prior carries a state.
strict: Whether an unresolved view raises rather than warns.
kwargs...: Additional keyword arguments, forwarded to the embedded prior and the processing.

Returns

pr::AbstractPriorResult: The embedded result with its two moments replaced by the posteriors.

Related

source