GitHub

The optimiser on the partial-fit seam: private API

PortfolioOptimisers.returns_buffer — Function
returns_buffer(state::SampleBufferState)
returns_buffer(state::PriorCarryState)
returns_buffer(state::AbstractPartialFitState)

Reads the buffer of asset returns out of the state a prior carries.

Two states carry rows at the prior layer, and each keeps them in a different place: a SampleBufferState is the rows, and a PriorCarryState holds them in buf. The optimiser's read-out reads the returns through this verb rather than through the state's fields, so a third state with rows of its own adds one method here and nothing else. The buffer returned carries the factor rows too, where the prior's tree reads them, so the read-out takes F off it through factor_buffer when the fold context holds no factor column of its own. A state that carries no rows — an exact-fold state of the moment layer, which a prior never holds — is refused by name.

Arguments

  • state: The state the prior carries.

Validation

  • state carries rows. An ArgumentError is thrown otherwise.

Returns

  • buffer::SampleBufferState: The rows, the masks folded beside them, and the factor rows where the prior records them.

Related

source
PortfolioOptimisers.prior_returns_buffer — Function
prior_returns_buffer(pe::AbstractPriorEstimator)
prior_returns_buffer(pe::Union{<:HighOrderPriorEstimator, <:BlackLittermanPrior})

Reads the buffer of asset returns out of the prior an optimiser forwarded its observations to.

The rows are owned once, at the bottom of the chain, and the two priors that wrap another prior own none of their own — HighOrderPriorEstimator and BlackLittermanPrior build their result around the embedded prior's, so this walks down to it. Every other prior carries its rows in its own cache, and partial_fit_cache refuses one that has folded nothing.

Arguments

  • pe: The prior estimator the optimiser forwards to.

Validation

  • The prior that owns the rows carries a state. An ArgumentError is thrown otherwise.

Returns

  • buffer::SampleBufferState: The rows the optimiser has folded, and the masks beside them.

Related

source
PortfolioOptimisers.step_active_mask — Function
step_active_mask(rd::ReturnsResult) -> Any

Reads the active mask an observation's Asset Panel carries into the prior's step, and refuses what the step cannot carry.

Three shapes reach here. No panel, or a static one, contributes no mask: the panel is context, pinned by the fold context. A time-varying panel of masks alone — the ingestion layer's shape — hands its active mask over, and the mask rides into the prior's buffer beside the rows. A time-varying panel that carries Panel Fields is refused: a time-varying field's rows are sample, and the buffers of the seam hold numbers alone, so the step has nowhere to keep a categorical or a tensor field and cannot rebuild the panel a read-out needs.

The estimation mask does not travel the step. The exact folds of the moment layer take an active mask and no estimation mask, and only the two regime-adjusted families read one, so a step whose estimation universe is narrower than its active universe would fold a plain member with the wrong universe in silence. It is refused instead, by name, where the two differ.

Arguments

  • rd: The carrier of the observations being folded.

Validation

  • rd.iv is nothing. An ArgumentError is thrown otherwise: the prior's step takes no implied-volatility surface.
  • A time-varying rd.pnl carries no Panel Field. An ArgumentError is thrown otherwise.
  • A time-varying rd.pnl has its estimation mask equal to its active mask. An ArgumentError is thrown otherwise.

Returns

  • amsk::Option{<:AbstractMatrix{<:Bool}}: The active mask of the observations, or nothing.

Related

source
PortfolioOptimisers.fold_prior — Function
fold_prior(pe::AbstractPriorEstimator, rd::ReturnsResult)
fold_prior(pe::AbstractPriorResult, rd::ReturnsResult)
fold_prior(pe::TimeDependent, rd::ReturnsResult)

Forwards the observations of a carrier to the prior, in the prior's own arity.

The one forward the optimiser's step makes, and the rule for every optimiser: an optimiser forwards the observation to its prior and to nothing else, and everything it holds beside the prior takes its ordinary batch fit at read-out, from the reconstituted fold context. The carrier is unpacked on the way down exactly as prior unpacks it in batch — rd.X and rd.F verbatim, with a missing rd.F refused by name at this door when the prior's tree requires one, through needs_factor_returns — and the active mask of a time-varying panel rides as the keyword the prior's step takes. What the prior does with F is the prior's own decision: its tree records it, drops it, or takes what it is given, exactly as its batch verb does.

Two refusals. A prior that is already a AbstractPriorResult has no state to fold into: it is batch configuration, and an optimiser holding one runs optimise(opt, rd). A TimeDependent on the prior is refused because a schedule swaps the estimator that carries the state, and a member that never saw the folded rows cannot be handed them. No loop resolves a schedule before stepping: a schedule reaches stateless fields only, and the fold loop's online arm refuses one on the prior at warm-up (assert_online_entry).

Arguments

  • pe: The prior the optimiser holds.
  • rd: The carrier of the observations to fold, observations × assets.

Validation

  • rd.X is not nothing. An IsNothingError is thrown otherwise.
  • rd.F is not nothing when needs_factor_returns(pe) === true. An IsNothingError is thrown otherwise.
  • Everything step_active_mask refuses.

Returns

  • pe: The prior, with the observations folded into its state.

Related

source
PortfolioOptimisers.fold_context — Function
fold_context(
    cache::Union{Nothing, ReturnsBufferState},
    rd::ReturnsResult,
    max_history::Union{Nothing, Integer},
    own_returns::Bool,
    own_factors::Bool
) -> ReturnsBufferState{__T_nx, __T_X, __T_nf, __T_F, __T_nb, __T_B, __T_ts, __T_pnl, __T_max_history} where {__T_nx<:Union{Nothing, AbstractVector{<:AbstractString}}, __T_X<:Union{Nothing, SampleBufferState}, __T_nf<:Union{Nothing, AbstractVector{<:AbstractString}}, __T_F<:Union{Nothing, SampleBufferState}, __T_nb<:Union{Nothing, AbstractVector{<:AbstractString}}, __T_B<:Union{Nothing, SampleBufferState, AbstractVector}, __T_ts<:Union{Nothing, AbstractVector}, __T_pnl<:Union{Nothing, AssetPanel}, __T_max_history<:Union{Nothing, Integer}}

Folds the observations of a carrier into the fold context a host keeps, seeding it on the first step.

Arguments

  • cache: The context the host carries, or nothing before the first step.
  • rd: The carrier of the observations.
  • max_history: The cap the context takes when it is seeded, which is the cap of the buffer that holds the returns so the two stay in step.
  • own_returns: Whether the context keeps the returns itself, which is true for a head that holds no prior.
  • own_factors: Whether the context keeps the factor column itself, which is true for a head that holds no prior and for a host whose prior's tree never reads it.

Returns

  • state::ReturnsBufferState: The context after the observations.

Related

source
PortfolioOptimisers.fold_returns — Function
fold_returns(host, rd::ReturnsResult) -> Any

The step of a host that holds a prior and a fold context: forward to the prior, then record the context.

The prior is folded first, because the context takes its cap from the buffer the prior seeds — a wrapper's max_history windows the prior's fit, and the context must drop the same rows at the same step so the read-out's carrier lines up with the prior's rows.

F is owned once. The context keeps the factor column only when the prior's tree never reads it — when needs_factor_returns answers false — because otherwise the prior's own buffer records it and the read-out reads it back through prior_returns_buffer, as it reads the returns. A tree that requires it, or one that takes what it is given, therefore holds F in one place.

Arguments

  • host: An estimator holding pe and cache.
  • rd: The carrier of the observations, observations × assets.

Returns

  • host: The host with its prior folded and its context recorded.

Related

source
PortfolioOptimisers.online_state_seed — Method
online_state_seed(opt::Union{<:EqualWeighted, <:RandomWeighted}, max_history)
online_state_seed(opt::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling}, max_history)

Seeds the fold context an Online declares on a prior-less head, and refuses the wrapper on a host that holds a prior.

Online(EqualWeighted(); max_history = w) windows the rows the head keeps, so its seed is a ReturnsBufferState carrying the cap. A host that holds a prior takes its window from the prior — Online wraps the prior, and the context follows the prior's buffer — so wrapping the host itself would declare a second cap over the same rows, and it is refused by name.

Arguments

  • opt: The estimator the wrapper wraps.
  • max_history: The wrapper's cap.

Returns

  • state::ReturnsBufferState: The empty context to seed.

Related

source
PortfolioOptimisers.update_online_member — Function
update_online_member(pe::AbstractPriorEstimator)
update_online_member(pe::Online)
update_online_member(pe)

Resolves the Online declarations under a host's pe slot, and passes anything else through.

The slot admits a prior estimator, a prior result and a schedule. A wrapper is resolved; a prior estimator is scanned for wrappers in its own fields; a result and a schedule hold no wrapper to resolve and are returned as they are — the step refuses them by name later, where the reason is stated.

Related

source
PortfolioOptimisers.update_online_estimator — Method
update_online_estimator(opt::JuMPOptimisationEstimator)
update_online_estimator(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})
update_online_estimator(opt::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling})

Resolves the Online declarations an optimiser carries in its prior, at warm-up.

An optimiser hands its prior across a boundary of its own, so it writes the recursion the generic method does not: the JuMP and hierarchical heads recurse into the bundle they hold, and the bundle, the meta-optimisers and InverseVolatility recurse into pe. The forward reaches the prior alone, as the step does — a wrapper inside an inner optimiser of a meta-optimiser, or inside a fallback, is a batch configuration of that optimiser and is not resolved here.

Arguments

  • opt: The optimiser.

Returns

  • opt: The optimiser, with every wrapper under its prior resolved to an estimator carrying a seeded buffer.

Related

source
PortfolioOptimisers.assert_stateless_schedule — Function
assert_stateless_schedule(opt::JuMPOptimisationEstimator)
assert_stateless_schedule(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})
assert_stateless_schedule(opt::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling})
assert_stateless_schedule(opt)

Refuse a TimeDependent schedule on a field that carries a state, by name.

A schedule replaces its field's value every fold, and a state is threaded through that value, so the two write one slot with opposite intentions and the schedule wins: the value it hands a fold never saw the rows folded before it. The one such field a schedule can reach is a host's pe, whose bound admits one; a JuMP or hierarchical head's opt holds the bundle and its bound refuses a schedule at construction, so the heads only recurse. The walk is the one update_online_estimator makes, and a schedule on any other field — a meta-optimiser's inner optimisers included, which the read-out refits from the buffer — composes with no rule, because it writes the per-fold copy while the state threads through the unresolved estimator. Carrying a state across a swap is a possible future extension and is not built.

Related

source
PortfolioOptimisers.assert_online_entry — Function
assert_online_entry(p::Pipeline)
assert_online_entry(o::Online{<:Pipeline})

Refuses a Pipeline that cannot take the online step, at the entry of the fold loop's online arm and before any fit, by name.

The walk goes over the steps of the route. On the host route, six refusals. A state anywhere — the Pipeline's own cache or a step's — because the loop starts cold. No row owner: a pipeline with neither a prior nor an optimisation step has nothing to fold into. A TimeDependent schedule as the row owner, which is the case only when no prior step precedes the optimisation step: a schedule swaps the estimator that carries the state; with a prior step before it, the schedule swaps a stateless step and composes. An optimisation step that owns the rows is held to assert_online_entry's own refusals. And every data step before the owner must fold or defer: a window-valued configuration (PriceGapFill with a statistic fill, MissingDataFilter with row_thr < 1), a callable PipelineStep writing :prices or :returns, a nested Pipeline, and a caller's preprocessing estimator with no online form are refused, and the message names both routes: give the step a partial_fit_transform, or declare a refit with Online(pipe). And a capped owner — an Online on the owner's path carrying max_history, read through step_online_cap — is refused when a row-local step folds before it: the owner's window is counted in its own rows, and the step's carry reaches across the window's front edge, so the run equals no batch scheme. The rolling window through a Pipeline is Online(pipe; max_history = w). A capped owner behind universe-only steps alone, or on returns input, stays on the host route, because the read-out refits a universe-only step over the owner's capped rows.

On the refit route, two: a state anywhere, as above, and an Online member below the Online(pipe), because no member folds under a refit and its cap would be silently ignored.

Validation

  • Everything above. An ArgumentError naming the step is thrown otherwise.

Related

source
PortfolioOptimisers.returns_result — Method
returns_result(host::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling})
returns_result(host::Union{<:EqualWeighted, <:RandomWeighted})
returns_result(opt::JuMPOptimisationEstimator)
returns_result(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})

Rebuilds the ReturnsResult of the observations an optimiser has folded.

The reconstitution verb of the read-out, on the optimiser: the rows come from the prior's buffer — or from the head's own, where it holds no prior — and every other column and the pinned context come from the ReturnsBufferState the host keeps. The factor column comes from whichever of the two owns it: the context where the prior's tree never reads it, and the prior's buffer otherwise. The result is a fresh carrier, equal field by field to the one a batch fit over the same observations would have read.

Arguments

  • host: The optimiser, or the bundle it holds.

Validation

  • The host carries a fold context. An ArgumentError is thrown otherwise.

Returns

  • rd::ReturnsResult: The carrier of the observations folded so far.

Related

source
PortfolioOptimisers.held_timestamps — Function
held_timestamps(host::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling, <:EqualWeighted, <:RandomWeighted})
held_timestamps(opt::JuMPOptimisationEstimator)
held_timestamps(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})
held_timestamps(::PreviousWeights)

The timestamps an optimiser's Fold Context holds, or nothing when it holds none.

The accessor Resume's alignment check reads. The ReturnsBufferState keeps the timestamps of the observations folded, trimmed from the front under a cap, so under Online(pe; max_history = w) the last w survive. The arms mirror returns_result's: a head forwards to the bundle it holds, and a host reads its own context. PreviousWeights reads nothing and keeps no context, so it holds no timestamps and a resume refuses it. A Pipeline host answers through the state its row owner keeps.

Arguments

  • opt: The stepped optimiser.

Returns

  • ts::Option{<:AbstractVector}: The held timestamps, in order, or nothing.

Related

source
held_timestamps(p::Pipeline)

The timestamps a stepped Pipeline holds, through the state its row owner keeps, or nothing.

Three arms, by the state the pipeline carries. Under Online(pipe) the PipelineBufferState holds the input carrier itself, and its timestamps are the answer. A prior owner leaves the Pipeline its own ReturnsBufferState, which holds them. An optimisation owner keeps its own Fold Context, and the pipeline holds none, so the owner answers (held_timestamps). At the price level a PricesToReturns step drops the first row, and the returns' timestamps are the prices' from the second row on, so the held span still equals its rows of the price carrier.

Related

source
PortfolioOptimisers.readout_without_state — Function
readout_without_state(host, pe::AbstractPriorResult)
readout_without_state(host, pe)

The fold-less entry of a host that has taken no step.

A host whose prior is already a result needs no returns to solve, and is handed back with an empty carrier: that is the batch entry optimise(opt) has always had. Any other host has nothing to answer from — no state and no prior result — and is refused by name, pointing at the step that fills the state and at the batch verb that takes the returns.

Related

source
PortfolioOptimisers.online_readout — Function
online_readout(host::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling})
online_readout(host::Union{<:EqualWeighted, <:RandomWeighted})
online_readout(opt::JuMPOptimisationEstimator)
online_readout(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})
online_readout(opt::FiniteAllocationOptimisationEstimator)
online_readout(opt::OptimisationEstimator)

Turns a folded optimiser into the batch call that reads it out: an estimator and a carrier.

The read-out reconstitutes the fold context and calls the ordinary batch path, so nothing above the prior needs a method of its own. It rebuilds the carrier with returns_result, swaps the folded prior for its read-out — a prior result, which the batch path does not refit, so the prior's fold is what the solve reads and the step stays quadratic in the assets — and drops the context, because the estimator it hands back is a batch configuration over that carrier. The state is never written: the read-out is a pure function of it, callable any number of times for the same answer, which is what lets the fallback chain walk unchanged.

The path is chosen by dispatch on the host's cache. A host carrying nothing has taken no step, and readout_without_state decides what that means: an optimiser whose prior is already a result is handed back with an empty carrier, which is the batch entry it has always had, and any other is refused by name. A finite allocation is refused by name, as its step is.

Arguments

  • opt: The optimiser to read out.

Returns

  • (opt, rd)::Tuple: The batch estimator and the carrier to run it over.

Related

source
PortfolioOptimisers.show_fields — Method
show_fields(
    opt::Union{EqualWeighted, HierarchicalOptimiser, InverseVolatility, JuMPOptimiser, NestedClustered, RandomWeighted, Stacking, SubsetResampling}
) -> Tuple

Renders every field of a host of the online step except its cache.

The state a cache holds is the running detail of an incremental fit, not the configuration a reader looks the type up for, and it prints under the optimiser at every site that renders one. Set set_show_nothing_fields! for the type to render it.

Arguments

  • opt: The host.

Returns

  • fields::Tuple: Every field name but cache.

Related

source