Online optimisation: private API

PortfolioOptimisers.step_active_mask — Function
step_active_mask(rd::ReturnsResult) -> Any

Reads the active mask of the Asset Panel that a step carries to an exact fold that is not a prior, and refuses what that fold cannot carry.

The step of a covariance estimator in covariance_forecast_evaluation and the step of an online portfolio selection read the mask through this function. Each caller decides on the implied volatilities itself: the head of an online portfolio selection records them with its rows, and the covariance step gives them to a buffer and nothing to an exact fold. A prior does not: it folds through partial_fit! over the ReturnsResult, whose route decides what the step honours. A ReturnsResult with no panel, or with a static one, gives no mask, because the fold context pins a static panel. A time-varying panel that holds masks alone, which is the shape the ingestion layer writes, gives its active mask, and the fold records the mask beside the rows. This function refuses a time-varying panel that holds Panel Fields, because these folds read none and hold numbers alone.

The estimation mask does not travel these folds. The exact folds of the moment layer take an active mask and no estimation mask, so a step whose estimation universe is narrower than its active universe would fit the wrong universe without a word. The function refuses such a panel with an ArgumentError.

Algorithm

  1. Refuse rd without returns with assert_prior_fold_returns.
  2. Return nothing when rd holds no panel, or when its panel is static.
  3. Refuse a time-varying panel that holds a Panel Field.
  4. Refuse a time-varying panel whose estimation mask differs from its active mask.
  5. Return the active mask pnl.amsk.

Arguments

  • rd: The returns data to fold.

Validation

  • The rules of assert_prior_fold_returns.
  • A time-varying rd.pnl holds no Panel Field. An ArgumentError is thrown otherwise.
  • A time-varying rd.pnl has an estimation mask equal to its active mask. An ArgumentError is thrown otherwise.

Returns

  • amsk::Option{<:AbstractMatrix{<:Bool}}: The active mask of the observations, or nothing.

Related

source
PortfolioOptimisers.prior_step_returns — Function
prior_step_returns(rd::ReturnsResult, x_src::Nothing)
prior_step_returns(rd::ReturnsResult, x_src::DataSource)
prior_step_returns(rd::ReturnsResult, x_src::AbstractReturnsSource)

Returns the observations that the prior of an optimiser folds in its online step.

The step of a prior reads a ReturnsResult, and the batch fit of the prior reads the matrix that its source picks. Under ExcessSource that is the returns in excess of the benchmark. So the function gives the prior a ReturnsResult that holds X - B in its X and every other field of rd unchanged. The prior is the one reader of this ReturnsResult in an optimiser. The Fold Context records rd itself, with its raw X, so optimise(opt) with no returns rebuilds the raw returns data.

covariance_forecast_evaluation reads this ReturnsResult too. Every reader of an evaluation reads one frame, the fit, the online step and the test rows, so the whole fold loop of the evaluation runs on it.

Algorithm

The method that Julia selects is the algorithm.

  1. nothing or DataSource: return rd.
  2. Any other source: read its matrix with benchmark_relative_returns, giving X, and return a ReturnsResult with X in place of rd.X.

Arguments

  • rd: The returns result to use.
  • x_src: The source of the returns that the fit reads, nothing or an AbstractReturnsSource. Only ExcessSource reads the benchmark: it subtracts B from the asset returns. An optimiser passes the source of its group, see group_source.

Validation

Returns

  • rd::ReturnsResult: The observations that the prior folds.

Related

source
PortfolioOptimisers.fold_prior — Function
fold_prior(pe::AbstractPriorEstimator, rd::ReturnsResult,
           gsrc::Option{<:AbstractReturnsSource} = returns_source(pe))
fold_prior(pe::AbstractPriorResult, rd::ReturnsResult,
           gsrc::Option{<:AbstractReturnsSource} = nothing)
fold_prior(pe::TimeDependent, rd::ReturnsResult,
           gsrc::Option{<:AbstractReturnsSource} = nothing)

Forwards the observations of rd to the prior, in the arity of the prior's own step.

This is the one forward that the step of an optimiser makes. An optimiser forwards each observation to its prior and to nothing else. optimise(opt) with no returns fits every other member in batch, from the fold context it rebuilds. The step folds rd through fold_prior_step, the verb of partial_fit! over the ReturnsResult, which reads it as prior reads it in batch. The route of the prior decides what the step honours. A refit honours the estimation mask and records the Panel Fields of a prior that reads them. The carry of EmpiricalPrior refuses an estimation mask that differs from the active mask, and each route refuses a Panel Field that its prior does not read. The tree of the prior decides what to do with F, as its batch function does.

A prior that is already an AbstractPriorResult has no state to fold into. It is batch configuration, and an optimiser that holds one runs optimise(opt, rd). This function also refuses a TimeDependent schedule on the prior. A schedule swaps the estimator that carries the state, and a member that never saw the folded rows cannot take them over.

The prior folds the returns that its batch fit reads, in the frame that its own x_src states. Under ExcessSource these are the returns in excess of the benchmark, as prior reads them, so prior_step_returns gives the prior the observations of rd with X - B in place of X.

A batch fit drops an observation with a gap in a Shared Series that it reads, see shared_series_view. The step cannot drop the row, because the buffer of the prior and the Fold Context hold one row each for every observation, and optimise(opt) with no returns rebuilds the returns data from both. So the step marks the observation, and the prior keeps the mark in its buffer: every fit through the buffer drops the observation, and a carry fold leaves it out of its running moments. The prior cannot find a gap of the benchmark, because in the Excess Frame it reads X - B and not B, so this function finds the marks and hands them to the prior. A gap of a vector benchmark follows the group of the head, gsrc, and not the prior alone: a head drops it when any reader of its group states ExcessSource(), so a raw prior beside an excess reader marks it too. optimise(opt) with no returns cuts the rebuilt returns data by the same rule, see batch_from_state.

Algorithm

  1. Mark the observations with a Shared Series gap with shared_series_gaps: a gap of a vector benchmark under the source of the group gsrc, and a gap of the factor returns when reads_factor_returns answers true for pe. Mark a gap of a matrix benchmark in the estimation mask under the source of pe with benchmark_estimation_mask, as the batch fit of the prior marks it.
  2. Take the observations that the prior folds with prior_step_returns under the source of pe, giving rdp.
  3. Fold rdp and the marks into pe with fold_prior_step, which refuses by route what the prior cannot honour.

Arguments

  • pe: The prior that the optimiser holds.
  • rd: The returns data to fold, observations × assets.
  • gsrc: The source of the group of the head, see group_source. It decides the marks of a gap of a vector benchmark. The default is the source of pe, for a prior that no head holds. The prior step of a Pipeline passes the source of the group of the optimiser that receives it, see fold_pipeline_owner.

Validation

Returns

  • pe: The prior, with the observations folded into its state.

Related

source
PortfolioOptimisers.fold_context — Function
fold_context(
    cache::Union{Nothing, ReturnsBufferState},
    rd::ReturnsResult,
    max_history::Union{Nothing, Integer},
    own_returns::Bool,
    own_factors::Bool,
    own_exogenous::Bool
) -> ReturnsBufferState

Folds the observations of rd into the fold context of an optimiser, and seeds the context on the first step.

Algorithm

  1. Take cache as the context state. When cache is nothing, seed an empty ReturnsBufferState with the cap max_history instead.
  2. Fold rd into state with partial_fit!, under own_returns, own_factors and own_exogenous.

Arguments

  • cache: The context that the optimiser holds, or nothing before the first step.
  • rd: The returns data to fold.
  • max_history: The cap of a new context. It is the cap of the buffer that holds the returns, so the context and that buffer drop the same rows.
  • own_returns: Whether the context keeps the returns itself. It is true for a head that holds no prior.
  • own_factors: Whether the context keeps the factor column itself. It is true for a head that holds no prior, and for an optimiser whose prior never reads factor returns.
  • own_exogenous: Whether the context keeps the Exogenous Series itself. It is true for a head that holds no prior, and for an optimiser whose prior never reads the series.

Validation

Returns

  • state::ReturnsBufferState: The context after the observations.

Related

source
PortfolioOptimisers.fold_returns — Function
fold_returns(opt, rd::ReturnsResult) -> Any
fold_returns(
    opt,
    rd::ReturnsResult,
    gsrc::Union{Nothing, AbstractReturnsSource}
) -> Any

Takes the step of an optimiser that holds a prior and a fold context. It forwards the observations to the prior, then records rd in the context.

The prior folds first, because the context takes its cap from the buffer of the prior. The max_history of an Online wrapper caps the rows of the prior, and the context must drop the same rows at the same step, so the ReturnsResult that optimise(opt) with no returns rebuilds matches the rows of the prior.

The prior folds the returns in the frame that its own x_src states. Under ExcessSource the buffer of the prior holds the returns in excess of the benchmark, and the context keeps the raw returns beside them. Under nothing or DataSource the buffer holds the raw returns, and the context keeps no copy. So the ReturnsResult that optimise(opt) with no returns rebuilds holds the raw X and B, as the batch path reads them, and optimise(opt) with no returns equals optimise(opt, rd[1:t]) in every frame. The marks of a gap of a vector benchmark follow the group of the head, gsrc.

One place holds F. The context keeps the factor column only when the tree of the prior never reads it, that is, when needs_factor_returns answers false. Otherwise the buffer of the prior records it, and optimise(opt) with no returns reads it back through prior_returns_buffer, as it reads the returns. One place holds the Exogenous Series on the same terms. The context keeps it only when reads_exogenous_series answers false, and the buffer of the prior records it otherwise.

Algorithm

  1. Fold rd into opt.pe with fold_prior under the source of the group gsrc, giving the folded prior pe.
  2. Read the buffer rows of pe with prior_returns_buffer.
  3. Set own_factors to true when needs_factor_returns(pe) === false, and own_exogenous to true when reads_exogenous_series(pe) is false.
  4. Fold rd into opt.cache with fold_context, under the cap rows.max_history, giving cache. The context keeps the raw returns when the prior states a source other than nothing and DataSource, because the buffer of the prior then holds another matrix. Otherwise it keeps no returns.
  5. Rebuild opt with pe and cache.

Arguments

  • opt: An estimator that holds pe and cache.
  • rd: The returns data to fold, observations × assets.
  • gsrc: The source of the group of the head that holds opt, see group_source. The default is the group of opt itself.

Validation

Returns

  • opt: The optimiser, with its prior folded and its context recorded.

Related

source
PortfolioOptimisers.fold_optimiser — Function
fold_optimiser(opt::Union{JuMPOptimisationEstimator, HierarchicalRiskParity,
                          HierarchicalEqualRiskContribution,
                          SchurComplementHierarchicalRiskParity},
               rd::ReturnsResult, gsrc::Option{<:AbstractReturnsSource})
fold_optimiser(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator,
                          InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator,
                          BaseSubsetResamplingOptimisationEstimator},
               rd::ReturnsResult, gsrc::Option{<:AbstractReturnsSource})
fold_optimiser(opt::OptimisationEstimator, rd::ReturnsResult,
               gsrc::Option{<:AbstractReturnsSource})

Takes the online step of an optimiser that holds a prior, with the marks of a gap of a vector benchmark under the group source gsrc.

This is the step of partial_fit!, which passes the source of the group of opt itself, see group_source. A Pipeline whose optimiser owns the rows passes the source of the group of the Pipeline instead, because the steps whose values the optimiser receives join that group, see pipeline_gap_sources. An optimiser that holds no prior marks no gap, so it takes its own step and ignores gsrc.

Algorithm

The method that Julia selects is the algorithm.

  1. A JuMP head or a hierarchical head refuses a ReturnsTracking in its tree, through assert_step_tracks_no_series. It folds the bundle it holds in opt through fold_returns under gsrc, because the bundle does not see the risk measures of the head. It rebuilds itself around the result.
  2. A configuration or a meta-optimiser refuses a ReturnsTracking in the same way, and folds through fold_returns under gsrc.
  3. Any other optimiser takes its step through partial_fit!.

Arguments

  • opt: The optimiser to fold into.
  • rd: The returns data to fold, observations × assets.
  • gsrc: The source that decides the marks of a gap of a vector benchmark, see group_source.

Validation

Returns

  • opt: The optimiser, with its prior folded and its context recorded.

Related

source
PortfolioOptimisers.online_state_seed — Method
online_state_seed(opt::Union{EqualWeighted, RandomWeighted, BestConstantRebalancedPortfolio}, max_history)
online_state_seed(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator}, max_history)

Seeds the fold context that an Online wrapper declares on a head with no prior, and refuses the wrapper on an optimiser that holds a prior.

Online(EqualWeighted(); max_history = w) caps the rows that the head keeps, so its seed is an empty ReturnsBufferState with the cap w. An optimiser that holds a prior takes its cap from the prior. Online wraps the prior, and the context follows the buffer of the prior, so a wrapper on the optimiser would declare a second cap over the same rows.

Arguments

  • opt: The estimator that the wrapper wraps.
  • max_history: The cap of the wrapper.

Validation

  • opt holds no prior. An ArgumentError is thrown otherwise, and it tells the caller to wrap the prior.

Returns

  • state::ReturnsBufferState: The empty context to seed.

Related

source
PortfolioOptimisers.update_online_estimator — Method
update_online_estimator(opt::JuMPOptimisationEstimator)
update_online_estimator(opt::Union{HierarchicalRiskParity, HierarchicalEqualRiskContribution, SchurComplementHierarchicalRiskParity})
update_online_estimator(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator})

Resolves the Online wrappers in the prior of an optimiser, at the warm-up.

The JuMP and hierarchical heads resolve the bundle they hold, and the bundle, the meta-optimisers and InverseVolatility resolve their pe. The slot takes a prior estimator, a prior result or a schedule. A prior result and a schedule hold no wrapper to resolve, so they pass unchanged, and fold_prior refuses them later at the step with the reason. The step forwards observations to that prior alone, so this function resolves no other wrapper. A wrapper in an inner optimiser of a meta-optimiser, in a fallback, or in any other field stays unresolved, and the entry of the online arm of the fold loop refuses it with an error that names the field.

Arguments

  • opt: The optimiser.

Validation

Returns

  • opt: The optimiser, with each wrapper in its prior resolved to an estimator that carries a seeded buffer.

Related

source
PortfolioOptimisers.online_unreached_path — Function
online_unreached_path(opt)

Names the first Online wrapper in an optimiser that the warm-up leaves unresolved, or answers nothing.

The warm-up of an optimiser resolves the wrappers in its prior alone, through update_online_estimator. A wrapper in any other field, such as an inner optimiser of a meta-optimiser or a fallback, never gets a seeded buffer, and the batch fit of optimise(opt) with no returns refuses it. This function runs the warm-up and names the wrapper that is left, so the entry of the online arm can refuse it before the first fold. One method answers every estimator. An estimator with no wrapper outside the route of the warm-up answers nothing, so a user optimiser that holds an Online outside its prior gets its path too. An OnlinePortfolioSelection head and a PreviousWeights step their fallback, so their warm-up resolves the fallback, and the function names a wrapper that is left there, such as one in an inner optimiser of a meta-optimiser that is the fallback.

Algorithm

  1. Resolve the wrappers of opt with update_online_estimator.
  2. Return the dotted path of the first wrapper in the result, from online_wrapper_path.

Arguments

  • opt: The optimiser that enters the online arm.

Validation

Returns

  • path::Option{<:String}: The dotted path of the unresolved wrapper, or nothing.

Related

source
PortfolioOptimisers.assert_stateless_schedule — Function
assert_stateless_schedule(opt::JuMPOptimisationEstimator)
assert_stateless_schedule(opt::Union{HierarchicalRiskParity, HierarchicalEqualRiskContribution, SchurComplementHierarchicalRiskParity})
assert_stateless_schedule(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator})
assert_stateless_schedule(opt)
assert_stateless_schedule(pe::TimeDependent, opt)
assert_stateless_schedule(pe, opt)

Refuses a TimeDependent schedule on a field that carries a state, with an error that names the field.

A schedule replaces the value of its field at every fold, and the loop threads the state through that value. The replacement drops the state, so the value that a fold receives never saw the rows folded before it. The one such field that a schedule can reach is the pe of an optimiser, whose type bound admits a schedule. The opt field of a JuMP head or of a hierarchical head holds the bundle, and its bound refuses a schedule at construction, so the heads only recurse. This function walks the route of update_online_estimator. A schedule on any other field passes, the inner optimisers of a meta-optimiser included. The loop resolves such a schedule on the copy it fits at each fold, and the state stays on the estimator it threads. No schedule carries a state across a swap.

Algorithm

The method that Julia selects is the algorithm.

  1. A JuMP head or a hierarchical head: recurse on the bundle in opt.opt.
  2. An optimiser that holds a prior: check its prior with the two-argument method, assert_stateless_schedule(opt.pe, opt). The optimiser is the second argument, so that the message names it.
  3. A TimeDependent prior: throw. Any other prior passes.
  4. Any other estimator passes.

Arguments

  • opt: The estimator that enters the online arm of the fold loop, or the optimiser that holds pe.
  • pe: The prior of opt.

Validation

  • No pe on the route of the step is a TimeDependent. An ArgumentError that names the optimiser is thrown otherwise.

Related

source
PortfolioOptimisers.assert_online_entry — Method
assert_online_entry(est::TimeDependent)
assert_online_entry(est)

Refuses, at the entry of the online arm of the fold loop, an estimator that is not the configuration alone.

The loop starts cold. It reads its argument as configuration, as the batch loop does, where prior(pe, X) ignores any state that pe carries. The warm-up folds the first training window into an estimator that has folded nothing, so a state at entry would count its rows twice. Resume is the entry for an estimator that carries a state. Every refusal runs before the first solve.

Algorithm

  1. Refuse a TimeDependent schedule of optimisers at the root.
  2. Refuse a schedule on the pe of an optimiser, through assert_stateless_schedule.
  3. Find the first state in the tree of est with online_entry_state, and refuse est when there is one.
  4. Find the first wrapper that the warm-up leaves unresolved with online_unreached_path, and refuse est when there is one.

Arguments

  • est: The estimator that the loop takes.

Validation

  • est is not a TimeDependent. An ArgumentError is thrown otherwise, because the loop threads one estimator and a schedule is a different one at each fold.
  • No pe on the route of the step holds a TimeDependent. An ArgumentError is thrown otherwise.
  • No cache in the tree of est holds a state. An ArgumentError that names the field is thrown otherwise.
  • Every Online wrapper in est sits where the warm-up resolves it. An ArgumentError that names the field is thrown otherwise.

Related

source
PortfolioOptimisers.assert_step_tracks_no_series — Function
assert_step_tracks_no_series(opt)

Refuses, at the online step of an optimiser, a ReturnsTracking anywhere in its tree, with an error that names the field.

A returns series is fixed at construction, and each step changes the rows that a tracking error reads. An expanding step adds rows, so the series has the length of the rows at no more than one step, and tracking_benchmark refuses every other step. A capped step under max_history drops the oldest rows, so the series and the window keep the same length and cover different rows. No length check sees that misalignment, so the step refuses the series before it folds a row. The walk is returns_tracking_path.

Arguments

  • opt: The optimiser that takes the step.

Validation

  • No field in the tree of opt holds a ReturnsTracking. An ArgumentError that names the field is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.step_fallback — Function
step_fallback(fb::OptimisationEstimator, rd::ReturnsResult)
step_fallback(fb, rd::ReturnsResult)

Folds the rows of an online step into the fallback of a head that rebuilds no returns data.

Two heads read out with no returns data to hand on: an OnlinePortfolioSelection head reads out a Recursion Result, and a PreviousWeights reads only its w. So their fallback gets no rows when optimise(opt) runs with no data. A fallback that steps with the head holds its own state, and optimise(fb) with no data reads that state out as the online step of any optimiser does. So every fallback that the batch verb admits also answers online, and so does the fallback of a PreviousWeights that is itself a fallback. A fallback estimator steps through partial_fit!. Any other value passes unchanged: nothing, a precomputed result, which answers itself, and a TimeDependent schedule, whose entries are batch configuration and carry no state.

Arguments

  • fb: The fallback of the head.
  • rd: The returns data of the step.

Validation

Returns

  • fb: The fallback, stepped when it is an estimator.

Related

source
PortfolioOptimisers.read_out_fallback — Function
read_out_fallback(opt::OptimisationEstimator, res::OptimisationResult; kwargs...)

Walks the fallback chain of a head that reads out with no data, after its own result res. The two callers are the methods of optimise(opt) with no data for OnlinePortfolioSelection and PreviousWeights.

The batch verb hands the returns data of the head to each fallback. A head that reads out with no data has none to hand on, so each fallback reads itself out with optimise(fb) and no data, from the rows that step_fallback folded into it. The chain of the answer then starts with the failed pair of the head, and the chain that the fallback walked follows it, as in the batch verb.

Algorithm

  1. Return res when it succeeded or when opt.fb is nothing.
  2. Otherwise warn, and read the fallback out with optimise(opt.fb; kwargs...), giving fbres.
  3. Return fbres through factory with the chain of the failed pair of opt and res, followed by the chain of fbres when the fallback is an estimator that walked one. A precomputed result answers itself, and its own fb is not a chain that this function walked.

Arguments

  • opt: The head, with its TimeDependent schedules reset.
  • res: The result of the head with no data.
  • kwargs: The keyword arguments of optimise with no data on the fallback, passed on unchanged.

Returns

  • res::OptimisationResult: res, or the Result of the fallback chain when res failed.

Related

source
PortfolioOptimisers.returns_result — Method
returns_result(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator})
returns_result(opt::Union{EqualWeighted, RandomWeighted, BestConstantRebalancedPortfolio})
returns_result(opt::JuMPOptimisationEstimator)
returns_result(opt::Union{HierarchicalRiskParity, HierarchicalEqualRiskContribution, SchurComplementHierarchicalRiskParity})

Rebuilds the ReturnsResult of the observations that an optimiser has folded.

This is the step of optimise(opt) with no returns that rebuilds the returns data. The rows come from the context when it keeps them: in a head that holds no prior, and in an optimiser whose prior folds the returns in excess of the benchmark under x_src = ExcessSource(). Otherwise they come from the buffer of the prior. Every other column and the pinned context come from the ReturnsBufferState of the optimiser. The factor column comes from the context when the tree of the prior never reads it, and from the buffer of the prior otherwise. The result is a new ReturnsResult, equal field by field to the one that a batch fit over the same observations reads.

Arguments

  • opt: The optimiser, or the bundle it holds.

Validation

  • opt carries a fold context. An ArgumentError is thrown otherwise.

Returns

  • rd::ReturnsResult: The observations folded so far.

Related

source
PortfolioOptimisers.held_timestamps — Function
held_timestamps(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator, EqualWeighted, RandomWeighted, BestConstantRebalancedPortfolio})
held_timestamps(opt::JuMPOptimisationEstimator)
held_timestamps(opt::Union{HierarchicalRiskParity, HierarchicalEqualRiskContribution, SchurComplementHierarchicalRiskParity})
held_timestamps(::PreviousWeights)
held_timestamps(cache::ReturnsBufferState)
held_timestamps(::Nothing)

Reads the timestamps that the fold context of an optimiser holds, or answers nothing when it holds none.

The alignment check of Resume reads this accessor. The ReturnsBufferState keeps the timestamps of the folded observations and drops the oldest under a cap, so under Online(pe; max_history = w) it holds the last w. A head forwards to the bundle it holds, and an optimiser that holds the context reads its cache, as in returns_result. A Fold Context answers its timestamps. An optimiser that has taken no step holds no context, so it answers nothing. PreviousWeights reads no observation and keeps no context, so it answers nothing, and a resume refuses it.

Arguments

  • opt: The stepped optimiser.

Returns

  • ts::Option{<:AbstractVector}: The held timestamps, in order, or nothing.

Related

source
held_timestamps(opt::OnlinePortfolioSelection)

Returns every timestamp that the head's state holds, or nothing before the first step.

Related

source
held_timestamps(p::Pipeline)

Returns the timestamps that a stepped Pipeline holds, through the state of its row owner, or nothing.

The state that the pipeline carries selects the answer.

  • Under Online(pipe) the PipelineBufferState holds the input data, and the timestamps of that data are the answer.
  • A prior owner leaves the Pipeline its own ReturnsBufferState, which holds them.
  • An optimisation owner keeps its own Fold Context, and the pipeline holds none, so the owner answers (held_timestamps).
  • A pipeline whose owner carries no state, or that has no owner, has taken no step, and answers nothing.

At the price level the answer of the fold route starts one row after the answer of the refit route. A PricesToReturns step drops the first price row, so the returns carry the timestamps of the prices from the second row on. Both answers are a run of the timestamps of the price data that ends at the last row folded, which is what the alignment check of Resume reads.

Related

source
held_timestamps(p::Pipeline, cache::PipelineBufferState)
held_timestamps(p::Pipeline, cache::ReturnsBufferState)
held_timestamps(p::Pipeline, ::Nothing)

The three arms of held_timestamps(p::Pipeline), selected by dispatch on the state that the pipeline carries, which the one-argument method passes.

A buffer of the input data answers with the timestamps of that data, and a Fold Context answers with its own. With no state, the row owner (pipeline_row_owner) answers when it carries a state (online_entry_state), and the method answers nothing otherwise.

Related

source
PortfolioOptimisers.batch_without_state — Function
batch_without_state(opt, pe::AbstractPriorResult)
batch_without_state(opt, pe)

Answers for an optimiser that has taken no step.

An optimiser whose prior is already a result needs no returns to solve. This function returns it with an empty ReturnsResult, which is the batch entry that optimise(opt) has always had. Any other optimiser has no state and no prior result to answer from, and this function refuses it with an ArgumentError. The message names the step that fills the state and the batch function that takes the returns.

Validation

Related

source
PortfolioOptimisers.batch_from_state — Function
batch_from_state(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility, NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator})
batch_from_state(opt::Union{EqualWeighted, RandomWeighted, BestConstantRebalancedPortfolio})
batch_from_state(opt::JuMPOptimisationEstimator)
batch_from_state(opt::Union{HierarchicalRiskParity, HierarchicalEqualRiskContribution, SchurComplementHierarchicalRiskParity})
batch_from_state(opt::FiniteAllocationOptimisationEstimator)
batch_from_state(opt::OptimisationEstimator)

Turns a folded optimiser into the batch call that reads it out, an estimator and a ReturnsResult.

optimise(opt) with no returns rebuilds the ReturnsResult and calls the ordinary batch path, so no member above the prior needs a method of its own. The prior becomes prior(pe), a prior result, which the batch path does not refit, so the solve reads the folded prior.

The rebuilt ReturnsResult holds every observation that the step folded, and the folded prior drops the observations of a Shared Series gap, see shared_series_buffer. The folded prior records the observations that it dropped in its ss_gaps, see shared_series_record, so the function returns the whole ReturnsResult. The head then drops the marked observations itself, see prior_shared_series_rows, and the prior and every reader that pairs its observations with the last observations of the returns data read the same observations, as in the batch fit. Each fallback receives the whole ReturnsResult and drops by its own rule, as in the batch fit. This function drops the context, because the estimator it returns is a batch configuration over the rebuilt ReturnsResult. It never writes the state, so two calls on one estimator give the same answer.

The cache of the optimiser selects the branch. An optimiser whose cache is nothing has taken no step, and batch_without_state answers for it. This function returns any other optimiser with an empty ReturnsResult, and its batch function decides whether it answers without returns. optimise(opt) with no returns does not reach this function for a PreviousWeights, because that head has its own method of optimise(opt) with no data.

Algorithm

  1. A JuMP head or a hierarchical head reads out the bundle it holds in opt, and rebuilds itself around the bundle.
  2. An optimiser with no state goes to batch_without_state.
  3. Rebuild rd with returns_result.
  4. Rebuild the optimiser with cache = nothing. An optimiser that holds a prior also replaces pe with prior(pe), the result of the folded prior.
  5. Return the optimiser and rd.

Arguments

  • opt: The optimiser to read out.

Validation

  • opt is not a finite allocation. An ArgumentError is thrown otherwise, because a finite allocation has no step.
  • Everything batch_without_state and returns_result refuse.

Returns

  • (opt, rd)::Tuple: The batch estimator and the ReturnsResult to run it over.

Related

source
batch_from_state(opt::OnlinePortfolioSelection)

Refuses the head with an ArgumentError, because the head has no batch estimator to return.

The head gives a Recursion Result. optimise(opt) rebuilds no returns data, runs no batch path and reads the state directly.

Related

source
PortfolioOptimisers.show_fields — Method
show_fields(
    opt::Union{BaseClusteringOptimisationEstimator, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator, BestConstantRebalancedPortfolio, EqualWeighted, InverseVolatility, JuMPOptimiser, NestedClustered, RandomWeighted}
) -> Any

Renders every field of an optimiser that takes the online step, except its cache.

The cache holds the state of an incremental fit, not the configuration that a reader looks up, and it would print under every optimiser that holds it. set_show_nothing_fields!(:Name, true) renders it for the type Name.

Arguments

  • opt: The optimiser.

Returns

  • fields::Tuple: Every field name except cache.

Related

source
PortfolioOptimisers.optimiser_shared_series_rows — Function
optimiser_shared_series_rows(rd::ReturnsResult,
                             opt::Union{OptimisationEstimator, OptimisationResult})
optimiser_shared_series_rows(rd::ReturnsResult,
                             opt::Union{JuMPOptimisationEstimator,
                                        HierarchicalRiskParity,
                                        HierarchicalEqualRiskContribution,
                                        SchurComplementHierarchicalRiskParity})
optimiser_shared_series_rows(rd::ReturnsResult,
                             opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator,
                                        InverseVolatility, NestedClustered,
                                        BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator})

Return the observations of rd that one solve of opt keeps, or nothing when it keeps every observation, see shared_series_solve.

The optimisers that hold a prior are the ones whose online step folds the prior, so the two unions are the unions of partial_fit!. An optimiser drops a gap of a vector benchmark when any reader of its group states ExcessSource, see group_source, and a gap of the factor returns when reads_factor_returns answers true for its prior, see prior_shared_series_rows. A raw reader of the group loses the rows too, so every reader of the head pairs the same observations. An outer estimator reads the group of the whole head and the prior of the configuration that it holds in opt. An optimiser that holds no prior and reads no benchmark, and a precomputed result, keep every observation.

Algorithm

The method that Julia selects is the algorithm.

  1. An optimiser with no prior, or a result: return nothing.
  2. An outer estimator: return prior_shared_series_rows(rd, group_source(opt), opt.opt.pe).
  3. A configuration or a meta-optimiser: return prior_shared_series_rows(rd, group_source(opt), opt.pe).

Related

source
PortfolioOptimisers.prior_shared_series_rows — Function
prior_shared_series_rows(rd::ReturnsResult, gsrc::Option{<:AbstractReturnsSource}, pe)
    -> Option{Vector{Int}}

Return the observations of rd that a head keeps for its group and its prior, or nothing when it keeps every observation.

The head drops the observations of a gap that shared_series_rows finds under the source of its group gsrc, and of a gap of the factor returns when its prior estimator reads them. A prior that is already a result reads no factor return in this solve, so the head reads the record of the result instead. When rd is the returns data that the result read, the head also drops the observations that pe.ss_gaps marks, see shared_series_kept_rows. Then the readers of the head pair the rows of pe.X with the last rows of the cut data, as they do for a prior that the head fits. So after an online fold the head reads the whole returns data, and each fallback cuts it by its own rule, see batch_from_state.

Algorithm

  1. Find the observations that the group and the prior estimator keep with shared_series_rows, giving rows.
  2. Find the observations that the record of a prior result keeps with shared_series_kept_rows, giving kept. An estimator, and rd with no asset returns, give nothing.
  3. Return the observations that both keep, with intersect_kept_rows.

Arguments

  • rd: The returns result to use.
  • gsrc: The source of the group of the head, see group_source.
  • pe: The prior of the head, an estimator, a prior result, or nothing.

Returns

  • rows::Option{Vector{Int}}: The positions of the kept observations, or nothing.

Related

source
PortfolioOptimisers.optimiser_shared_series_view — Function
optimiser_shared_series_view(rd::ReturnsResult,
                             opt::Union{OptimisationEstimator, OptimisationResult},
                             rows = nothing)
optimiser_shared_series_view(rd::ReturnsResult,
                             opt::Union{JuMPOptimisationEstimator,
                                        HierarchicalRiskParity,
                                        HierarchicalEqualRiskContribution,
                                        SchurComplementHierarchicalRiskParity,
                                        JuMPOptimiser, BaseClusteringOptimisationEstimator,
                                        InverseVolatility, NestedClustered,
                                        BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator},
                             rows::Option{<:VecInt} = optimiser_shared_series_rows(rd, opt))

Return the returns data that one solve of opt reads, with the observations that optimiser_shared_series_rows drops dropped, see shared_series_solve.

An optimiser that holds a prior cuts rd with shared_series_cut under the source of its group, which also refreshes the estimation mask of a matrix benchmark. An optimiser that holds no prior, and a precomputed result, return rd.

Related

source
PortfolioOptimisers.fits_prior — Function
fits_prior(opt::Union{JuMPOptimisationEstimator, HierarchicalRiskParity,
                      HierarchicalEqualRiskContribution,
                      SchurComplementHierarchicalRiskParity})
fits_prior(opt::Union{JuMPOptimiser, BaseClusteringOptimisationEstimator, InverseVolatility,
                      NestedClustered, BaseStackingOptimisationEstimator, BaseSubsetResamplingOptimisationEstimator})

Answer true when one solve of opt fits its prior on the returns data, so that the prior of its result counts the observations of the data after the cut of the solve.

A prior that is already a result is not fitted again, and it keeps the record of the data that it read. shared_series_solve asks only after a cut that dropped an observation, and only an optimiser that holds a prior drops one, so the function has no method for an optimiser that holds no prior.

Returns

  • flag::Bool: Whether the solve fits a prior.

Related

source
PortfolioOptimisers.set_result_prior — Function
set_result_prior(res::Union{NaiveOptimisationResult, NestedClusteredResult,
                            StackingResult, SubsetResamplingResult,
                            SchurComplementHierarchicalRiskParityResult,
                            HierarchicalResult, ResolvedConstraintBlock},
                 f::Function)
set_result_prior(res::VecRcb, f::Function)
set_result_prior(res::ResolvedJuMPOptimiser, f::Function)
set_result_prior(res::JuMPOptimisationResult, f::Function)
set_result_prior(res::Union{RiskJuMPOptimisationResult,
                            NonRiskJuMPOptimisationResult,
                            HierarchicalRiskParityResult,
                            HierarchicalEqualRiskContributionResult},
                 f::Function)

Rebuild an optimisation result with the prior result f(pr) in place of its own prior pr.

shared_series_solve is the one caller. It replaces the prior with the same prior plus the record of the cut, so every other field stays correct. The function reads the prior from the field that holds it, by the type of the result, so no caller reads res.pr of a result whose type it does not know.

Algorithm

The method that Julia selects is the algorithm.

  1. A result that holds the prior pr in its first field, and a ResolvedConstraintBlock: rebuild it through its positional constructor with f(pr) as the first field.
  2. The ResolvedJuMPOptimiser of a JuMP head: compute f(pr) once, and rebuild the bundle with it in its first field. Rebuild its block Result in cbr with the same prior, by step 1, so that the block and the bundle hold one prior. When cbr is a vector of block Results, rebuild its first entry, the block of the head, by step 1. Each later block keeps its own prior, because it resolved against the prior of its own block on its own rows.
  3. The shared core of a JuMP head: rebuild it with the bundle of step 2, through the inner constructor, as set_retcode does.
  4. A JuMP head, or a hierarchical head, that holds the core in its first field jr or hr: rebuild it through its positional constructor with the core of step 1 or step 2.

Validation

  • The prior of each type is in its first field, or in the core in its first field. Every concrete type above obeys this rule, so the method checks nothing.

Returns

  • res: The rebuilt result.

Related

source