GitHub

Online (b): private API

PortfolioOptimisers.CVE_Onl — Type
const CVE_Onl = Union{<:CrossValidationEstimator, <:Online{<:CrossValidationEstimator}}

Alias for a cross-validation scheme, plain or an Online Scheme.

An Online Scheme is an Online around a walk-forward, built by the function constructor of the scheme, OnlineIndexWalkForward, OnlineDateWalkForward or OnlineHindsightSplit. Its supertype is AbstractEstimator and not CrossValidationEstimator, because a struct has one supertype. Each function that takes a scheme dispatches on this alias, or on an alias built from it, so the wrapped form reaches the same functions as the plain form. The fold loop then reads folds_are_stepped from its type.

Related

source
PortfolioOptimisers.assert_sample_buffer — Method
assert_sample_buffer(_::Online)

Refuses an Online that reached a fold without being resolved, naming the warm-up that resolves it.

update_online_estimator replaces each wrapper that it can reach at warm-up. So a wrapper that reaches a fold is one that the warm-up did not see. A callable TimeDependent makes one. The value of a schedule is computed at each fold, after the warm-up, so nothing seeds a wrapper that the callable returns. The constructor of a schedule refuses a wrapper in its vector and default forms. It cannot refuse the return value of a callable, because that value does not exist until the fold.

A callable that returns the estimator that the wrapper would make does not solve the problem. A schedule replaces the value of the field at each fold, and the step passes the buffer through that field. So a schedule that returns a captured estimator returns the buffer as it was when the closure was built, and it discards each step since then.

Arguments

  • ::Online: The unresolved wrapper.

Validation

  • Always throws an ArgumentError.

Returns

  • Never returns.

Related

source
PortfolioOptimisers.online_candidate_fields — Function
online_candidate_fields(x)

Field names of x whose type admits an Online, the candidate set that online_fields narrows by value.

fieldtype alone decides whether a field can hold a wrapper, because a @concrete estimator records the type of the value in the type parameter of the field. The type of a field that holds a static estimator does not intersect Online. So a generated function computes the tuple once for each estimator type, and for an estimator with no wrapper the tuple is empty at compile time. The warm-up then walks no field.

Related

source
PortfolioOptimisers.online_fields — Function
online_fields(
    est::Union{AbstractEstimator, CovarianceEstimator}
) -> Any

Returns the tuple of field names of est whose values are Online.

The scan reads the fields of the outer estimator, so the constructor signatures that admit a wrapper, through Online_Option and Onl, decide which estimators can take the step from a buffer. No list is kept by hand. The scan visits only the candidates of online_candidate_fields, and the compiler removes the other fields.

Arguments

  • est: Estimator whose fields are scanned.

Returns

  • fns::Tuple: The field names holding an Online.

Related

source
PortfolioOptimisers.update_online_estimator — Function
update_online_estimator(est)

Resolves the Online declarations of an estimator, seeding the sample buffer each one asks for.

The warm-up calls it once, before the first fold. It replaces each wrapper with the estimator that the wrapper holds, rebuilt through its keyword constructor with the state of online_state_seed in cache. That state is an empty SampleBufferState with the cap of the wrapper, unless the family has a seed of its own. The function then rebuilds the outer estimator through its keyword constructor, so each construction check runs again. The result holds no Online, and each estimator that was wrapped carries the state that its refit or its carry reads.

The function finds a wrapper in the fields of the estimator that it receives, and in the fields of an estimator inside a wrapper. An outer estimator that passes estimators across a boundary of its own, such as a meta-optimiser or a pipeline, has a method of its own that recurses, as it has for update_time_dependent_estimator.

The function takes no fold context, unlike update_time_dependent_estimator. The seed reads nothing from a fold, because the wrapper resolves once and not at each fold.

Algorithm

  1. For an Online, rebuild the wrapped estimator with cache set to the state of online_state_seed, and continue with that estimator.
  2. Find the fields of the estimator that hold a wrapper, with online_fields. Return the estimator when there is none.
  3. Resolve the value of each of these fields with this function.
  4. Rebuild the estimator with the resolved values through rebuild_estimator.

Arguments

  • est: Estimator, wrapper, or nothing.

Returns

  • Estimator carrying a seeded buffer wherever a wrapper stood, and holding no Online.

Related

source
update_online_estimator(pe::Union{<:HighOrderPriorEstimator, <:BlackLittermanPrior})

Resolves the Online declarations under the prior that a wrapping prior embeds, at warm-up.

The generic method scans the fields of the wrapping prior alone, and a wrapping prior holds its embedded prior in its own pe field, so the wrapping prior defines the recursion itself. HighOrderPriorEstimator(; pe = BlackLittermanPrior(; pe = Online(EmpiricalPrior()), ...)) resolves through this method. The generic scan of the own fields of the wrapping prior finds no wrapper there, because pe holds a plain prior, so the generic method would leave the wrapper two levels down unseeded. A wrapper in the own pe of the wrapping prior, as in HighOrderPriorEstimator(; pe = Online(EmpiricalPrior())), is seeded the same way, because the recursion meets it at the first level. The generic method still scans the co-moments of a HighOrderPriorEstimator.

Algorithm

  1. Resolve every field that online_fields names with update_online_estimator, giving repl.
  2. Resolve pe.pe with update_online_estimator, which recurses through a wrapping prior.
  3. Rebuild the prior from repl and the resolved pe.pe, the second taking precedence for pe, and return it.

Arguments

  • pe: The wrapping prior.

Returns

  • pe: The prior, with every wrapper under its embedded prior resolved.

Related

source
update_online_estimator(opt::JuMPOptimisationEstimator)
update_online_estimator(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})
update_online_estimator(opt::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling})

Resolves the Online wrappers in the prior of an optimiser, at the warm-up.

The JuMP and hierarchical heads resolve the bundle they hold, and the bundle, the meta-optimisers and InverseVolatility resolve their pe. The step forwards observations to that prior alone, so this function resolves no other wrapper. A wrapper in an inner optimiser of a meta-optimiser, in a fallback, or in any other field stays unresolved, and the entry of the online arm of the fold loop refuses it with an error that names the field.

Arguments

  • opt: The optimiser.

Validation

Returns

  • opt: The optimiser, with each wrapper in its prior resolved to an estimator that carries a seeded buffer.

Related

source
update_online_estimator(opt::Union{<:OnlinePortfolioSelection, <:PreviousWeights})

Resolves the Online wrappers in the fallback of the head, at the warm-up.

The two heads step their fallback through step_fallback, so a wrapper in the prior of a fallback estimator declares the window that the fallback folds. The method resolves the fallback as the warm-up resolves an optimiser. A PreviousWeights that is the fallback of the head resolves its own fallback in turn. Any other fallback holds no wrapper to resolve, so the head passes unchanged. The recursion of the head holds no wrapper.

Related

source
update_online_estimator(p::Pipeline)

Resolves the Online declarations in the steps of a Pipeline, at warm-up.

On the fold route update_online_step resolves every step. So an Online(EmpiricalPrior(); max_history = w) step becomes the prior with its buffer, and an optimisation step resolves the wrappers under its own prior. On the refit route the pipeline carries a PipelineBufferState, which Online(pipe) seeded, and the method returns the pipeline unchanged. No member folds under a refit, and assert_online_entry already refused any wrapper below the root with an error that names it.

Related

source
PortfolioOptimisers.estimator_fields — Function
estimator_fields(x)

Field names of x whose type admits an estimator, the candidate set that online_wrapper_path and copy_states walk.

fieldtype alone decides whether a field can hold an estimator, because a @concrete estimator records the type of the value in the type parameter of the field. So a generated function computes the tuple once for each estimator type, as for online_candidate_fields. A field that holds a vector of estimators or a result is not a candidate. A vector is a batch configuration that the step never folds into, and a result carries no state field. A walk that must reach the estimators a vector holds reads every field instead, as online_entry_state and seed_window_path do. A field that holds a TimeDependent schedule is a candidate, because a schedule is an estimator. The TimeDependent methods of the two walks answer nothing for it, because the entries of a schedule resolve at each fold.

Related

source
PortfolioOptimisers.online_entry_state — Function
online_entry_state(est)
online_entry_state(v::AbstractVector)
online_entry_state(p::Pair)
online_entry_state(::TimeDependent)

Names the first field of an estimator tree that carries a partial-fit state, or answers nothing.

The cold start of the fold loop uses this walk. The online arm of fold_loop reads its argument as configuration only, so it refuses by name an estimator that holds a state, and this walk finds the state. The answer is the path from the root, for example "opt.pe.me.cache" for the state of a mean estimator under a JuMP head. The walk reads every field, not only the fields of estimator_fields, because a state can stand in an estimator that a vector holds. A Descriptor that the caller folded before the call, in the factor list of a CrossSectionalFactorPrior, is such an estimator, and the first step of the carry fold reads its state. So the first partial_fit! of that prior refuses a state with this walk too, because a direct call does not pass through the fold loop. A vector answers with the index of its element in brackets, as element_path states, and a Pair answers through its value, so the path is "factors[2].descriptors[1].cache" for that Descriptor. A TimeDependent schedule answers nothing, because its entries are batch configuration that resolves at each fold, and the loop passes no state through them. A value that is not an estimator also answers nothing.

Algorithm

  1. Answer "cache" when est has a cache field that is not nothing.
  2. Walk every field of est in order, and find the path in each field with this function.
  3. Answer the first path found, with the field name before it, as tree_path joins them, or nothing when no field gives one.

Arguments

  • est: The estimator, or any value a field holds.
  • v: A vector that a field holds.
  • p: A Pair that a vector holds, such as a member of a factor list.

Returns

  • path::Option{<:String}: The path of the first state found, or nothing.

Related

source
online_entry_state(o::Online)

Name the state that a wrapped estimator carries at the entry of the online arm of the fold loop, or answer nothing.

This is the Online method of online_entry_state, for a wrapper at the root of covariance_forecast_evaluation. A wrapper holds no cache of its own, so the walk enters the estimator that it wraps and puts est. before the path that it finds there. For a covariance estimator that the caller folded before the call, the path is est.cache. The generic walk through estimator_fields would give the same answer. This method gives it without a request for a field that the wrapper does not have.

Related

source
online_entry_state(p::Pipeline)

Names the first state that a Pipeline carries at the entry of the online arm of the fold loop, or answers nothing.

The answer is "cache" when the pipeline holds a state of its own. Otherwise the method walks the steps in order, and answers the path of the first state under a step, prefixed with the name of the step.

Related

source
PortfolioOptimisers.assert_cold_carry_entry — Function
assert_cold_carry_entry(est, ::Nothing, name::AbstractString) -> nothing
assert_cold_carry_entry(est, ::Any, name::AbstractString) -> nothing

Refuses a partial-fit state in a member of a carry at its first step.

A carry reads every member as configuration at its first step: the carried rows start empty, and each member folds the new observations. A member that holds a state would fold them onto the observations of that state, so its moment would read rows that the carried rows and every refitted member do not read, and the prior would not equal the batch fit over the folded observations. The fold loop refuses such an estimator at its entry, and a direct partial_fit! does not pass through the fold loop, so each carry calls this function before its first step. The second argument marks the state of the carry itself: nothing at the first step, and any other value after it, when each member holds the state that the carry folded. A carry that keeps its state in a field passes that field. A carry that keeps its rows in a member, as HighOrderPriorEstimator keeps them in its embedded prior, passes the online_entry_state of that member.

Algorithm

The method that Julia selects is the algorithm.

  1. cache is not nothing: return, because the members hold the states that the carry folded.
  2. cache is nothing: find the first state in est with online_entry_state, and refuse it.

Arguments

  • est: The estimator that starts its carry.
  • cache: The state of the carry, or the path of the state of the member that carries the rows, and nothing at the first step.
  • name: The name of the estimator in the message.

Validation

  • At the first step, no member of est holds a partial-fit state. An ArgumentError that names the path of the state is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.online_wrapper_path — Function
online_wrapper_path(est)
online_wrapper_path(::Online)
online_wrapper_path(::TimeDependent)

Names the first field of an estimator tree that holds an Online declaration, or answers nothing.

assert_batch_entry uses this walk. Only the warm-up of the online arm of the fold loop resolves a wrapper, with update_online_estimator. Without the check, a batch fit that reaches a wrapper gets a MethodError at prior(pe, X), not a refusal. The walk reads the fields of estimator_fields, and the answer is the dotted path from the root. It is "pe" for a wrapped prior on a naive optimiser, and "opt.pe" for a wrapped prior under a JuMP head. The walk does not enter a field that holds a wrapper, because the warm-up resolves the wrappers below it. A wrapper at the root answers "", the empty path. A TimeDependent schedule answers nothing, because its constructor refuses a wrapper among its entries. A value that is not an estimator also answers nothing.

Algorithm

  1. Answer "" when est is an Online.
  2. Walk the fields of estimator_fields in order, and answer the name of the first field that holds a wrapper.
  3. For a field that holds no wrapper, find the path in the field with this function, and answer it with the field name and a dot before it.
  4. Answer nothing when no field gives a path.

Arguments

  • est: The estimator, or any value a field holds.

Returns

  • path::Option{<:String}: The dotted path of the first wrapper found, "" when est is itself one, or nothing.

Related

source
PortfolioOptimisers.seed_window_path — Function
seed_window_path(est)
seed_window_path(v::AbstractVector)
seed_window_path(p::Pair)
seed_window_path(::Any)

Names the first field of an estimator tree that holds a windowed estimator under SeedWindow, or answers nothing.

The constructor of Online uses this walk to refuse a seed window, because the wrapper refits its estimator at each step. The answer is the path from the root to the rule field, such as "ce.rule" for the covariance member of a prior. A windowed estimator under a seed window answers "rule" through its own method. A value that is not an estimator answers nothing.

The walk reads every field, not only the fields of estimator_fields. A refit reaches the estimators that a vector holds, such as the factor list of a CrossSectionalFactorPrior, the Descriptors of a DescriptorScores and the priors of an OpinionPoolingPrior, so the refusal must reach them too. A vector answers with the index of its element in brackets, and a Pair answers through its value, so a Descriptor in a factor list gives a path such as "factors[2].descriptors[1].ce.rule". element_path walks the vector.

Algorithm

  1. Walk every field of est in order, and find the path in each field with this function.
  2. Answer the first path found, with the field name before it, as tree_path joins them.
  3. Answer nothing when no field gives a path.

Arguments

  • est: The estimator, or any value a field holds.
  • v: A vector that a field holds.
  • p: A Pair that a vector holds, such as a member of a factor list.

Returns

  • path::Option{<:String}: The path of the first seed window found, or nothing.

Related

source
seed_window_path(
    _::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> String

Names the rule field of a windowed estimator under a SeedWindow, the end of the walk of seed_window_path.

Arguments

  • est: Windowed estimator under a seed window.

Returns

  • path::String: "rule".

Related

source
PortfolioOptimisers.assert_batch_entry — Function
assert_batch_entry(est, entry::AbstractString)

Refuses an estimator that holds an Online at the entry of a batch fit, with an error that names the path of the wrapper.

A wrapper is a declaration that the online arm of the fold loop resolves at its warm-up. A batch fit runs no warm-up. A batch fit is a plain optimise, or a fold of a scheme that is not an Online Scheme. So the wrapper would reach prior(pe, X) unresolved and get a MethodError that names the whole type. The refusal names the dotted path that online_wrapper_path finds, and the two exits. One exit is an Online Scheme, such as OnlineIndexWalkForward. The other exit is the estimator without the wrapper. Each caller passes an optimiser, which is never a wrapper, so the path at this entry is never the empty path.

Algorithm

  1. Find the path of the first wrapper in est with online_wrapper_path.
  2. Refuse est when the path is not nothing, and name the path and the entry function.

Arguments

  • est: The estimator handed to the entry function.
  • entry: The name of the entry function, as the message reads it.

Validation

  • No field in the tree of est holds an Online. An ArgumentError naming the field is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.online_state_seed — Method
online_state_seed(
    _::Union{AbstractEstimator, CovarianceEstimator},
    max_history::Union{Nothing, Integer}
) -> Union{ReturnsBufferState{Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ReturnsBufferState{Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, __T_max_history} where __T_max_history<:Integer}

Builds the empty state an Online seeds into the estimator it wraps.

One sample buffer works for each estimator that refits, so the seed needs no knowledge of the type. A prior whose batch verb reads a factor matrix beside the returns records the factor rows in the same buffer, and needs_factor_returns decides whether its fold receives them. An estimator whose state is not a sample buffer has a method of its own that returns the state that its call with no data reads. A prior-less optimiser head is such an estimator, because its state is a fold context. Nothing else about the wrapper changes.

The function takes the estimator and not its type, so a family that sizes its seed from a field can read that field.

Arguments

  • est: The estimator the wrapper wraps, read for its type.
  • max_history: The wrapper's cap, carried into the state.

Returns

  • state::AbstractPartialFitState: The empty state to seed, carrying the cap and no observations.

Related

source
PortfolioOptimisers.tree_path — Function
tree_path(
    head::Union{AbstractString, Symbol},
    path::AbstractString
) -> Any

Joins the name of a field, or the bracketed index of an element, to the path that a walk found below it.

A path that starts with a bracket follows the name with no dot, so a walk names a member of a vector field as "factors[2]" and not as "factors.[2]". An empty path answers the name alone, so a walk whose leaf answers "" names the field that holds the leaf, such as "tr". Every other path follows the name after a dot.

Arguments

  • head: The field name, or the bracketed index of an element, such as "[2]".
  • path: The path below head.

Returns

  • path::String: The joined path.

Related

source
PortfolioOptimisers.element_path — Function
element_path(walk, v::AbstractVector) -> Any

Walks the elements of a vector in order with walk, and answers the path of the first element that gives one, or nothing.

The walks of an estimator tree that must see the estimators a vector holds call this function: seed_window_path, online_entry_state and returns_tracking_path. The answer starts with the index of the element in brackets, as tree_path joins it.

Arguments

  • walk: The walk, a function that answers a path or nothing for one element.
  • v: The vector.

Returns

  • path::Option{<:String}: The path of the first element that gives one, or nothing.

Related

source