Resume: private API

PortfolioOptimisers.OptimiserResume — Type
const OptimiserResume = Resume{<:Any, <:NonFiniteAllocationOptimisationEstimator}

Groups the Resume declarations whose Result carries an optimiser, so that the methods that take an optimiser can take them.

The methods of cross_val_predict and fit_and_predict that take an optimiser dispatch on this alias. A Resume whose Result carries a Pipeline takes the methods for a Pipeline instead, through PipelineResume. The second type parameter of Resume is the type of the estimator that the Result carries.

Related

source
PortfolioOptimisers.resume_fold_count — Function
resume_fold_count(opt, rd, train_idx) -> Int

Finds, by timestamp, the last fold that a Resume holds, and refuses an rd whose rows do not carry the state.

The last timestamp that the state holds names the last row it folded. The fold whose training window ends at that row is the last fold that the run holds, and the loop continues from the fold after it. The count comes from the state and not from length(res.pred), because a resumed Result holds the new folds only, and the chain Resume(res2) must skip every fold before it.

The check is exact over the held span, and it costs one comparison per held timestamp. Without a cap the state holds every row that it folded, so the check pins the whole prefix. A row dropped or inserted before r moves the row of rd at r, and a scheme with a different window, step or purge moves the training ends. Under a cap the state holds its last rows only, so the check pins those rows and nothing before them. An rd that differs before the held span then resumes to the same weights, because no row before the span reaches the state.

Algorithm

  1. Read the timestamps of rd through data_timestamps, giving given. Refuse nothing.
  2. Read the timestamps of the state through held_timestamps, giving held. Refuse nothing.
  3. Take the last entry of held, giving stop.
  4. Find the first fold whose training window ends at a row with timestamp stop, giving n_old. Refuse when no fold ends there.
  5. Take the last row of that training window, giving r, and the length of held, giving h.
  6. Refuse unless h <= r and held equals given[(r - h + 1):r].
  7. Return n_old.

Arguments

  • opt: The stepped estimator that the Result carries.
  • rd: The full history extended.
  • train_idx: The training windows of every fold that the scheme enumerates over rd.

Validation

  • rd holds timestamps. An ArgumentError is thrown otherwise.
  • The state holds timestamps. An ArgumentError is thrown otherwise.
  • The training window of some fold ends at the last timestamp that the state holds. An ArgumentError is thrown otherwise.
  • The held timestamps equal the timestamps of rd over the held span. An ArgumentError is thrown otherwise.

Returns

  • n_old::Int: The index of the last fold that the run holds.

Related

source
PortfolioOptimisers.assert_resume_scheme — Function
assert_resume_scheme(::Online{<:WalkForwardEstimator})
assert_resume_scheme(cv::WFCVER)
assert_resume_scheme(cv::CVER)

Refuses, by name, a scheme that a Resume cannot re-enter.

Two kinds of scheme are refused: a scheme that is not a walk-forward, and a walk-forward that is not an Online Scheme. Each refusal is a method on the refused type, so dispatch decides it and the message names the cause.

Validation

  • cv is an Online Scheme over a walk-forward. An ArgumentError is thrown otherwise.

Related

source
PortfolioOptimisers.assert_resume_folds — Function
assert_resume_folds(n_old::Integer, n::Integer)

Refuses an extended history that adds no fold to the run that a Resume continues.

Arguments

  • n_old: The index of the last fold that the Result holds.
  • n: The number of folds that the scheme enumerates over the extended history.

Validation

  • n > n_old. An ArgumentError is thrown otherwise.

Related

source
PortfolioOptimisers.assert_resume_full_fold — Function
assert_resume_full_fold(test_idx::VecInt, pred::PredictionResult, ts)

Refuses a Resume whose last held fold was partial.

A fold was partial when its test window in the longer history runs past the last row that the fold predicted. With reduce_test = true the last fold of the old run predicted the rows left at the end of its history. In the longer history those rows are the first part of a full window, with the same training end and the same weights. The fold cannot be skipped without a loss of rows, and it cannot be completed without an entry in the middle of a window, so it is terminal.

The check compares timestamps and not counts. A price-level window of L rows predicts L - 1 returns, and the last return has the timestamp of the last price, so one comparison of timestamps holds at both levels. The message names the workaround with two resumes.

Arguments

  • test_idx: The test window of the last held fold, over the extended history.
  • pred: The prediction of the last held fold.
  • ts: The timestamps of the extended history.

Validation

  • The last timestamp that pred predicted equals ts[last(test_idx)]. An ArgumentError is thrown otherwise.

Related

source
PortfolioOptimisers.copy_states — Function
copy_states(est)
copy_states(td::TimeDependent)

Copies every partial-fit state that an estimator tree carries, and returns the tree rebuilt around the copies.

The walk reads the fields of estimator_fields, as online_wrapper_path does, with a rebuild where that verb makes a report. It does not enter a vector, because no fold writes a state into one, and online_entry_state refuses at the entry a state that a caller put there. An estimator whose subtree holds no state comes back as it is. A TimeDependent schedule comes back unchanged, because its entries are batch configuration that the fold loop resolves per fold and threads no state through. Any value that is not an estimator comes back unchanged too.

Two callers use it. Resume calls it once at entry, so the Result it holds is never written. That copy is necessary because partial_fit! promises nothing about a kept estimator, and a buffer appends into its backing array. The generic method of partial_fit calls it before every fold. The value form therefore keeps its promise on an estimator that folds through the states of its members, such as a HighOrderPriorEstimator or a hierarchical optimiser, as well as on a leaf that carries its own cache. Every state type defines Base.copy, so one copy per state is enough.

Algorithm

  1. Read the cache field of est, or nothing when est has no such field, and copy it through copy_state, giving cache.
  2. Call copy_states on every estimator-valued field of est (estimator_fields), giving repl.
  3. Set moved when an entry of repl is not the object (!==) that est holds in that field.
  4. Return est when cache is nothing and moved is false.
  5. Otherwise rebuild est through rebuild_struct from repl, and from cache too when it is not nothing.

Arguments

  • est: The estimator, or any value that a field holds.

Returns

  • est: The tree, with every state copied.

Related

source
copy_states(p::Pipeline)

Copies every partial-fit state that a Pipeline carries, and rebuilds the pipeline around the copies, for Resume.

The states are the own cache of the pipeline and the states under every step, which copy_states copies. A PipelineStep copies through copy_states(ps::PipelineStep), and a callable and a result pass through the fallback unchanged. A fold of the copy leaves the held rows of the original unchanged.

Related

source
copy_states(ps::PipelineStep)

Copies the partial-fit states under the estimator that a PipelineStep wraps.

The method rebuilds the wrapper around the copied estimator with rebuild_struct, so the step keeps its reads, its writes and its target. A wrapped callable passes through the fallback of copy_states unchanged.

Related

source
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(r::Resume)
needs_previous_weights(r::Resume)
assert_time_dependent_fold_count(r::Resume, n::Integer, all_binds::Bool = true)

Answers the three traits of the fold loop for the estimator that the Result of a Resume carries.

The fold loop reads the traits off the root that it is handed. The loop resolves a Resume at entry and makes the per-fold copy of the estimator that its Result carries, so that estimator answers.

Related

source