Resume: private API
PortfolioOptimisers.OptimiserResume — Type
const OptimiserResume = Resume{<:Any, <:NonFiniteAllocationOptimisationEstimator}Groups the Resume declarations whose Result carries an optimiser, so that the methods that take an optimiser can take them.
The methods of cross_val_predict and fit_and_predict that take an optimiser dispatch on this alias. A Resume whose Result carries a Pipeline takes the methods for a Pipeline instead, through PipelineResume. The second type parameter of Resume is the type of the estimator that the Result carries.
Related
PortfolioOptimisers.resume_fold_count — Function
resume_fold_count(opt, rd, train_idx) -> IntFinds, by timestamp, the last fold that a Resume holds, and refuses an rd whose rows do not carry the state.
The last timestamp that the state holds names the last row it folded. The fold whose training window ends at that row is the last fold that the run holds, and the loop continues from the fold after it. The count comes from the state and not from length(res.pred), because a resumed Result holds the new folds only, and the chain Resume(res2) must skip every fold before it.
The check is exact over the held span, and it costs one comparison per held timestamp. Without a cap the state holds every row that it folded, so the check pins the whole prefix. A row dropped or inserted before r moves the row of rd at r, and a scheme with a different window, step or purge moves the training ends. Under a cap the state holds its last rows only, so the check pins those rows and nothing before them. An rd that differs before the held span then resumes to the same weights, because no row before the span reaches the state.
Algorithm
- Read the timestamps of
rdthroughdata_timestamps, givinggiven. Refusenothing. - Read the timestamps of the state through
held_timestamps, givingheld. Refusenothing. - Take the last entry of
held, givingstop. - Find the first fold whose training window ends at a row with timestamp
stop, givingn_old. Refuse when no fold ends there. - Take the last row of that training window, giving
r, and the length ofheld, givingh. - Refuse unless
h <= randheldequalsgiven[(r - h + 1):r]. - Return
n_old.
Arguments
opt: The stepped estimator that the Result carries.rd: The full history extended.train_idx: The training windows of every fold that the scheme enumerates overrd.
Validation
rdholds timestamps. AnArgumentErroris thrown otherwise.- The state holds timestamps. An
ArgumentErroris thrown otherwise. - The training window of some fold ends at the last timestamp that the state holds. An
ArgumentErroris thrown otherwise. - The held timestamps equal the timestamps of
rdover the held span. AnArgumentErroris thrown otherwise.
Returns
n_old::Int: The index of the last fold that the run holds.
Related
PortfolioOptimisers.assert_resume_scheme — Function
assert_resume_scheme(::Online{<:WalkForwardEstimator})
assert_resume_scheme(cv::WFCVER)
assert_resume_scheme(cv::CVER)Refuses, by name, a scheme that a Resume cannot re-enter.
Two kinds of scheme are refused: a scheme that is not a walk-forward, and a walk-forward that is not an Online Scheme. Each refusal is a method on the refused type, so dispatch decides it and the message names the cause.
Validation
cvis an Online Scheme over a walk-forward. AnArgumentErroris thrown otherwise.
Related
PortfolioOptimisers.assert_resume_folds — Function
assert_resume_folds(n_old::Integer, n::Integer)Refuses an extended history that adds no fold to the run that a Resume continues.
Arguments
n_old: The index of the last fold that the Result holds.n: The number of folds that the scheme enumerates over the extended history.
Validation
n > n_old. AnArgumentErroris thrown otherwise.
Related
PortfolioOptimisers.assert_resume_full_fold — Function
assert_resume_full_fold(test_idx::VecInt, pred::PredictionResult, ts)Refuses a Resume whose last held fold was partial.
A fold was partial when its test window in the longer history runs past the last row that the fold predicted. With reduce_test = true the last fold of the old run predicted the rows left at the end of its history. In the longer history those rows are the first part of a full window, with the same training end and the same weights. The fold cannot be skipped without a loss of rows, and it cannot be completed without an entry in the middle of a window, so it is terminal.
The check compares timestamps and not counts. A price-level window of L rows predicts L - 1 returns, and the last return has the timestamp of the last price, so one comparison of timestamps holds at both levels. The message names the workaround with two resumes.
Arguments
test_idx: The test window of the last held fold, over the extended history.pred: The prediction of the last held fold.ts: The timestamps of the extended history.
Validation
- The last timestamp that
predpredicted equalsts[last(test_idx)]. AnArgumentErroris thrown otherwise.
Related
PortfolioOptimisers.copy_states — Function
copy_states(est)
copy_states(td::TimeDependent)Copies every partial-fit state that an estimator tree carries, and returns the tree rebuilt around the copies.
The walk reads the fields of estimator_fields, as online_wrapper_path does, with a rebuild where that verb makes a report. It does not enter a vector, because no fold writes a state into one, and online_entry_state refuses at the entry a state that a caller put there. An estimator whose subtree holds no state comes back as it is. A TimeDependent schedule comes back unchanged, because its entries are batch configuration that the fold loop resolves per fold and threads no state through. Any value that is not an estimator comes back unchanged too.
Two callers use it. Resume calls it once at entry, so the Result it holds is never written. That copy is necessary because partial_fit! promises nothing about a kept estimator, and a buffer appends into its backing array. The generic method of partial_fit calls it before every fold. The value form therefore keeps its promise on an estimator that folds through the states of its members, such as a HighOrderPriorEstimator or a hierarchical optimiser, as well as on a leaf that carries its own cache. Every state type defines Base.copy, so one copy per state is enough.
Algorithm
- Read the
cachefield ofest, ornothingwhenesthas no such field, and copy it throughcopy_state, givingcache. - Call
copy_stateson every estimator-valued field ofest(estimator_fields), givingrepl. - Set
movedwhen an entry ofreplis not the object (!==) thatestholds in that field. - Return
estwhencacheisnothingandmovedis false. - Otherwise rebuild
estthroughrebuild_structfromrepl, and fromcachetoo when it is notnothing.
Arguments
est: The estimator, or any value that a field holds.
Returns
est: The tree, with every state copied.
Related
copy_states(p::Pipeline)Copies every partial-fit state that a Pipeline carries, and rebuilds the pipeline around the copies, for Resume.
The states are the own cache of the pipeline and the states under every step, which copy_states copies. A PipelineStep copies through copy_states(ps::PipelineStep), and a callable and a result pass through the fallback unchanged. A fold of the copy leaves the held rows of the original unchanged.
Related
copy_states(ps::PipelineStep)Copies the partial-fit states under the estimator that a PipelineStep wraps.
The method rebuilds the wrapper around the copied estimator with rebuild_struct, so the step keeps its reads, its writes and its target. A wrapped callable passes through the fallback of copy_states unchanged.
Related
PortfolioOptimisers.copy_state — Function
copy_state(state::AbstractPartialFitState)
copy_state(::Nothing)Copies the partial-fit state in a cache field, and returns nothing when the field holds none.
This is the cache arm of copy_states.
Related
PortfolioOptimisers.data_timestamps — Function
data_timestamps(rd::ReturnsResult)
data_timestamps(pr::PricesResult)Returns the timestamps of rd or of pr, or nothing when it holds none.
A PricesResult always holds them, because its TimeArray has a timestamp column.
Related
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(r::Resume)
needs_previous_weights(r::Resume)
assert_time_dependent_fold_count(r::Resume, n::Integer, all_binds::Bool = true)Answers the three traits of the fold loop for the estimator that the Result of a Resume carries.
The fold loop reads the traits off the root that it is handed. The loop resolves a Resume at entry and makes the per-fold copy of the estimator that its Result carries, so that estimator answers.
Related