Prediction CV: private API
The pipeline fold loop
cross_val_predict over a Pipeline fits the whole pipeline on each training window and predicts on each test window. It also replaces each TimeDependent schedule of the pipeline with its value for the fold, before fit runs. So no value that the pipeline passes to the optimiser is a schedule, and neither fit nor run_step reads the fold.
A scheme built by OnlineIndexWalkForward or OnlineDateWalkForward is an online scheme. With one, the loop fits the pipeline once on the first training window, adds the rows of each later fold with partial_fit!, and gets the fitted result from fit(pipe) with no data. Online(pipe) goes through the same cross_val_predict methods, and refits the whole pipeline at each fold on a buffer of the rows so far. The online pipeline page documents both.
PortfolioOptimisers.Pipeline_OnlPipe — Type
const Pipeline_OnlPipe = Union{Pipeline, Online{<:Pipeline}, Resume{<:Any, <:Pipeline}}Alias for the three estimators that the cross_val_predict methods of a Pipeline give to the fold loop. The group exists because the three share one body for each scheme, and this alias is the type that the method of each scheme dispatches on. cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::CVER) and cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::MultipleRandomised) are the two methods.
A plain pipeline takes the fold route. Online(pipe) declares a refit from a buffer of the input data. Resume(res) continues an online run whose Result holds a pipeline.
Related
Combinatorial and asset-resampling over a returns-level pipeline
A pipeline over returns runs the schemes with many paths in the same way as a plain optimiser. CombinatorialCrossValidation fits each split on its training rows, which need not be contiguous, and predicts its test groups. MultipleRandomised runs the inner walk-forward of each path on a view of the input that holds a subset of the assets. The pipeline fits again on that subset, and never takes a subset of a state it already fitted.
PortfolioOptimisers.pipeline_path_fit_and_predict — Function
pipeline_path_fit_and_predict(pipe::Pipeline_OnlPipe, data::Prices_RR, folds, path_id; ex, wd, hwd, fa, pws, store_weight_path, strict, cv) -> MultiPeriodPredictionResultRun one path of MultipleRandomised over a Pipeline on price-level or returns-level data. The path is an inner walk-forward over a subset of the assets.
Algorithm
- Read the training rows
train_idxand the test rowstest_idxof each fold fromfolds.foldsholds one(train, test, asset)tuple for each fold of the path, in the order of the split. - For fold
i, viewdataon the asset subset of the fold withport_opt_view(data, :, cols), which takes the assets third at both levels. The pipeline fits again on each subset, so it never selects from a fitted universe. - Through
fold_loop, fit the pipeline on the training rows of each fold withpipeline_fold_fit, and predict its test rows. The loop resolves each schedule for the fold, and the context holdspath_id. When the folds ofcvare a timeline and the pipelineneeds_previous_weights, the loop runs the folds of the path in sequence and gives each fold the weights of the previous fold. This gives the predictions andest, the estimator that the online arm threaded. - Sort the predictions by their test rows with
sort_predictions.
Returns
MultiPeriodPredictionResult: The predictions of the path, with theidpath_idand the estimatorest.
Related
Schedules in the steps of a pipeline
The functions below apply the schedule logic to each step of a pipeline. The check for a schedule looks through every step. So does the replacement of each schedule by its value for the fold, which also unwraps a schedule inside a PipelineStep. A run with no folds sets each schedule step to its default. After the replacement, pipeline_step_factory gives the weights of the previous fold, w_prev, to the optimisation steps.
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(p::Pipeline) -> Any
Return true when a step of the Pipeline holds a schedule.
Three steps can hold one: a TimeDependent step in the place of the optimiser, an optimisation step with a scheduled field, and a nested pipeline that holds one of the two. A nested pipeline in an Online wrapper answers as the pipeline that it wraps, and the online arm of the fold loop resolves its schedule at each fold. A PipelineStep answers for the estimator that it wraps. Every other step answers false through the fallback of is_time_dependent. A preprocessing step, a prior, a phylogeny, an uncertainty set, a constraint estimator and a callable hold no schedule. A schedule for one of these families is a field of the optimisation step.
Related
PortfolioOptimisers.assert_pipeline_step_fold_count — Function
assert_pipeline_step_fold_count(x, n::Integer, all_binds::Bool)Check the number of folds n against the schedules of one Pipeline step.
An optimisation step, a TimeDependent step and a nested pipeline, bare or in an Online wrapper, answer through assert_time_dependent_fold_count. That check sizes the entries of a schedule to the loop, and it does not read the default. A PipelineStep answers for the estimator that it wraps, and a wrapped callable passes. Every other step passes. The steps that this check reads are the steps for which is_time_dependent can answer true, so a schedule that the loop resolves at each fold is always sized.
Validation
- Everything
assert_time_dependent_fold_countrefuses.
Related
PortfolioOptimisers.update_time_dependent_estimator — Function
update_time_dependent_estimator(
p::Pipeline,
ctx::TimeDependentContext
) -> Pipeline
update_time_dependent_estimator(
p::Pipeline,
ctx::TimeDependentContext,
all_binds::Bool
) -> Pipeline
Resolve the schedules of a Pipeline for the fold that ctx describes.
The fold loop does this swap before fit runs, so fit and run_step never see a fold. When fit runs on the training window of the fold, each schedule step is an optimiser or a precomputed result. So inject_context and maybe_inject_step never receive a schedule.
Algorithm
- Resolve each step with
update_time_dependent_estimator, giving the new steps. A schedule step gives its entry for the fold, an estimator or a precomputed result. An optimisation step resolves its scheduled fields, and a nested pipeline resolves its own steps. Every other step, a callable among them, comes back unchanged through the fallback. - Return a pipeline with the same names, the new steps and the same
cache, throughrebuild_struct.
Validation
- Everything
update_time_dependent_estimatorrefuses for a step. This includes an entry of a callable schedule that is not an optimiser.
Related
PortfolioOptimisers.update_time_dependent_estimator — Function
update_time_dependent_estimator(
ps::PipelineStep,
ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
ps::PipelineStep,
ctx::TimeDependentContext,
all_binds::Bool
) -> Any
Resolve one PipelineStep of a Pipeline for the fold that ctx describes.
Algorithm
- Resolve a wrapper of a schedule to the entry of the schedule for the fold, through the
TD_OptE_Optmethod ofupdate_time_dependent_estimator, with no wrapper. The wrapper only declares the slot that dispatch cannot infer, andrun_stepdispatches on the entry. An entry can be a result, and a wrapper cannot hold one. - Resolve the estimator of any other wrapper by
update_time_dependent_estimator, givingnewest. A callable comes back unchanged. - Rebuild the wrapper around
newestwithrebuild_structwhennewestis a new value. Return the wrapper unchanged otherwise.
Validation
- Everything
update_time_dependent_estimatorrefuses for the estimator of the wrapper.
Related
PortfolioOptimisers.reset_time_dependent_step — Function
reset_time_dependent_step(est)Replace one Pipeline step with its value outside a fold.
Algorithm
- Replace a
TimeDependentstep with itsdefault, reset in turn throughreset_time_dependent_estimator. An optimisation step has a static value to fall back to, and a schedule step has none, so thedefaultis the only value outside a fold. - Reset an optimisation step and a nested pipeline through
reset_time_dependent_estimator. - Replace a
PipelineStepthat wraps a schedule with the value of step 1, with no wrapper. Return aPipelineStepthat wraps a callable unchanged. Reset the estimator of any otherPipelineStepby this algorithm, givingnewest. Rebuild the wrapper aroundnewestwhennewestis a new value. - Return every other step unchanged.
Validation
- A
TimeDependentstep has adefault. ATimeDependentDefaultErrorthat namescross_val_predictis thrown otherwise.
Related
PortfolioOptimisers.reset_time_dependent_estimator — Method
reset_time_dependent_estimator(p::Pipeline) -> Pipeline
Replace each schedule of a Pipeline with its value outside a fold, through reset_time_dependent_step.
The fold-less fit calls this function first when the pipeline is_time_dependent. A pipeline that update_time_dependent_estimator resolved for one fold holds no schedule, so a fit inside the fold loop does not call it.
Validation
- Everything
reset_time_dependent_steprefuses.
Related
PortfolioOptimisers.pipeline_step_factory — Function
pipeline_step_factory(est, w)Give the weights w of the previous fold to one Pipeline step.
Algorithm
- Give
wto an optimisation step and to a nested pipeline throughfactory. A turnover, a fee and a tracking term read the weights, and every other field passes through. - Return a
PipelineStepthat wraps a callable unchanged. Givewto the estimator of any otherPipelineStepby this algorithm, givingnewest. Rebuild the wrapper aroundnewestwhennewestis a new value. - Return every other step unchanged.
Related
The rows of a pipeline fold
A fold of returns data records its rows on its PredictionResult when every fitted step of the pipeline keeps the observations of the window. A realised factor_attribution of the cross-validation reads those rows to find the block rows of each fold. A preprocessing step states whether it keeps them with keeps_observations.
PortfolioOptimisers.fitted_step_keeps_observations — Function
fitted_step_keeps_observations(fitted, data::AbstractReturnsResult) -> BoolReturn true when one fitted step of a pipeline keeps every observation of a returns window.
It is the companion of apply_fitted_step, and it reads the same data levels. The window is always returns data, because its one caller is the returns-data predict of a PipelineResult, and no step converts returns to prices. A returns preprocessing step changes the window, so it keeps the observations exactly when keeps_observations says so. A prices preprocessing step, a PricesToReturns step, and the result of every other step, such as a prior or an optimisation result, return the window unchanged, so they keep the observations. A nested PipelineResult keeps them when each of its own steps does.
Arguments
fitted: The fitted result of one step, from aPipelineResult.data: The data window that the step receives.
Returns
flag::Bool:truewhen the window that the step returns holds the observations ofdata, in the same order.
Related
PortfolioOptimisers.apply_fitted_steps_keeping — Function
apply_fitted_steps_keeping(results::Union{Tuple, AbstractVector}, data::AbstractReturnsResult) -> (data′, kept::Bool)Apply the fitted steps of a pipeline to a data window, in step order, and say whether the window kept its observations.
It is apply_fitted_steps with one more answer. The returns-data method of predict(res::PipelineResult, data::AbstractReturnsResult, window) reads kept, and the fold records its rows when it is true.
Algorithm
- Set
kepttotrue. - For each fitted result
finresults, in step order, setkepttofalsewhenfitted_step_keeps_observationsisfalseforfand the currentdata. Then replacedatawithapply_fitted_step(f, data). - Return the last
dataandkept.
Arguments
results: The fitted result of each step, from aPipelineResult.data: The data window.
Returns
data′: The changed data window, asapply_fitted_stepsreturns it.kept::Bool:truewhen every step keeps the observations of the window.
Related
PortfolioOptimisers.pipeline_fold_prediction — Function
pipeline_fold_prediction(pred::PredictionResult, test_idx::Colon, kept::Bool)
pipeline_fold_prediction(pred::PredictionResult, test_idx::VecInt, kept::Bool)Record the rows of a pipeline fold on its prediction, when the fitted steps kept them.
The weight-level predict of the optimisation result receives the window after the fitted steps, so it cannot know the rows of the window and records idx = nothing. When every step kept the observations, the rows of the window are test_idx, and the method records them. A realised factor_attribution of the cross-validation then finds the block rows of each fold by position. A whole-sample prediction, test_idx = :, records no rows, as the whole-sample predict of an optimisation result does.
Arguments
pred: The prediction on the changed window.test_idx: The rows of the input data in the window, or:.kept: Whether the fitted steps kept every observation of the window, fromapply_fitted_steps_keeping.
Validation
- When
keptistrue, the prediction has one observation for each entry oftest_idx. Raises aDimensionMismatch, because a step answeredkeeps_observationswithtrueand changed the number of observations.
Returns
pred::PredictionResult:predwithidx = test_idxwhenkeptistrue, andpreditself otherwise.
Related