Prediction CV: private API

The pipeline fold loop

cross_val_predict over a Pipeline fits the whole pipeline on each training window and predicts on each test window. It also replaces each TimeDependent schedule of the pipeline with its value for the fold, before fit runs. So no value that the pipeline passes to the optimiser is a schedule, and neither fit nor run_step reads the fold.

A scheme built by OnlineIndexWalkForward or OnlineDateWalkForward is an online scheme. With one, the loop fits the pipeline once on the first training window, adds the rows of each later fold with partial_fit!, and gets the fitted result from fit(pipe) with no data. Online(pipe) goes through the same cross_val_predict methods, and refits the whole pipeline at each fold on a buffer of the rows so far. The online pipeline page documents both.

PortfolioOptimisers.Pipeline_OnlPipe — Type
const Pipeline_OnlPipe = Union{Pipeline, Online{<:Pipeline}, Resume{<:Any, <:Pipeline}}

Alias for the three estimators that the cross_val_predict methods of a Pipeline give to the fold loop. The group exists because the three share one body for each scheme, and this alias is the type that the method of each scheme dispatches on. cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::CVER) and cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::MultipleRandomised) are the two methods.

A plain pipeline takes the fold route. Online(pipe) declares a refit from a buffer of the input data. Resume(res) continues an online run whose Result holds a pipeline.

Related

source

Combinatorial and asset-resampling over a returns-level pipeline

A pipeline over returns runs the schemes with many paths in the same way as a plain optimiser. CombinatorialCrossValidation fits each split on its training rows, which need not be contiguous, and predicts its test groups. MultipleRandomised runs the inner walk-forward of each path on a view of the input that holds a subset of the assets. The pipeline fits again on that subset, and never takes a subset of a state it already fitted.

PortfolioOptimisers.pipeline_path_fit_and_predict — Function
pipeline_path_fit_and_predict(pipe::Pipeline_OnlPipe, data::Prices_RR, folds, path_id; ex, wd, hwd, fa, pws, store_weight_path, strict, cv) -> MultiPeriodPredictionResult

Run one path of MultipleRandomised over a Pipeline on price-level or returns-level data. The path is an inner walk-forward over a subset of the assets.

Algorithm

  1. Read the training rows train_idx and the test rows test_idx of each fold from folds. folds holds one (train, test, asset) tuple for each fold of the path, in the order of the split.
  2. For fold i, view data on the asset subset of the fold with port_opt_view(data, :, cols), which takes the assets third at both levels. The pipeline fits again on each subset, so it never selects from a fitted universe.
  3. Through fold_loop, fit the pipeline on the training rows of each fold with pipeline_fold_fit, and predict its test rows. The loop resolves each schedule for the fold, and the context holds path_id. When the folds of cv are a timeline and the pipeline needs_previous_weights, the loop runs the folds of the path in sequence and gives each fold the weights of the previous fold. This gives the predictions and est, the estimator that the online arm threaded.
  4. Sort the predictions by their test rows with sort_predictions.

Returns

Related

source

Schedules in the steps of a pipeline

The functions below apply the schedule logic to each step of a pipeline. The check for a schedule looks through every step. So does the replacement of each schedule by its value for the fold, which also unwraps a schedule inside a PipelineStep. A run with no folds sets each schedule step to its default. After the replacement, pipeline_step_factory gives the weights of the previous fold, w_prev, to the optimisation steps.

PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(p::Pipeline) -> Any

Return true when a step of the Pipeline holds a schedule.

Three steps can hold one: a TimeDependent step in the place of the optimiser, an optimisation step with a scheduled field, and a nested pipeline that holds one of the two. A nested pipeline in an Online wrapper answers as the pipeline that it wraps, and the online arm of the fold loop resolves its schedule at each fold. A PipelineStep answers for the estimator that it wraps. Every other step answers false through the fallback of is_time_dependent. A preprocessing step, a prior, a phylogeny, an uncertainty set, a constraint estimator and a callable hold no schedule. A schedule for one of these families is a field of the optimisation step.

Related

source
PortfolioOptimisers.assert_pipeline_step_fold_count — Function
assert_pipeline_step_fold_count(x, n::Integer, all_binds::Bool)

Check the number of folds n against the schedules of one Pipeline step.

An optimisation step, a TimeDependent step and a nested pipeline, bare or in an Online wrapper, answer through assert_time_dependent_fold_count. That check sizes the entries of a schedule to the loop, and it does not read the default. A PipelineStep answers for the estimator that it wraps, and a wrapped callable passes. Every other step passes. The steps that this check reads are the steps for which is_time_dependent can answer true, so a schedule that the loop resolves at each fold is always sized.

Validation

Related

source
PortfolioOptimisers.update_time_dependent_estimator — Function
update_time_dependent_estimator(
    p::Pipeline,
    ctx::TimeDependentContext
) -> Pipeline
update_time_dependent_estimator(
    p::Pipeline,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Pipeline

Resolve the schedules of a Pipeline for the fold that ctx describes.

The fold loop does this swap before fit runs, so fit and run_step never see a fold. When fit runs on the training window of the fold, each schedule step is an optimiser or a precomputed result. So inject_context and maybe_inject_step never receive a schedule.

Algorithm

  1. Resolve each step with update_time_dependent_estimator, giving the new steps. A schedule step gives its entry for the fold, an estimator or a precomputed result. An optimisation step resolves its scheduled fields, and a nested pipeline resolves its own steps. Every other step, a callable among them, comes back unchanged through the fallback.
  2. Return a pipeline with the same names, the new steps and the same cache, through rebuild_struct.

Validation

Related

source
PortfolioOptimisers.update_time_dependent_estimator — Function
update_time_dependent_estimator(
    ps::PipelineStep,
    ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
    ps::PipelineStep,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Resolve one PipelineStep of a Pipeline for the fold that ctx describes.

Algorithm

  1. Resolve a wrapper of a schedule to the entry of the schedule for the fold, through the TD_OptE_Opt method of update_time_dependent_estimator, with no wrapper. The wrapper only declares the slot that dispatch cannot infer, and run_step dispatches on the entry. An entry can be a result, and a wrapper cannot hold one.
  2. Resolve the estimator of any other wrapper by update_time_dependent_estimator, giving newest. A callable comes back unchanged.
  3. Rebuild the wrapper around newest with rebuild_struct when newest is a new value. Return the wrapper unchanged otherwise.

Validation

Related

source
PortfolioOptimisers.reset_time_dependent_step — Function
reset_time_dependent_step(est)

Replace one Pipeline step with its value outside a fold.

Algorithm

  1. Replace a TimeDependent step with its default, reset in turn through reset_time_dependent_estimator. An optimisation step has a static value to fall back to, and a schedule step has none, so the default is the only value outside a fold.
  2. Reset an optimisation step and a nested pipeline through reset_time_dependent_estimator.
  3. Replace a PipelineStep that wraps a schedule with the value of step 1, with no wrapper. Return a PipelineStep that wraps a callable unchanged. Reset the estimator of any other PipelineStep by this algorithm, giving newest. Rebuild the wrapper around newest when newest is a new value.
  4. Return every other step unchanged.

Validation

Related

source
PortfolioOptimisers.reset_time_dependent_estimator — Method
reset_time_dependent_estimator(p::Pipeline) -> Pipeline

Replace each schedule of a Pipeline with its value outside a fold, through reset_time_dependent_step.

The fold-less fit calls this function first when the pipeline is_time_dependent. A pipeline that update_time_dependent_estimator resolved for one fold holds no schedule, so a fit inside the fold loop does not call it.

Validation

Related

source
PortfolioOptimisers.pipeline_step_factory — Function
pipeline_step_factory(est, w)

Give the weights w of the previous fold to one Pipeline step.

Algorithm

  1. Give w to an optimisation step and to a nested pipeline through factory. A turnover, a fee and a tracking term read the weights, and every other field passes through.
  2. Return a PipelineStep that wraps a callable unchanged. Give w to the estimator of any other PipelineStep by this algorithm, giving newest. Rebuild the wrapper around newest when newest is a new value.
  3. Return every other step unchanged.

Related

source

The rows of a pipeline fold

A fold of returns data records its rows on its PredictionResult when every fitted step of the pipeline keeps the observations of the window. A realised factor_attribution of the cross-validation reads those rows to find the block rows of each fold. A preprocessing step states whether it keeps them with keeps_observations.

PortfolioOptimisers.fitted_step_keeps_observations — Function
fitted_step_keeps_observations(fitted, data::AbstractReturnsResult) -> Bool

Return true when one fitted step of a pipeline keeps every observation of a returns window.

It is the companion of apply_fitted_step, and it reads the same data levels. The window is always returns data, because its one caller is the returns-data predict of a PipelineResult, and no step converts returns to prices. A returns preprocessing step changes the window, so it keeps the observations exactly when keeps_observations says so. A prices preprocessing step, a PricesToReturns step, and the result of every other step, such as a prior or an optimisation result, return the window unchanged, so they keep the observations. A nested PipelineResult keeps them when each of its own steps does.

Arguments

  • fitted: The fitted result of one step, from a PipelineResult.
  • data: The data window that the step receives.

Returns

  • flag::Bool: true when the window that the step returns holds the observations of data, in the same order.

Related

source
PortfolioOptimisers.apply_fitted_steps_keeping — Function
apply_fitted_steps_keeping(results::Union{Tuple, AbstractVector}, data::AbstractReturnsResult) -> (data′, kept::Bool)

Apply the fitted steps of a pipeline to a data window, in step order, and say whether the window kept its observations.

It is apply_fitted_steps with one more answer. The returns-data method of predict(res::PipelineResult, data::AbstractReturnsResult, window) reads kept, and the fold records its rows when it is true.

Algorithm

  1. Set kept to true.
  2. For each fitted result f in results, in step order, set kept to false when fitted_step_keeps_observations is false for f and the current data. Then replace data with apply_fitted_step(f, data).
  3. Return the last data and kept.

Arguments

  • results: The fitted result of each step, from a PipelineResult.
  • data: The data window.

Returns

  • data′: The changed data window, as apply_fitted_steps returns it.
  • kept::Bool: true when every step keeps the observations of the window.

Related

source
PortfolioOptimisers.pipeline_fold_prediction — Function
pipeline_fold_prediction(pred::PredictionResult, test_idx::Colon, kept::Bool)
pipeline_fold_prediction(pred::PredictionResult, test_idx::VecInt, kept::Bool)

Record the rows of a pipeline fold on its prediction, when the fitted steps kept them.

The weight-level predict of the optimisation result receives the window after the fitted steps, so it cannot know the rows of the window and records idx = nothing. When every step kept the observations, the rows of the window are test_idx, and the method records them. A realised factor_attribution of the cross-validation then finds the block rows of each fold by position. A whole-sample prediction, test_idx = :, records no rows, as the whole-sample predict of an optimisation result does.

Arguments

  • pred: The prediction on the changed window.
  • test_idx: The rows of the input data in the window, or :.
  • kept: Whether the fitted steps kept every observation of the window, from apply_fitted_steps_keeping.

Validation

  • When kept is true, the prediction has one observation for each entry of test_idx. Raises a DimensionMismatch, because a step answered keeps_observations with true and changed the number of observations.

Returns

  • pred::PredictionResult: pred with idx = test_idx when kept is true, and pred itself otherwise.

Related

source