Validation
PortfolioOptimisers.cross_val_predict — Function
cross_val_predict(opt, rd::ReturnsResult, cv::CVER = KFold(); cols = :, ex = FLoops.ThreadedEx())Run cross-validated portfolio optimisation and return predictions over all folds.
Accepts either an optimisation estimator or an optimisation result. When cols is provided, restricts the optimisation to that subset of assets. Parallel fold execution is controlled by ex.
Arguments
opt: Optimisation estimator or result.rd::ReturnsResult: Returns data used for fitting and prediction.cv::CVER: Cross-validation scheme. Defaults toKFold().cols: Column selector. Defaults to:(all assets).ex: FLoops executor controlling parallelism. Defaults toFLoops.ThreadedEx().
Returns
- Cross-validation prediction result.
Related
cross_val_predict(r::OptimiserResume, rd::ReturnsResult, cv::CVER; cols = :, ex = FLoops.ThreadedEx())
fit_and_predict(r::OptimiserResume, rd::ReturnsResult, cv::CVER; cols = :, ex = FLoops.ThreadedEx(), id = nothing)Continue an online walk-forward from a Result, over the full history extended.
The doors that take an optimiser in the estimator slot take a Resume too. The carrier is viewed by cols as the one-shot door views it, and the estimator is not: the Result's estimator was threaded over the old run's view already, so the pinned context refuses a different one at the first delta step. The scheme is checked at the door (assert_resume_scheme) and the fold loop takes its resumed arm. The Result returned holds the new folds only, its opt folded through the last training end, and id.
Arguments
r: The declaration, holding the Result to continue.rd: The returns result to use.cv: The scheme the Result came from, a walk-forward withff = OnlineStep().cols: The asset view of the carrier.ex: Unread; the resumed arm runs in order.id: The identifier the Result carries.
Validation
- Everything
assert_resume_schemeand the resumed arm ofonline_foldsrefuse.
Returns
res::MultiPeriodPredictionResult: The new folds, carrying the threaded estimator.
Related
cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::CombinatorialCrossValidation; ex = FLoops.ThreadedEx(), kwargs...) -> PopulationPredictionResultRun combinatorial cross-validation over a price- or returns-level Pipeline.
Each split fits the whole workflow on its (possibly non-contiguous) training rows and predicts each of the split's disjoint test groups; sort_predictions! then recombines the per-split test-group predictions into the scheme's paths, exactly like the plain-optimiser combinatorial loop. Time-dependent steps resolve per split against the fold's TimeDependentContext before fit.
At the returns level the training rows are order-independent for moment-style fitted steps, so this is exact. At the price level a split's training rows are non-contiguous — there are gaps where the held-out test groups sit — so the fold's rolling transform (PricesToReturns) produces one spurious return per gap boundary (a boundary return spanning a gap). That is the rolling-window approximation: combinatorial paths at the price level cost a few boundary returns in each fold's training window. Test groups are contiguous, so predictions are unaffected. Use MultipleRandomised if you need contiguous training rows at the price level.
Related
cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::MultipleRandomised; ex = FLoops.ThreadedEx(), kwargs...) -> PopulationPredictionResultRun asset-resampling (multiple-randomised) cross-validation over a price- or returns-level Pipeline.
Each resampled path is an inner walk-forward over a random asset subset; the subset is applied to the input data (an asset view via pipeline_asset_view), and the pipeline is fitted fresh on the sub-universe — so the pipeline never needs to sub-select its fitted universe. Paths are run by pipeline_path_fit_and_predict and returned as a PopulationPredictionResult. Asset resampling keeps every observation window contiguous (it draws over assets, not rows), so — unlike combinatorial cross-validation — multiple-randomised is admissible at the price level too: a price-starting pipeline fits fresh on the asset-subset prices per fold, with no rolling-window violation.
Related
cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::CVER = KFold(); ex = FLoops.ThreadedEx(), id = nothing)Run cross-validated prediction over an entire Pipeline workflow and return a MultiPeriodPredictionResult.
Return type by cross-validation scheme
cross_val_predict is the single entry point for every scheme, but the result type depends on cv: single-path schemes return one series, multi-path schemes return a per-path collection. The type flips with the scheme, so result-navigation code written against one shape (e.g. a KFold run) breaks when the scheme is swapped — branch on the scheme, not on the run.
cv scheme | Return type | .pred holds |
|---|---|---|
KFold / walk-forward (CVER) | MultiPeriodPredictionResult | one series — one prediction per fold |
CombinatorialCrossValidation | PopulationPredictionResult | a per-path collection |
MultipleRandomised | PopulationPredictionResult | a per-path collection |
The combinatorial and asset-resampling schemes are dispatched by their own methods (see Related); the rest of this docstring describes the contiguous, single-path (CVER) method.
The input is split at its own level — price-level data by the prices-aware split methods (contiguous windows, so stateful preprocessing stays inside the fold), returns-level data as usual — and for each fold the whole workflow is fitted on the training window and predicts on the test window, exactly as fit/predict do for a holdout. This method covers the contiguous, single-path schemes (KFold and the walk-forwards). Combinatorial and asset-resampling schemes have their own methods for a returns-level pipeline (see cross_val_predict(pipe::Pipeline, data::AbstractReturnsResult, cv::CombinatorialCrossValidation) and cross_val_predict(pipe::Pipeline, data::AbstractReturnsResult, cv::MultipleRandomised)); for a price-starting pipeline they are rejected at split by the rolling-window rule.
This is the fold loop that consumes TimeDependent schedules in a pipeline: when the pipeline is time-dependent, fold i builds a TimeDependentContext — with rd the raw, pre-preprocessing input data, so pipeline-level callables see the fold's data before any step has transformed it — and swaps every schedule for its fold-i value via update_time_dependent_estimator before fit runs. A schedule step may resolve to an estimator (the fold optimises) or a precomputed result (the fold predicts only); injection never sees a schedule. The loop is fold_loop, shared with the optimiser-level schemes. The scheme states whether its folds are a timeline through folds_are_time_ordered. A walk-forward answers true, so a pipeline that needs_previous_weights runs sequentially and threads the previous fold's weights into the context's w_prev and, post-swap, into the optimisation steps via factory. A KFold answers false, because its folds are independent of each other. Its folds run in parallel, w_prev is nothing, and no factory pass runs — the same behaviour the optimiser-level KFold path already has.
A walk-forward that declares a Fold Fit (ff = OnlineStep()) sends the loop down its online arm: the pipeline is warmed up once on the first training window, each fold's new rows are folded through its steps into the row owner by partial_fit!, and the fold reads the pipeline out through fit(pipe) where a refit would have run, through pipeline_fold_fit. The run reaches the weights of the batch expanding walk-forward fold for fold, and Online(pipe) takes the same door as the declared refit from an input-carrier buffer.
Arguments
pipe: The pipeline.data: Price- or returns-level input data (Prices_RR).cv::CVER: Cross-validation scheme with contiguous, non-combinatorial folds. Defaults toKFold().folds_are_time_ordereddecides whether its folds thread the previous fold's weights, andfold_fitwhether the folds refit or fold.ex: FLoops executor controlling parallelism. Defaults toFLoops.ThreadedEx().id: Identifier stored on the result.
Returns
MultiPeriodPredictionResult: One prediction per fold, in fold order.
Related
PipelinefitTimeDependentfolds_are_time_orderedsearch_cross_validationMultiPeriodPredictionResult/PopulationPredictionResult(the two return shapes)cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::CombinatorialCrossValidation)cross_val_predict(pipe::Pipeline, data::Prices_RR, cv::MultipleRandomised)
cross_val_predict(o::Online{<:Pipeline}, data::Prices_RR, cv::CVER; ex = FLoops.ThreadedEx(), id = nothing)
cross_val_predict(o::Online{<:Pipeline}, data::Prices_RR, cv::MultipleRandomised; ex = FLoops.ThreadedEx(), kwargs...)Run a walk-forward over Online(pipe), the declared refit of a Pipeline from an input-carrier buffer.
The same doors as the pipeline's, with the wrapper threaded through the fold loop: the online arm resolves it at warm-up into a pipeline carrying a PipelineBufferState, every fold appends its new rows to the buffer, and the read-out is fit(pipe, buffer). So the run is exact for every configuration at batch cost — the window-valued steps the host route refuses included — and with max_history = w it equals the rolling batch walk-forward with warm-up w + purged_size. A scheme with no Fold Fit is refused by name, because the wrapper resolves only at the online arm's warm-up.
Related
cross_val_predict(r::PipelineResume, data::Prices_RR, cv::CVER; ex = FLoops.ThreadedEx(), id = nothing)Continue an online walk-forward over a Pipeline from its Result, over the full history extended.
The pipeline door of Resume: the scheme is checked (assert_resume_scheme), the holdout refused as the one-shot door refuses it, and the fold loop takes its resumed arm through pipeline_cross_val_predict. A host route and an Online(pipe) refit route resume alike, because the entry reads the state off the pipeline generically.
Related