Train/test split
Train/test splitting
A holdout split reserves the tail of the time-ordered observations as a test window and trains on the head. It comes in two forms: the free function train_test_split, which cuts data into a train/test pair, and the estimator TrainTestSplit (alias TTS), which carries the protocol inside a Pipeline as its first step — so every fitted step downstream sees the training window alone, and fit_predict(pipe, data) evaluates on the held-out window in one line.
Sizes are row counts (Integer) or fractions of the observations (AbstractFloat in (0, 1)). Giving one side makes the other its complement; giving both embargoes the rows between the two windows. See docs/adr/0031-holdout-split-as-a-pipeline-step.md.
The keyword form returns a bare (train, test) tuple; the estimator form, train_test_split(tts, data), returns the same TrainTestSplitResult a pipeline's split step produces, so one configured holdout can be reused inside and outside a pipeline.
Types
PortfolioOptimisers.TrainTestSplit — Type
struct TrainTestSplit{__T_train_size, __T_test_size} <: AbstractPreprocessingEstimatorPreprocessing estimator reserving the tail of the observations as a held-out test window.
The estimator form of train_test_split, and the way the holdout protocol enters a Pipeline: as the first step, it hands the training window to every step downstream and stashes the test window in its fitted TrainTestSplitResult. fit_predict(pipe, data) then evaluates the fitted workflow on that held-out window in one line.
It is the one preprocessing estimator that is not pinned to a data level: it splits whichever level the pipeline input provides, price or returns, since a holdout is a statement about rows, not about columns or units.
Replaying a fitted split on an unseen window is a pass-through — the fitted rows are training-window state, and applying them to new data would be meaningless — so predict(res, future_data) keeps working on genuinely new observations.
A pipeline containing a TrainTestSplit may not also be cross-validated: the split and the cross-validator are two evaluation protocols, and cross-validation already defines its own train/test windows. search_cross_validation rejects such a pipeline rather than silently shaving a second holdout off every fold.
Fields
train_size: Training observations as a count (Integer) or a fraction (AbstractFloatin(0, 1));nothingtakes the complement oftest_size.
test_size: Test observations, likewise;nothingtakes the complement oftrain_size.
Constructors
TrainTestSplit(; train_size::Option{<:Number} = nothing, test_size::Option{<:Number} = nothing,) -> TrainTestSplitKeywords correspond to the struct's fields. Sizes follow safe_index: a row count (Integer) or a fraction of the observations (AbstractFloat in (0, 1)); one side given makes the other its complement; both given embargoes the rows between them; neither given splits 75/25.
Examples
julia> pipe = Pipeline(; steps = (TrainTestSplit(; test_size = 0.2), PricesToReturns(), EmpiricalPrior(), EqualWeighted()));julia> pipe.names("split", "returns", "prior", "opt")Related
PortfolioOptimisers.TrainTestSplitResult — Type
struct TrainTestSplitResult{__T_train, __T_test} <: AbstractResultFitted result of a TrainTestSplit, carrying both windows of the holdout.
The test window is the payoff: it is the data the fitted pipeline has never seen, and what fit_predict(pipe, data) predicts on. The train window is kept alongside it so the raw data the workflow was fitted on is retrievable from the result rather than having to be re-derived.
Both are port_opt_views of the input at whichever level the split ran (price or returns).
Fields
train: The training window: the head of the observations, and the data every downstream step is fitted on.
test: The held-out test window: the tail of the observations, which no fitted step has seen.
Related
Functions
PortfolioOptimisers.train_test_split — Function
train_test_split(rd::ReturnsResult; train_size, test_size) -> (train, test)
train_test_split(pr::PricesResult; train_size, test_size) -> (train, test)Cut price- or returns-level data into a training window (the head) and a held-out test window (the tail).
The free-function form of TrainTestSplit; the windows are port_opt_views, so no data is copied. See safe_index for the sizing rules — complement when one side is given, embargo when both are.
Algorithm
- Read the observation count
Nfrom the asset data:size(rd.X, 1)at returns level, andsize(TimeSeries.values(pr.X), 1)at price level. - Resolve the two row ranges with
safe_index. - Return a
port_opt_viewof each range. The returns-level method passes aColonasset index after the row range, because that arity indexes observations first and assets second.
Arguments
rd/pr: The data to split.train_size: Training rows as a count (Integer) or a fraction (AbstractFloatin(0, 1));nothingtakes the complement oftest_size.test_size: Test rows, likewise;nothingtakes the complement oftrain_size. With neither given the split is 75/25.
Returns
(train, test): The two windows, of the same type as the input.
Examples
julia> rd = ReturnsResult(; nx = ["A"], X = reshape(collect(0.1:0.1:1.0), 10, 1));julia> train, test = train_test_split(rd; test_size = 0.2);julia> size(train.X, 1), size(test.X, 1)(8, 2)Related
train_test_split(
tts::TrainTestSplit,
data::Union{AbstractPricesResult, AbstractReturnsResult}
) -> TrainTestSplitResult
Split data under a TrainTestSplit, returning both windows as a TrainTestSplitResult.
The estimator-form counterpart of the keyword form: train_test_split(rd; test_size = 0.2) hands back a bare (train, test) tuple, while this hands back the same fitted result a pipeline's split step produces, so a holdout configured once can be reused verbatim inside and outside a Pipeline.
Algorithm
- Call the keyword form of
train_test_splitwithtts.train_sizeandtts.test_size, giving the two windows. - Wrap the pair in a
TrainTestSplitResult, in the order(train, test).
Related