Train/test split
Train/test splitting
A holdout split keeps the last observations as a test window and trains on the observations before them. It has two forms. train_test_split cuts data into a training part and a test part. TrainTestSplit, alias TTS, is an estimator that does the same as the first step of a Pipeline. Every fitted step after it then sees the training window only, and fit_predict(pipe, data) scores the pipeline on the test window.
A size is a number of rows, an Integer, or a fraction of the observations, any other Real in (0, 1). If you give one size, the other window takes the remaining rows. If you give both, the rows between the two windows go to neither, which is an embargo.
Called with keywords, train_test_split returns a (train, test) tuple. Called with a TrainTestSplit, as train_test_split(tts, data), it returns the TrainTestSplitResult that the split step of a pipeline makes. You can use one TrainTestSplit both inside and outside a pipeline.
Types
PortfolioOptimisers.TrainTestSplit — Type
struct TrainTestSplit{__T_train_size, __T_test_size} <: AbstractPreprocessingEstimatorPreprocessing estimator that holds out the last observations as a test window.
It is the estimator form of train_test_split, and it adds a holdout to a Pipeline. It must be the first step of the pipeline. It passes the training window to every later step, and it keeps the test window in its fitted TrainTestSplitResult. fit_predict(pipe, data) then predicts on the test window, which no fitted step has seen.
It is the one preprocessing estimator that takes both data levels. It splits the price or returns data that the pipeline input holds, because a holdout selects rows and reads no column.
A fitted split passes new data through unchanged. Its rows index the fitting data alone, so predict(res, future_data) predicts on every new observation.
A pipeline that holds a TrainTestSplit cannot also be cross-validated. Cross-validation makes its own train and test windows, so a split inside each fold removes a second test window from every training window. cross_val_predict and search_cross_validation throw an ArgumentError for such a pipeline.
Fields
train_size: The training observations, as aCountOfor aFractionOf.nothingtakes the complement oftest_size.
test_size: The test observations, as aCountOfor aFractionOf.nothingtakes the complement oftrain_size.
Constructors
TrainTestSplit(; train_size::Option{<:Real_CountFraction} = nothing, test_size::Option{<:Real_CountFraction} = nothing,) -> TrainTestSplitKeywords correspond to the struct's fields. A size is a CountOf of rows or a FractionOf of the observations, and the constructor reads a plain number through count_or_fraction. One size makes the other window its complement, two sizes embargo the rows between the windows, and no size gives a 75/25 split. safe_index checks the windows when the split runs on data.
Validation
- Each plain size passes
count_or_fraction.
Examples
julia> pipe = Pipeline(; steps = (TrainTestSplit(; test_size = 0.2), PricesToReturns(), EmpiricalPrior(), EqualWeighted()));julia> pipe.names("split", "returns", "prior", "opt")Related
PortfolioOptimisers.TrainTestSplitResult — Type
struct TrainTestSplitResult{__T_train, __T_test} <: AbstractResultFitted holdout split that holds the training window and the test window.
It is the result of a TrainTestSplit. The test window is the data that the fitted pipeline has not seen, and fit_predict(pipe, data) predicts on it. The result also keeps the train window, so a caller can read the training data from the result without a second split.
Both windows are port_opt_views of the input, at the data level of the input.
Fields
train: The training window. It is the first rows of the observations, and every later step fits on it.
test: The test window. It is the last rows of the observations, and no fitted step has seen it.
Related
Functions
PortfolioOptimisers.train_test_split — Function
train_test_split(rd::ReturnsResult; train_size, test_size) -> (train, test)
train_test_split(pr::PricesResult; train_size, test_size) -> (train, test)Cut price or returns data into a training window of the first rows and a test window of the last rows.
This is the function form of TrainTestSplit. port_opt_view makes each window. At the returns level each window holds views of the input arrays, so the split copies no data. At the price level port_opt_view indexes a TimeArray by date, and that copies the rows. safe_index states the sizing rules.
Algorithm
- Read the observation count
N, assize(rd.X, 1)at the returns level and assize(TimeSeries.values(pr.X), 1)at the price level. - Resolve the two row ranges
trainandtestwithsafe_index. - Return a
port_opt_viewof each range. The returns-level method gives:as the asset index after the row range, because the three-argument form ofport_opt_viewtakes the observations first and the assets second.
Arguments
rd,pr: The data to split.train_size: The training rows, as aCountOf, aFractionOfor a plain number thatcount_or_fractionreads.nothingtakes the complement oftest_size.test_size: The test rows, in the same forms.nothingtakes the complement oftrain_size. If neither size is given, the split is 75/25.
Validation
- The two sizes pass the checks of
safe_indexforNrows.
Returns
(train, test): The two windows. Each is aReturnsResultfor aReturnsResultinput, and aPricesResultfor aPricesResultinput.
Examples
julia> rd = ReturnsResult(; nx = ["A"], X = reshape(collect(0.1:0.1:1.0), 10, 1));julia> train, test = train_test_split(rd; test_size = 0.2);julia> size(train.X, 1), size(test.X, 1)(8, 2)Related
train_test_split(
tts::TrainTestSplit,
data::Union{AbstractPricesResult, AbstractReturnsResult}
) -> TrainTestSplitResult
Split data under a TrainTestSplit, and return both windows as a TrainTestSplitResult.
This is the estimator form of the keyword method. train_test_split(rd; test_size = 0.2) returns a (train, test) tuple, and this method returns the result that the split step of a Pipeline makes. So one TrainTestSplit gives the same windows inside and outside a pipeline.
Algorithm
- Call the keyword method of
train_test_splitwithtts.train_sizeandtts.test_size, givingtrainandtest. - Return
TrainTestSplitResult(train, test).
Validation
- The sizes of
ttspass the checks ofsafe_indexfor the rows ofdata.
Related