Train/test split: private API
PortfolioOptimisers.safe_index — Function
safe_index(
lo::Union{Nothing, Real, AbstractCountOrFraction},
hi::Union{Nothing, Real, AbstractCountOrFraction},
N::Integer
) -> Tuple{Any, Any}
safe_index(
lo::Union{Nothing, Real, AbstractCountOrFraction},
hi::Union{Nothing, Real, AbstractCountOrFraction},
N::Integer,
D::Union{Real, AbstractCountOrFraction}
) -> Tuple{Any, Any}
Return the (train, test) observation ranges of a holdout split over N time-ordered rows.
The training window is the first rows of the data and the test window is the last rows, so the test window is always the most recent one. Each size is a row count or a fraction of the observations, and split_count resolves it to a count.
- If neither size is given, the training window takes the fraction
Dof the rows, 75 % by default, and the test window takes the rest. - If one size is given, the other window takes the remaining rows. The two windows then partition the data.
- If both sizes are given, the training window takes the first
lorows and the test window takes the lasthirows. The rows between the two windows belong to neither window. This gap is an embargo, and only the two sizes set it. The cross-validatorsKFoldandCombinatorialCrossValidationtake a gap around each test fold aspurged_sizeandembargo_size.
Algorithm
Read a plain
loorhithroughcount_or_fraction. Then resolve the two window lengthsN_landN_h, through the branch thatloandhiselect.- Neither is given. Resolve
Dwithsplit_count, givingn. ThenN_l = nandN_h = N - n. - Only
lois given. Resolve it withsplit_count, givingn. ThenN_l = nandN_h = N - n. - Only
hiis given. Resolve it withsplit_count, givingn. ThenN_l = N - nandN_h = n. - Both are given. Resolve each with
split_counton its own. Neither length is the complement of the other, so the rows between the two windows belong to neither window.
- Neither is given. Resolve
Check that both windows are non-empty, and that
N_l + N_h <= N.Return the two ranges
1:N_land(N - N_h + 1):N. The embargoed rows lie between them.
Arguments
lo: The training rows, as aCountOf, aFractionOfor a plain number thatcount_or_fractionreads.nothingtakes the complement ofhi.hi: The test rows, in the same forms.nothingtakes the complement oflo.N: The number of observations.D = FractionOf(0.75): The training size when neither size is given.
Validation
- Each plain size passes
count_or_fraction, which throws aDomainErrorotherwise. - Both windows are non-empty. Otherwise the function throws an
ArgumentError. A size that takes every row, such astrain_size = N, leaves no row for the other window. N_l + N_h <= N, so that the windows do not overlap. Otherwise the function throws anArgumentError.
Returns
(train, test): The training and test row ranges, as twoUnitRange{Int}s.
Related
PortfolioOptimisers.split_count — Function
split_count(s::CountOf, N::Integer) -> Int64
Resolve one side of a train/test split into a row count.
A size is a CountOf of observations or a FractionOf of the observations. A count larger than N takes all N rows. The window checks of safe_index then reject a split that leaves either side empty.
Algorithm
The method that Julia selects is the first step, because a count and a fraction mean different things.
sis aCountOfof rows. Returnmin(Int(s.val), N).sis aFractionOfof the rows, withf = s.val. Roundf * Nto the nearest integer, givingn.- If
n / N, computed in the type off, exceedsf, subtract one fromn. Nownis the largest count whose share of the rows does not exceedf, which isfloor(f * N)in exact arithmetic. A directfloor(Int, f * N)loses a row when the product lands below a whole number, and0.29 * 100is28.999999999999996. - Return
clamp(n, 1, N). The clamp stops a small fraction of a short window from resolving to zero rows.
Arguments
s: One side of the split, as aCountOfof rows or aFractionOfof the observations.N: The number of observations.
Returns
n::Int: The number of rows that the side takes, in1:N.
Related