Grid search cross validation

struct GridSearchCrossValidation{__T_p, __T_cv, __T_r, __T_scorer, __T_ex, __T_train_score, __T_kwargs} <: AbstractSearchCrossValidationEstimator

Performs grid search cross-validation for portfolio optimisation estimators. Iterates over parameter grids, applies cross-validation splits, and scores each configuration to select the optimal parameters.

Fields

• p: Hyperparameter search grid.

• cv: Cross-validation estimator.

• r: Risk measure or vector of risk measures.

• scorer: Scoring function. Given the orientation-normalised score matrix (rows = CV splits, columns = parameter sets), it returns the column index of the best parameter set. The matrix is normalised so that higher is always better, whatever the risk measure, so a scorer selects the largest aggregate score (see CrossValidationSearchScorer).

• ex: Parallel execution strategy.

• train_score: Whether to also compute the training set score.

• kwargs: Additional keyword arguments.

Constructors

GridSearchCrossValidation(    p::MultiGSCVValType_VecMultiGSCVValType;    cv::CrossValidationEstimator = KFold(),    r::AbstractBaseRiskMeasure = ConditionalValueatRisk(),    scorer::CrossValSearchScorer = HighestMeanScore(),    ex::FLoops.Transducers.Executor = FLoops.ThreadedEx(),    train_score::Bool = false,    kwargs::NamedTuple = (;),) -> GridSearchCrossValidation

Positional and keyword arguments correspond to fields above.

Validation

• !isempty(p).
• If p is a vector of parameter sets: each element must not be empty.
• All keys in p must be of type GSCVKey (i.e. String, Symbol, or Integer).

Examples

julia> GridSearchCrossValidation(Dict("alpha" => [0.1, 0.2], "beta" => [1.0, 2.0]))GridSearchCrossValidation            p ┼ Dict{String, Vector{Float64}}: Dict("alpha" => [0.1, 0.2], "beta" => [1.0, 2.0])           cv ┼ KFold              │                   n ┼ Int64: 5              │         purged_size ┼ Int64: 0              │        embargo_size ┼ Int64: 0              │                  wd ┼ nothing              │                  fa ┼ nothing              │   store_weight_path ┼ Bool: false              │              strict ┴ Bool: false            r ┼ ConditionalValueatRisk              │   settings ┼ RiskMeasureSettings              │            │   scale ┼ Float64: 1.0              │            │      ub ┼ nothing              │            │     rke ┴ Bool: true              │      alpha ┼ Float64: 0.05              │          w ┴ nothing       scorer ┼ HighestMeanScore()           ex ┼ Transducers.ThreadedEx{@NamedTuple{}}: Transducers.ThreadedEx()  train_score ┼ Bool: false       kwargs ┴ @NamedTuple{}: NamedTuple()

Related

• MultiGSCVValType_VecMultiGSCVValType
• AbstractBaseRiskMeasure
• CrossValSearchScorer
• search_cross_validation

References

• [119] J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, 281–305 (2012). Section 2.

search_cross_validation(opt::NonFiniteAllocationOptimisationEstimator,
                       gscv::GridSearchCrossValidation,
                       rd::ReturnsResult)

Performs grid search cross-validation for portfolio optimisation estimators. Iterates over parameter grids, scores each configuration through the one fold loop over the search's scheme, and selects the optimal parameters using the provided scoring strategy.

Arguments

• opt: Portfolio optimisation estimator to be tuned.
• gscv: Grid search cross-validation estimator specifying parameter grid, CV splitter, risk measure, scorer, execution strategy, and options.
• rd: Returns result containing asset returns data.

Returns

• SearchCrossValidationResult: Result type containing the optimal estimator, test and train scores, parameter grid, and selected index.

Details

• Refuses an estimator carrying a partial-fit state once, before any candidate is built, through assert_search_entry: the search tunes the configuration alone, and under an OnlineStep the whole entry check of the online arm runs at the door.
• Fixes the folds once through pin_draw, so a scheme whose split draws at random scores every candidate over the same folds.
• Iterates over all parameter combinations in the grid, in parallel over gscv.ex.
• Scores each candidate through fit_and_predict(opt_i, rd, gscv.cv; ex = SequentialEx()), the one fold loop every cross-validation entry point runs, so the candidate runs the scheme it declared: a walk-forward threads the previous fold's weights through the scheme's pws, and resolves a TimeDependent schedule per fold, exactly as fit_and_predict(opt, rd, cv) does. The folds inside a candidate run in sequence.
• Under an OnlineStep, every candidate warms up cold on the first training window and steps through the folds, so the online search reads the matrix of the batch expanding search to the tolerance of the moment layer and of the solver, and picks the same column. Nothing is shared between candidates and nothing is reset.
• Writes one row per fold, in split's enumeration order, through write_candidate_scores!. Under a MultipleRandomised the loop returns one series per path, and score_rows lays each path's scores back onto its split rows.
• Selects the optimal parameter set based on cross-validation scores, through finite_candidate_index: the scorer is handed the candidates that finished every fold, and a candidate that failed one can never win. The result keeps the raw score matrix, so its columns line up with the grid and a reader sees which fold failed. A failed step holds NaN at that row, the column never reaches the scorer, and every later step of the candidate still runs and scores.

Related

• finite_candidate_index
• assert_search_entry
• pin_draw
• score_rows
• write_candidate_scores!
• fit_and_predict
• NonFiniteAllocationOptimisationEstimator
• RandomisedSearchCrossValidation
• ReturnsResult
• GridSearchCrossValidation

search_cross_validation(opt::NonFiniteAllocationOptimisationEstimator,
                        gscv::GridSearchCrossValidation{<:Any, <:CombinatorialCrossValidation},
                        rd::ReturnsResult)

Grid search cross-validation over a CombinatorialCrossValidation scheme.

Unlike the contiguous schemes (which score one fold per row), combinatorial cross-validation recombines its disjoint test groups into full-length backtest paths. Scoring a single split in isolation would mix groups belonging to different paths, so this method scores per-path instead: for each candidate the whole scheme is run through fit_and_predict (splits fitted, groups recombined by sort_predictions! into a PopulationPredictionResult), and expected_risk yields one score per path. The score matrix is therefore n_paths × n_candidates; the scorer selects across candidates exactly as for the other schemes, through finite_candidate_index, so a candidate that failed a path can never win. The randomised form delegates here through its grid.

train_scores (only when gscv.train_score) keeps every per-fold in-sample score rather than collapsing to one number per path: it is a Vector of n_paths matrices, one per path, each folds_in_path × n_candidates. (Test scores stay one-per-path because a path's out-of-sample returns pool into a single series, whereas its folds train on distinct in-sample windows.)

Related

• CombinatorialCrossValidation
• finite_candidate_index
• fit_and_predict
• expected_risk
• search_cross_validation