Search cross-validation: private API
PortfolioOptimisers.cv_data_eltype — Function
cv_data_eltype(rd::AbstractReturnsResult)
cv_data_eltype(pr::AbstractPricesResult)Return the element type of the score matrices of a Pipeline search over the data rd or pr.
The type comes from the data. A ReturnsResult and a PricesResult hold floating-point values, so the type holds the NaN of a failed fold.
Arguments
rd: The returns data, anAbstractReturnsResult.pr: The price data, anAbstractPricesResult.
Returns
T::Type: The element type ofrd.X, or of the values of the time arraypr.X.
Related
PortfolioOptimisers.is_pipeline_raw_path — Function
is_pipeline_raw_path(key::AbstractString) -> BoolReturn true when key is a property path whose root is the steps field of a Pipeline, such as "steps[1]" or "steps[2].fill".
pipeline_lens calls this function on a key whose first segment is not a step name. A Pipeline has two fields, names and steps, and only steps holds the estimators that a grid tunes. So the function admits the one root steps. It returns false for the path "names[1]", and for a typo such as "gapfill".
Algorithm
- Find
i, the position of the first.or[inkey. - Return
falsewhenkeyholds neither character. - Return
truewhen the text before positioniissteps, andfalseotherwise.
Arguments
key: The tuning key, as the grid gives it.
Returns
flag::Bool:truewhen the root ofkeyissteps.
Related
PortfolioOptimisers.pipeline_lens — Function
pipeline_lens(pipe::Pipeline, key::AbstractString)
pipeline_lens(pipe::Pipeline, key::Symbol)
pipeline_lens(pipe::Pipeline, key::Integer)
pipeline_lens(pipe::Pipeline, key::Expr)
pipeline_lens(pipe::Pipeline, key)Return the Accessors.jl lens on pipe that a tuning key of a Pipeline search names.
A key names a step by its name or by its position, or it gives a property path whose root is the steps field. A key that names a step addresses the whole step, so a grid value can replace a whole estimator. A String key can add a property path after the step name, such as "gap_fill.fill".
The function refuses every other key. A lens with a different root sets a field of the Pipeline itself, and that tunes no step. The typo "gapfill" and the path "names[1]", which addresses the table of step names, are two such keys.
Algorithm
For a String key:
- Split
keyat its first., givingparts. - Find
i, the position ofparts[1]inpipe.names. - When
parts[1]is not a step name, refusekeyunlessis_pipeline_raw_pathadmits it, and return the lens ofkeyfromparse_lens. - Build
step, the lens on entryiofpipe.steps. - When
keyholds no., returnstep. Otherwise return the lens ofparts[2]fromparse_lens, composed afterstep.
For a Symbol key:
- Find
i, the position ofkeyinpipe.names. - Refuse
keywhen it is not a step name, and return the lens on entryiofpipe.stepsotherwise.
For an Integer key, check that key is a step position, and return the lens on entry key of pipe.steps.
For an Expr key:
- Build
lensthroughparse_lens. - Find the root of
key, the first argument of the expression at each level down to aSymbol. - Refuse
keywhen its root is notsteps, and returnlensotherwise.
For a lens that the caller builds, return key through parse_lens unchanged.
Arguments
pipe: The pipeline that the search tunes.key: The tuning key, aGSCVKey.
Validation
- A
Stringkey starts with a step name, oris_pipeline_raw_pathadmits it. Otherwise anArgumentErrornames the key and suggests the closest step name. - A
Symbolkey is a step name. Julia reads a dottedSymbolas one name, so the function refuses it, and theArgumentErrorasks for aStringpath. - An
Integerkey is in1:length(pipe.steps). - The root of an
Exprkey issteps. - A key that reaches
parse_lenspasses its caps on length and depth.
Returns
lens: The Accessors.jl lens onpipe.
Related
PortfolioOptimisers.pipeline_lens_val_grid — Function
pipeline_lens_val_grid(pipe::Pipeline,
estval::AbstractVector{<:Pair{<:Any, <:AbstractVector}})
pipeline_lens_val_grid(pipe::Pipeline, estval::AbstractDict{<:Any, <:AbstractVector})
pipeline_lens_val_grid(pipe::Pipeline,
estvals::AbstractVector{<:Union{AbstractVector{<:Pair{<:Any, <:AbstractVector}},
AbstractDict{<:Any, <:AbstractVector}}})Build the grid of a Pipeline search, the lenses and the values of every candidate.
The grid is the grid of lens_val_grid. The difference is the lenses: pipeline_lens resolves each key on pipe, so a key can name a step by its name or by its position.
Mathematical definition
\[\begin{align} \Theta &= V_1 \times V_2 \times \cdots \times V_k\,, \\ |\Theta| &= \prod_{j=1}^{k} |V_j|\,, \\ \Theta &= \Theta^{(1)} \Vert \Theta^{(2)} \Vert \cdots \Vert \Theta^{(m)}\,, \\ |\Theta| &= \sum_{l=1}^{m} |\Theta^{(l)}|\,. \end{align}\]
Where:
- $\Theta$: Search grid, the sequence of grid points that
lens_val_gridbuilds fromp. For aPipeline,pipeline_lens_val_gridbuilds it. - $V_j$: Candidate values of the $j$-th key of one parameter set, the value vector that the set gives the key.
- $k$: Key count of one parameter set.
- $\Theta^{(l)}$: Grid of the $l$-th parameter set, by the first two equations.
- $m$: Parameter set count, the count of the parameter sets that the search concatenates.
- $\Vert$: Concatenation of two sequences.
The first two equations hold for one parameter set, and the last two for a vector of sets.
Algorithm
For one parameter set, estval:
- Check the count of the grid through
assert_search_grid_cap, before the function builds the product. - Collect the product of the value vectors into
vals, one tuple per grid point. - Resolve each key into a lens on
pipethroughpipeline_lens, and repeat that vector of lenses once per entry ofvals, givinglenses.
For a vector of parameter sets, estvals:
- Build the grid of each set by the steps above, giving
lenses_vals. - Concatenate the lenses and the values of the sets in the order of
estvals, givinglensesandvals. - Check
length(vals)throughassert_search_grid_cap.
Arguments
pipe: The pipeline that the search tunes.estval: One parameter set, a vector ofkey => valuespairs or a dictionary from key to values. A key is a tuning key thatpipeline_lensreads, such as"gap_fill.fill".estvals: A vector of parameter sets, each of the form ofestval.
Validation
- Every value vector holds at least one value, through
assert_search_grid_cap. - The grid holds at most
RESOURCE_LIMITS[].max_search_gridcandidates, throughassert_search_grid_cap. The function checks the count of a set before it builds the product of that set. Underestvalsit checks each set as it builds that set, and the sum of the sets after the concatenation. - Every key names a step or a path rooted at
steps, throughpipeline_lens.
Returns
lenses::AbstractVector: One vector of lenses per grid point.vals::AbstractVector{<:Tuple}: One tuple of values per grid point, in the order of the lenses at that point. Inside one set the first key varies fastest, and a dictionary gives its keys in its own order of iteration.
Related
lens_val_grid: the grid of a search over an optimiser.pipeline_lensassert_search_grid_capsearch_cross_validation
PortfolioOptimisers.assert_search_candidates — Method
assert_search_candidates(pipe::Pipeline, lens_grid, val_grid) -> NothingRefuse a Pipeline search in which a candidate holds a TrainTestSplit, before the search scores any candidate.
A lens can write a TrainTestSplit into a step, so the function checks each candidate and not pipe alone. It also checks pipe when the grid leaves the split in place. The check runs before the candidates run in parallel, so the caller gets the ArgumentError and not a TaskFailedException that holds it.
Algorithm
For each grid point, build the candidate through search_candidate, and check it through assert_no_holdout.
Arguments
pipe: The pipeline that the search tunes.lens_grid: The lenses of each grid point.val_grid: The values of each grid point.
Validation
- No candidate holds a
TrainTestSplit, throughassert_no_holdout.
Returns
nothing.
Related