Search cross-validation: private API

PortfolioOptimisers.cv_data_eltype — Function
cv_data_eltype(rd::AbstractReturnsResult)
cv_data_eltype(pr::AbstractPricesResult)

Return the element type of the score matrices of a Pipeline search over the data rd or pr.

The type comes from the data. A ReturnsResult and a PricesResult hold floating-point values, so the type holds the NaN of a failed fold.

Arguments

Returns

  • T::Type: The element type of rd.X, or of the values of the time array pr.X.

Related

source
PortfolioOptimisers.is_pipeline_raw_path — Function
is_pipeline_raw_path(key::AbstractString) -> Bool

Return true when key is a property path whose root is the steps field of a Pipeline, such as "steps[1]" or "steps[2].fill".

pipeline_lens calls this function on a key whose first segment is not a step name. A Pipeline has two fields, names and steps, and only steps holds the estimators that a grid tunes. So the function admits the one root steps. It returns false for the path "names[1]", and for a typo such as "gapfill".

Algorithm

  1. Find i, the position of the first . or [ in key.
  2. Return false when key holds neither character.
  3. Return true when the text before position i is steps, and false otherwise.

Arguments

  • key: The tuning key, as the grid gives it.

Returns

  • flag::Bool: true when the root of key is steps.

Related

source
PortfolioOptimisers.pipeline_lens — Function
pipeline_lens(pipe::Pipeline, key::AbstractString)
pipeline_lens(pipe::Pipeline, key::Symbol)
pipeline_lens(pipe::Pipeline, key::Integer)
pipeline_lens(pipe::Pipeline, key::Expr)
pipeline_lens(pipe::Pipeline, key)

Return the Accessors.jl lens on pipe that a tuning key of a Pipeline search names.

A key names a step by its name or by its position, or it gives a property path whose root is the steps field. A key that names a step addresses the whole step, so a grid value can replace a whole estimator. A String key can add a property path after the step name, such as "gap_fill.fill".

The function refuses every other key. A lens with a different root sets a field of the Pipeline itself, and that tunes no step. The typo "gapfill" and the path "names[1]", which addresses the table of step names, are two such keys.

Algorithm

For a String key:

  1. Split key at its first ., giving parts.
  2. Find i, the position of parts[1] in pipe.names.
  3. When parts[1] is not a step name, refuse key unless is_pipeline_raw_path admits it, and return the lens of key from parse_lens.
  4. Build step, the lens on entry i of pipe.steps.
  5. When key holds no ., return step. Otherwise return the lens of parts[2] from parse_lens, composed after step.

For a Symbol key:

  1. Find i, the position of key in pipe.names.
  2. Refuse key when it is not a step name, and return the lens on entry i of pipe.steps otherwise.

For an Integer key, check that key is a step position, and return the lens on entry key of pipe.steps.

For an Expr key:

  1. Build lens through parse_lens.
  2. Find the root of key, the first argument of the expression at each level down to a Symbol.
  3. Refuse key when its root is not steps, and return lens otherwise.

For a lens that the caller builds, return key through parse_lens unchanged.

Arguments

  • pipe: The pipeline that the search tunes.
  • key: The tuning key, a GSCVKey.

Validation

  • A String key starts with a step name, or is_pipeline_raw_path admits it. Otherwise an ArgumentError names the key and suggests the closest step name.
  • A Symbol key is a step name. Julia reads a dotted Symbol as one name, so the function refuses it, and the ArgumentError asks for a String path.
  • An Integer key is in 1:length(pipe.steps).
  • The root of an Expr key is steps.
  • A key that reaches parse_lens passes its caps on length and depth.

Returns

  • lens: The Accessors.jl lens on pipe.

Related

source
PortfolioOptimisers.pipeline_lens_val_grid — Function
pipeline_lens_val_grid(pipe::Pipeline,
                       estval::AbstractVector{<:Pair{<:Any, <:AbstractVector}})
pipeline_lens_val_grid(pipe::Pipeline, estval::AbstractDict{<:Any, <:AbstractVector})
pipeline_lens_val_grid(pipe::Pipeline,
                       estvals::AbstractVector{<:Union{AbstractVector{<:Pair{<:Any, <:AbstractVector}},
                                                       AbstractDict{<:Any, <:AbstractVector}}})

Build the grid of a Pipeline search, the lenses and the values of every candidate.

The grid is the grid of lens_val_grid. The difference is the lenses: pipeline_lens resolves each key on pipe, so a key can name a step by its name or by its position.

Mathematical definition

\[\begin{align} \Theta &= V_1 \times V_2 \times \cdots \times V_k\,, \\ |\Theta| &= \prod_{j=1}^{k} |V_j|\,, \\ \Theta &= \Theta^{(1)} \Vert \Theta^{(2)} \Vert \cdots \Vert \Theta^{(m)}\,, \\ |\Theta| &= \sum_{l=1}^{m} |\Theta^{(l)}|\,. \end{align}\]

Where:

  • $\Theta$: Search grid, the sequence of grid points that lens_val_grid builds from p. For a Pipeline, pipeline_lens_val_grid builds it.
  • $V_j$: Candidate values of the $j$-th key of one parameter set, the value vector that the set gives the key.
  • $k$: Key count of one parameter set.
  • $\Theta^{(l)}$: Grid of the $l$-th parameter set, by the first two equations.
  • $m$: Parameter set count, the count of the parameter sets that the search concatenates.
  • $\Vert$: Concatenation of two sequences.

The first two equations hold for one parameter set, and the last two for a vector of sets.

Algorithm

For one parameter set, estval:

  1. Check the count of the grid through assert_search_grid_cap, before the function builds the product.
  2. Collect the product of the value vectors into vals, one tuple per grid point.
  3. Resolve each key into a lens on pipe through pipeline_lens, and repeat that vector of lenses once per entry of vals, giving lenses.

For a vector of parameter sets, estvals:

  1. Build the grid of each set by the steps above, giving lenses_vals.
  2. Concatenate the lenses and the values of the sets in the order of estvals, giving lenses and vals.
  3. Check length(vals) through assert_search_grid_cap.

Arguments

  • pipe: The pipeline that the search tunes.
  • estval: One parameter set, a vector of key => values pairs or a dictionary from key to values. A key is a tuning key that pipeline_lens reads, such as "gap_fill.fill".
  • estvals: A vector of parameter sets, each of the form of estval.

Validation

  • Every value vector holds at least one value, through assert_search_grid_cap.
  • The grid holds at most RESOURCE_LIMITS[].max_search_grid candidates, through assert_search_grid_cap. The function checks the count of a set before it builds the product of that set. Under estvals it checks each set as it builds that set, and the sum of the sets after the concatenation.
  • Every key names a step or a path rooted at steps, through pipeline_lens.

Returns

  • lenses::AbstractVector: One vector of lenses per grid point.
  • vals::AbstractVector{<:Tuple}: One tuple of values per grid point, in the order of the lenses at that point. Inside one set the first key varies fastest, and a dictionary gives its keys in its own order of iteration.

Related

source
PortfolioOptimisers.assert_search_candidates — Method
assert_search_candidates(pipe::Pipeline, lens_grid, val_grid) -> Nothing

Refuse a Pipeline search in which a candidate holds a TrainTestSplit, before the search scores any candidate.

A lens can write a TrainTestSplit into a step, so the function checks each candidate and not pipe alone. It also checks pipe when the grid leaves the split in place. The check runs before the candidates run in parallel, so the caller gets the ArgumentError and not a TaskFailedException that holds it.

Algorithm

For each grid point, build the candidate through search_candidate, and check it through assert_no_holdout.

Arguments

  • pipe: The pipeline that the search tunes.
  • lens_grid: The lenses of each grid point.
  • val_grid: The values of each grid point.

Validation

Returns

  • nothing.

Related

source