Search scorers
PortfolioOptimisers.HighestMeanScore — Type
struct HighestMeanScore{__T_scale} <: CrossValidationSearchScorerA CrossValidationSearchScorer that selects the parameter set with the highest mean score across cross-validation splits.
When called with a score matrix (rows = CV splits, columns = parameter sets), it returns the column index with the largest mean score. Its candidate_scores are the column means, so a search orders its candidates by the mean score. A tie goes to the first column, in the index form and in the order.
The scale is the weight of the scorer in a vector of search scorers, the same word and mechanism as settings.scale of a risk measure. A negative scale is a negative weight. The scale is inert for one scorer, so neither form above reads it.
Fields
scale: Weight of this search scorer in a vector of search scorers, read throughscorer_scale. A negative scale is a negative weight. It is inert for one scorer.
Constructors
HighestMeanScore(; scale::Number = 1.0) -> HighestMeanScoreKeyword arguments correspond to the fields above.
Validation
scaleis finite. Otherwise anIsNonFiniteError.
Examples
julia> scorer = PortfolioOptimisers.HighestMeanScore();julia> scores = [0.5 0.8; 0.6 0.7];julia> scorer(scores)2julia> PortfolioOptimisers.candidate_scores(scorer, scores)2-element Vector{Float64}: 0.55 0.75Related
PortfolioOptimisers.CrossValidationSearchScorer — Type
abstract type CrossValidationSearchScorer <: AbstractCrossValidationScorerDefines the interface for scoring strategies used in search cross-validation. Implementations select the optimal parameter set based on cross-validation scores.
It is the branch of AbstractCrossValidationScorer that selects a candidate of a search. Its sibling PredictionScorer selects a path of a cross-validation.
Interfaces
candidate_scores(::CrossValidationSearchScorer, scores::AbstractMatrix): Returns one value per column ofscores, bigger is better. The search orders the candidates by this value, so it can keep then_bestbest ones, and it can combine the values of a vector of scorers throughs_sca. A subtype that defines no method givesnothing, and the search then calls the index form below and keeps one candidate.(::CrossValidationSearchScorer)(scores::AbstractMatrix): Returns the index of the optimal parameter set, the column with the best (highest) aggregate score. It must equalargmaxofcandidate_scoreswhen the subtype defines both.scorer_scale(::CrossValidationSearchScorer): Optional. Returns the weight of the scorer in a vector of scorers. The fallback gives1, so a subtype with ascalefield defines this method to read it.
Arguments
scores: Matrix of scores, where each column corresponds to a parameter set and each row to a cross-validation split. The matrix is orientation-normalised: risk-measure scores are negated so that higher is always better, whatever measureris. A scorer therefore returns the index of the column with the best (highest) aggregate score.
Returns
candidate_scores:VecNum, one value per column of the matrix the scorer received, in the order of the columns, and no valueNaN. The search refuses a vector of another length with aDimensionMismatch, and aNaNwith anIsNonFiniteError.- The index form:
Int, the index of the optimal parameter set, as a position in the matrix the scorer received.
A scorer never sees a failed candidate
The search, through finite_candidate_order, hands the scorer the columns whose every entry is finite, and maps the order back to the parameter grid itself. A scorer therefore reads a matrix of finite numbers alone, and it may compute anything on it, such as a mean, a spread or a rank, and a candidate that failed a fold cannot win. The matrix a scorer receives is a view of the score matrix, and its column count can be smaller than the grid. So a scorer must return values or a position in that view, and must not index the grid itself.
Examples
julia> struct MyScore <: PortfolioOptimisers.CrossValidationSearchScorer endjulia> function PortfolioOptimisers.candidate_scores(::MyScore, X::AbstractMatrix) return vec(minimum(X; dims = 1)) endjulia> (s::MyScore)(X::AbstractMatrix) = argmax(PortfolioOptimisers.candidate_scores(s, X))julia> scores = [0.5 0.6; 0.7 0.8];julia> scorer = MyScore()MyScore()julia> PortfolioOptimisers.candidate_scores(scorer, scores)2-element Vector{Float64}: 0.5 0.6julia> scorer(scores)2Related
PortfolioOptimisers.candidate_scores — Function
candidate_scores(scorer::CrossValSearchScorer, X)
candidate_scores(scorer::HighestMeanScore, X::MatNum)Give one value per candidate of a search, bigger is better, so that the search can order the candidates.
The column j of X holds the scores of candidate j over the rows of the scheme, signed so that a bigger score is better. The search sorts the candidates by these values, in descending order, and keeps the first n_best. The sort is stable, so a tie keeps the order of the grid.
The method on CrossValSearchScorer is the fallback. It gives nothing, which states that the scorer gives no value through this method. A function scorer, and a subtype of CrossValidationSearchScorer that defines no method, give nothing. The search then calls the scorer once, through candidate_values, and reads the form from the return. A function can give a vector of values or an index, and the index form of a subtype gives an index. The search keeps one candidate for an index, and it refuses n_best > 1 with an ArgumentError.
Arguments
scorer: The search scorer.X: Therows × candidatesscore matrix that the scorer receives. Every entry is finite.
Returns
vals::Option{<:VecNum}: One value per column ofX, ornothingwhen the scorer gives no value per candidate.
Examples
julia> PortfolioOptimisers.candidate_scores(HighestMeanScore(), [0.5 0.75 0.25; 1.0 0.5 0.25])3-element Vector{Float64}: 0.75 0.625 0.25Related
PortfolioOptimisers.scorer_scale — Function
scorer_scale(scorer::CrossValSearchScorer)
scorer_scale(scorer::HighestMeanScore)Give the weight of a search scorer in a vector of search scorers.
candidate_values multiplies the values of each member of a vector by this weight before s_sca reduces them. It is the same word and mechanism as settings.scale of a risk measure, which oriented_member_score reads. A negative scale is a negative weight. The scale is inert for one scorer.
The method on CrossValSearchScorer is the fallback, and it gives 1. So a function, and a subtype of CrossValidationSearchScorer with no scale field, have the weight one. A subtype with a scale field defines a method that reads it. A function writes its weight into its own values.
Arguments
scorer: The search scorer.
Returns
scale::Number: The weight of the scorer.
Examples
julia> PortfolioOptimisers.scorer_scale(HighestMeanScore(; scale = 2.0))2.0julia> PortfolioOptimisers.scorer_scale(X -> vec(sum(X; dims = 1)))1Related