Asset selection
Asset selectors narrow the universe from the data: drop constant columns, keep the best or worst assets by a risk measure, prune redundant ones. They are ordinary returns-preprocessing estimators — they know nothing about pipelines, and a Pipeline drives them through fit_preprocessing and apply_preprocessing like any other step.
The universe a selector chooses on the training window is its fitted state. Applying the fitted result to an unseen window replays that universe rather than re-deciding it, which is what makes a selector safe inside cross-validation.
See docs/adr/0029-asset-selection-is-returns-preprocessing.md for the design rationale, and PortfolioOptimisers.AbstractAssetSelector for the seam every selector shares.
Scoring assets with a risk measure
A ScoreSelector scores each asset by evaluating a risk measure on that asset's own return series, then hands the scores to a rule. Any risk measure whose supports_precomputed_returns is true may be used, which covers the quantile and drawdown families, the moment measures, and MeanReturn. bigger_is_better tells the ordinal rules which end of the ordering is "best".
Two measures are notable exceptions. Variance and StandardDeviation are WeightsInput measures: their functors consume portfolio weights, not a return series, so they cannot score a single asset and are rejected at construction. Use SCM(), which computes the same quantity from a return series — ZeroVarianceFilter spells this for you.
PortfolioOptimisers.ScoreSelector Type
struct ScoreSelector{__T_score, __T_rule} <: AbstractAssetSelectorAsset selector that scores every asset with a risk measure and keeps the assets a rule admits.
score is any AbstractBaseRiskMeasure that can be evaluated on a bare return series — asset i's score is score(X[:, i]). That reuses the whole risk-measure family: ConditionalValueatRisk and the drawdown measures score risk, SCM() scores variance, MeanReturn scores mean return. bigger_is_better tells the ordinal rules which end is "best".
rule decides what to do with the scores: an absolute band (ThresholdRule) or a count/fraction taken from the tails (RankRule, QuantileRule).
The selected universe is fitted state, so a ScoreSelector is safe inside cross-validation: assets are chosen on the training window and the same universe is replayed on test windows.
Fields
score: Risk measure scoring each asset's return series (AbstractBaseRiskMeasure).rule: Rule mapping the scores to a keep-mask (AbstractSelectionRule).
Constructors
ScoreSelector(;
score::AbstractBaseRiskMeasure,
rule::AbstractSelectionRule,
) -> ScoreSelectorKeywords correspond to the struct's fields.
Validation
supports_precomputed_returns(score).VarianceandStandardDeviationareWeightsInputmeasures and are rejected with a pointer toSCM().
Examples
julia> rd = ReturnsResult(; nx = ["A", "B", "C"], X = [0.1 0.0 -0.2; -0.1 0.0 0.3; 0.2 0.0 -0.1]);
julia> sel = ScoreSelector(; score = SCM(), rule = ThresholdRule(; lo = 1e-12));
julia> PortfolioOptimisers.fit_preprocessing(sel, rd).nx
2-element Vector{String}:
"A"
"C"Related
sourcePortfolioOptimisers.CompleteAssetSelector Type
struct CompleteAssetSelector <: AbstractAssetSelectorAsset selector that drops every asset column holding a missing or NaN observation.
The returns-level counterpart of MissingDataFilter's column threshold, for pipelines fed returns data directly (where the price stages never run). It has no observation-dropping mode: a fitted selector cannot decide which rows of an unseen window to drop without breaking the weights/returns alignment.
Constructors
CompleteAssetSelector() -> CompleteAssetSelectorExamples
julia> rd = ReturnsResult(; nx = ["A", "B"], X = [0.1 0.2; 0.3 NaN]);
julia> PortfolioOptimisers.fit_preprocessing(CompleteAssetSelector(), rd).nx
1-element Vector{String}:
"A"Related
sourceSelection rules
A rule turns per-asset scores into a keep-mask. ThresholdRule is literal — it compares raw scores against absolute bounds and ignores orientation, because a zero-variance filter must drop the low-variance assets. RankRule and QuantileRule are ordinal — they consult bigger_is_better and take counts (or fractions) from each tail.
Ties at a rank cut are excluded entirely, so an ordinal rule may return fewer assets than asked. If the 20th and 21st assets score equally, RankRule(; best = 20) keeps 19: the tied block is dropped rather than split arbitrarily. This is the library's "if we cannot tell them apart, trust neither" tie policy.
PortfolioOptimisers.AbstractSelectionRule Type
abstract type AbstractSelectionRule <: AbstractAlgorithmAbstract supertype for the rules that turn per-asset scores into a keep-mask.
A selection rule is an AbstractAlgorithm: it is consumed through ScoreSelector and never used on its own. Rules split into two kinds.
Literal:
ThresholdRulecompares a score against absolute bounds and ignoresbigger_is_better. A threshold on a variance means what it says; reinterpreting it as "keep the better ones" would invert the intent of a zero-variance filter.Ordinal:
RankRuleandQuantileRulesort assets from best to worst — consultingbigger_is_better, so:bestis lowest risk for a risk measure and highest value for a return measure — and take counts or fractions from each tail.
Related
sourcePortfolioOptimisers.ThresholdRule Type
struct ThresholdRule{__T_lo, __T_hi} <: AbstractSelectionRuleKeep assets whose score falls strictly inside the band (lo, hi).
Both bounds are optional and literal: lo and hi are compared against the raw score, never reinterpreted through bigger_is_better. Omitting a bound leaves that side unbounded.
Fields
lo: Exclusive lower bound on the score;nothingleaves the lower side unbounded.hi: Exclusive upper bound on the score;nothingleaves the upper side unbounded.
Constructors
ThresholdRule(;
lo::Option{<:Number} = nothing,
hi::Option{<:Number} = nothing,
) -> ThresholdRuleKeywords correspond to the struct's fields.
Validation
At least one of
lo,hiis notnothing.If both are given,
lo < hi.
Examples
julia> ThresholdRule(; lo = 1e-12) # drop (near-)constant assets
ThresholdRule
lo ┼ Float64: 1.0e-12
hi ┴ nothingRelated
sourcePortfolioOptimisers.RankRule Type
struct RankRule{__T_best, __T_worst, __T_action} <: AbstractSelectionRuleTake best and/or worst assets from the tails of the score ordering, then keep or drop them.
best and worst are counts taken from each end, not positions: best = 20 means twenty assets, not rank twenty. Which end is "best" comes from bigger_is_better on the score, so RankRule(; best = 20) keeps the twenty lowest-risk assets for a risk measure and the twenty highest-return assets for MeanReturn. Giving both takes both tails. action = :drop complements the whole selection, which is how "drop the five worst" is said without knowing the universe size.
Counts saturate at the number of assets: best = 50 on a 30-asset window keeps all 30 rather than throwing, so a hyperparameter search over best is never killed by its largest point.
Warning
Ties at the cut are excluded entirely, so a rule may return fewer assets than asked. If the 20th and 21st assets have equal scores, RankRule(; best = 20) keeps 19 — the tied block is dropped rather than split arbitrarily. This is the library's "if we cannot tell them apart, trust neither" tie policy, shared with find_uncorrelated_indices, which removes both assets of an exactly-tied correlated pair. A window whose scores are all equal therefore selects nothing, and fit_preprocessing throws.
Fields
best: Number of assets to take from the best end;nothingtakes none.worst: Number of assets to take from the worst end;nothingtakes none.action::keepretains the taken assets,:dropretains everything else.
Constructors
RankRule(;
best::Option{<:Integer} = nothing,
worst::Option{<:Integer} = nothing,
action::Symbol = :keep,
) -> RankRuleKeywords correspond to the struct's fields.
Validation
At least one of
best,worstis notnothing.Any given count is
>= 0, and at least one is> 0.action in (:keep, :drop).
Examples
julia> RankRule(; worst = 5, action = :drop) # drop the five worst
RankRule
best ┼ nothing
worst ┼ Int64: 5
action ┴ Symbol: :dropRelated
sourcePortfolioOptimisers.QuantileRule Type
struct QuantileRule{__T_best, __T_worst, __T_action} <: AbstractSelectionRuleRankRule with the tail sizes given as fractions of the asset universe.
best and worst are fractions in (0, 1), converted to counts as round(Int, fraction * n) on the window being fitted. Everything else — orientation via bigger_is_better, the action complement, count saturation, and the tie policy that excludes a straddling tied block — is identical to RankRule.
Fractions and counts are separate types on purpose: best = 1 (one asset) and best = 1.0 (the whole universe) would otherwise differ only by a literal's type.
Fields
best: Fraction of the universe to take from the best end;nothingtakes none.worst: Fraction of the universe to take from the worst end;nothingtakes none.action::keepretains the taken assets,:dropretains everything else.
Constructors
QuantileRule(;
best::Option{<:Real} = nothing,
worst::Option{<:Real} = nothing,
action::Symbol = :keep,
) -> QuantileRuleKeywords correspond to the struct's fields.
Validation
At least one of
best,worstis notnothing.Any given fraction lies in
(0, 1).action in (:keep, :drop).
Examples
julia> QuantileRule(; worst = 0.1, action = :drop) # drop the worst decile
QuantileRule
best ┼ nothing
worst ┼ Float64: 0.1
action ┴ Symbol: :dropRelated
sourceDiscarding redundant assets
A RedundancySelector discards assets that duplicate information already carried by others. Its alg decides what "redundant" means, and its score decides which member of a redundancy group survives.
PairwiseCorrelation is greedy: it drops one asset at a time until no surviving pair exceeds the threshold, and never chains. CorrelationComponents reads the same correlations transitively, treating a chain A ~ B ~ C as one blob and keeping a single representative — a stronger reduction, and a different answer on the same input. ClusterGroups partitions with clusterise and keeps one representative per cluster.
Leaving score as nothing falls back to the correlation algorithms' own survivor rule: the asset with the lowest summary correlation to the rest of the universe. ClusterGroups has no such fallback and requires a score.
PortfolioOptimisers.RedundancySelector Type
struct RedundancySelector{__T_alg, __T_score} <: AbstractAssetSelectorAsset selector that discards assets which duplicate information already carried by others.
alg decides what "redundant" means and returns the keep-mask: greedy pairwise correlation pruning (PairwiseCorrelation), one representative per correlated blob (CorrelationComponents), or one representative per cluster (ClusterGroups).
score decides which asset survives a redundancy group — a risk measure evaluated on each asset's own return series, oriented by bigger_is_better, exactly as in ScoreSelector. Leaving it nothing falls back to the correlation algorithms' own rule: the asset with the lowest summary correlation to the rest of the universe survives. ClusterGroups has no such fallback and requires a score.
Fields
alg: Algorithm deciding which assets are redundant (AbstractRedundancyAlgorithm).score: Risk measure choosing the survivor of each redundancy group;nothinguses the algorithm's own rule (AbstractBaseRiskMeasure).
Constructors
RedundancySelector(;
alg::AbstractRedundancyAlgorithm = PairwiseCorrelation(),
score::Option{<:AbstractBaseRiskMeasure} = nothing,
) -> RedundancySelectorKeywords correspond to the struct's fields.
Validation
If
scoreis given,supports_precomputed_returns(score).If
requires_score(alg),scoreis notnothing.
Examples
julia> rd = ReturnsResult(; nx = ["A", "B", "C"],
X = [0.10 0.10 -0.05; -0.10 -0.10 0.07; 0.05 0.05 -0.02;
0.02 0.02 0.09]);
julia> sel = RedundancySelector(; alg = PairwiseCorrelation(; t = 0.99), score = SCM());
julia> PortfolioOptimisers.fit_preprocessing(sel, rd).nx
1-element Vector{String}:
"C"A and B are identical, so neither survives — the tie policy discards both.
Related
sourcePortfolioOptimisers.AbstractRedundancyAlgorithm Type
abstract type AbstractRedundancyAlgorithm <: AbstractAlgorithmAbstract supertype for the algorithms that decide which assets a RedundancySelector discards as redundant.
Each algorithm answers the same question — given the data and, optionally, a per-asset score, which columns survive? — and returns a keep-mask. The keep-mask, not a partition into groups, is the seam: PairwiseCorrelation drops one asset at a time and may keep two members of the same correlated blob, which "partition, then keep the best of each group" cannot express.
Two algorithms do partition, and share groups_argbest:
CorrelationComponentsgroups by connected component of the over-threshold correlation graph.ClusterGroupsgroups byclusteriseassignment.
Interfaces
Concrete redundancy algorithms must implement:
redundancy_keep(alg::MyAlgorithm, rd, scores, bib) -> BitVector.requires_score(::MyAlgorithm) -> Bool, if the algorithm cannot pick a survivor without one.
Related
sourcePortfolioOptimisers.PairwiseCorrelation Type
struct PairwiseCorrelation{__T_ce, __T_t, __T_absolute, __T_measure} <: AbstractRedundancyAlgorithmGreedy pairwise correlation pruning: drop assets until no surviving pair exceeds t.
Correlated pairs are visited from most to least correlated, and the worse asset of each pair is removed. "Worse" means the higher drop score: the RedundancySelector's score when it has one, otherwise each asset's summary correlation to the rest of the universe — so the asset that is redundant with most of the universe goes first.
This algorithm never chains. If ρ(A, B) = 0.97 and ρ(B, C) = 0.97 but ρ(A, C) = 0.10, it drops B and keeps both A and C, honouring the literal promise that no surviving pair exceeds t. CorrelationComponents reads the same inputs transitively and keeps only one of the three.
Delegates to find_uncorrelated_indices.
Fields
ce: Covariance estimator supplying the correlation matrix.t: Correlation at or above which two assets are considered redundant.absolute: Whether to compare the absolute value of the correlation.measure: Reducer producing the fallback drop score from each column of the correlation matrix; ignored when the selector carries ascore.
Constructors
PairwiseCorrelation(;
ce::StatsBase.CovarianceEstimator = PortfolioOptimisersCovariance(),
t::Number = 0.95,
absolute::Bool = false,
measure::Num_VecToScaM = MeanValue(),
) -> PairwiseCorrelationKeywords correspond to the struct's fields.
Related
sourcePortfolioOptimisers.CorrelationComponents Type
struct CorrelationComponents{__T_ce, __T_t, __T_absolute, __T_measure} <: AbstractRedundancyAlgorithmGroup assets by connected component of the over-threshold correlation graph, and keep the best-scoring member of each.
Two assets share an edge when their (absolute) correlation is at or above t. Components are transitive, so this reads a chain A ~ B ~ C as one redundant blob even when A and C are uncorrelated, and keeps a single asset from it. That is a stronger claim than PairwiseCorrelation's, and a stronger reduction; choose it when you want one representative per correlated blob rather than a guarantee about surviving pairs.
A component whose best score is tied keeps nobody (see groups_argbest).
Fields
ce: Covariance estimator supplying the correlation matrix.t: Correlation at or above which two assets share an edge.absolute: Whether to compare the absolute value of the correlation.measure: Reducer producing the fallback score from each column of the correlation matrix; ignored when the selector carries ascore. Lower is better, so the surviving representative is the least redundant member of its component.
Constructors
CorrelationComponents(;
ce::StatsBase.CovarianceEstimator = PortfolioOptimisersCovariance(),
t::Number = 0.95,
absolute::Bool = false,
measure::Num_VecToScaM = MeanValue(),
) -> CorrelationComponentsKeywords correspond to the struct's fields.
Related
sourcePortfolioOptimisers.ClusterGroups Type
struct ClusterGroups{__T_cle} <: AbstractRedundancyAlgorithmGroup assets by clustering them, and keep the best-scoring member of each cluster.
Clusters come from clusterise, so the whole clustering family — hierarchical linkage, DBHT, the non-hierarchical algorithms, and the optimal-number-of-clusters estimators — is available for deciding what "redundant" means. Unlike the correlation algorithms there is no natural fallback survivor rule, so a RedundancySelector using ClusterGroups must carry a score.
A cluster whose best score is tied keeps nobody (see groups_argbest).
Fields
cle: Clustering estimator partitioning the assets (AbstractClustersEstimator).
Constructors
ClusterGroups(;
cle::AbstractClustersEstimator = ClustersEstimator(),
) -> ClusterGroupsKeywords correspond to the struct's fields.
Related
sourceInternals
PortfolioOptimisers.asset_scores Function
asset_scores(
score::AbstractBaseRiskMeasure,
X::AbstractMatrix{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}}
) -> AnyEvaluate score on every asset column of X.
Columns are passed as views: a risk-measure functor reads its argument and never writes to it.
Validation
- Every score is finite. A
NaNscore (a drawdown measure on a constant series, say) would make the ordering meaningless, so it throws rather than sorting arbitrarily.
Related
sourcePortfolioOptimisers.rule_keep Function
rule_keep(
rule::ThresholdRule,
scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
_::Bool
) -> BitVectorTurn per-asset scores into a keep-mask under a selection rule.
bib is the bigger_is_better flag of the score that produced them; ThresholdRule ignores it.
Related
sourcePortfolioOptimisers.tail_mask Function
tail_mask(
scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
k::Integer,
bib::Bool,
tail::Symbol
) -> BitVectorReturn the mask of the k assets furthest into the tail end of the score ordering.
tail is :best or :worst; which raw direction that is comes from bib, the bigger_is_better flag of the score. The count saturates at length(scores).
Assets tied with the k-th score are included only when the whole tied block fits within k. Otherwise the block straddles the cut and is excluded — the "trust neither" tie policy, which is why the returned mask may hold fewer than k assets.
Related
sourcePortfolioOptimisers.tail_action_mask Function
tail_action_mask(
best::Union{Nothing, Integer},
worst::Union{Nothing, Integer},
action::Symbol,
scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
bib::Bool
) -> BitVectorUnion the two tail masks and apply the rule's action.
Related
sourcePortfolioOptimisers.groups_argbest Function
groups_argbest(
groups,
scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
bib::Bool
) -> BitVectorKeep the single best-scoring member of each redundancy group.
A group whose best score is tied keeps nobody: under the library's "if we cannot tell them apart, trust neither" policy, two indistinguishable assets are both discarded — the same stance find_uncorrelated_indices takes on an exactly-tied correlated pair. A singleton group is trivially unambiguous and always survives.
Arguments
groups: Vector of index vectors partitioning the assets.scores: Per-asset scores.bib: Whether a larger score is better.
Returns
keep::BitVector: One survivor per unambiguous group.
Related
sourcePortfolioOptimisers.correlation_components Function
correlation_components(
rho::AbstractMatrix{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
t::Number
) -> Vector{Vector{Int64}}Return the connected components of the graph whose edges are the pairs of rho at or above t.
A union-find pass over the strict lower triangle, so components are transitive and every asset lands in exactly one of them (a singleton when it has no over-threshold partner).
Returns
groups::Vector{Vector{Int}}: The components, each a vector of asset indices.
Related
sourcePortfolioOptimisers.drop_scores Function
drop_scores(
scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
bib::Bool
) -> AnyConvert per-asset scores into drop scores, where higher means "discard me first".
A risk measure with bigger_is_better == false (lower risk is better) already reads as a drop score; one with bigger_is_better == true is negated.
Related
sourcePortfolioOptimisers.redundancy_keep Function
redundancy_keep(alg::AbstractRedundancyAlgorithm, rd, scores, bib) -> BitVectorReturn the keep-mask a redundancy algorithm admits.
scores is nothing when the RedundancySelector carries no score; otherwise it is the per-asset score vector and bib is the score's bigger_is_better flag.
Related
sourcePortfolioOptimisers.requires_score Function
requires_score(_::AbstractRedundancyAlgorithm) -> BoolReturn whether a redundancy algorithm needs a score to pick the survivor of a redundancy group.
Correlation-based algorithms fall back on each asset's summary correlation to the rest of the universe when no score is given, so they return false. ClusterGroups has no such fallback and returns true.
Related
sourcePortfolioOptimisers.assert_scoreable Function
assert_scoreable(score::AbstractBaseRiskMeasure)Validate that score can be evaluated on a single asset's return series.
Scoring asset i is score(X[:, i]), which is exactly the precomputed-returns path supports_precomputed_returns governs. A WeightsInput measure — Variance and StandardDeviation among them — consumes portfolio weights instead, and cannot score an asset.
Related
sourcePortfolioOptimisers.assert_selection_action Function
assert_selection_action(action::Symbol)Validate the action field shared by the ordinal selection rules.
Related
sourcePortfolioOptimisers.assert_tail_counts Function
assert_tail_counts(
best::Union{Nothing, Integer},
worst::Union{Nothing, Integer},
name::Symbol
)Validate the best/worst tail sizes shared by the ordinal selection rules.
Related
sourcePortfolioOptimisers.assert_correlation_threshold Function
assert_correlation_threshold(t::Number)Validate a correlation threshold.
Related
source