Skip to content
18

Asset selection

Asset selectors narrow the universe from the data: drop constant columns, keep the best or worst assets by a risk measure, prune redundant ones. They are ordinary returns-preprocessing estimators — they know nothing about pipelines, and a Pipeline drives them through fit_preprocessing and apply_preprocessing like any other step.

The universe a selector chooses on the training window is its fitted state. Applying the fitted result to an unseen window replays that universe rather than re-deciding it, which is what makes a selector safe inside cross-validation.

See docs/adr/0029-asset-selection-is-returns-preprocessing.md for the design rationale, and PortfolioOptimisers.AbstractAssetSelector for the seam every selector shares.

Scoring assets with a risk measure

A ScoreSelector scores each asset by evaluating a risk measure on that asset's own return series, then hands the scores to a rule. Any risk measure whose supports_precomputed_returns is true may be used, which covers the quantile and drawdown families, the moment measures, and MeanReturn. bigger_is_better tells the ordinal rules which end of the ordering is "best".

Two measures are notable exceptions. Variance and StandardDeviation are WeightsInput measures: their functors consume portfolio weights, not a return series, so they cannot score a single asset and are rejected at construction. Use SCM(), which computes the same quantity from a return series — ZeroVarianceFilter spells this for you.

PortfolioOptimisers.ScoreSelector Type
julia
struct ScoreSelector{__T_score, __T_rule} <: AbstractAssetSelector

Asset selector that scores every asset with a risk measure and keeps the assets a rule admits.

score is any AbstractBaseRiskMeasure that can be evaluated on a bare return series — asset i's score is score(X[:, i]). That reuses the whole risk-measure family: ConditionalValueatRisk and the drawdown measures score risk, SCM() scores variance, MeanReturn scores mean return. bigger_is_better tells the ordinal rules which end is "best".

rule decides what to do with the scores: an absolute band (ThresholdRule) or a count/fraction taken from the tails (RankRule, QuantileRule).

The selected universe is fitted state, so a ScoreSelector is safe inside cross-validation: assets are chosen on the training window and the same universe is replayed on test windows.

Fields

Constructors

julia
ScoreSelector(;
    score::AbstractBaseRiskMeasure,
    rule::AbstractSelectionRule,
) -> ScoreSelector

Keywords correspond to the struct's fields.

Validation

Examples

julia
julia> rd = ReturnsResult(; nx = ["A", "B", "C"], X = [0.1 0.0 -0.2; -0.1 0.0 0.3; 0.2 0.0 -0.1]);

julia> sel = ScoreSelector(; score = SCM(), rule = ThresholdRule(; lo = 1e-12));

julia> PortfolioOptimisers.fit_preprocessing(sel, rd).nx
2-element Vector{String}:
 "A"
 "C"

Related

source
PortfolioOptimisers.CompleteAssetSelector Type
julia
struct CompleteAssetSelector <: AbstractAssetSelector

Asset selector that drops every asset column holding a missing or NaN observation.

The returns-level counterpart of MissingDataFilter's column threshold, for pipelines fed returns data directly (where the price stages never run). It has no observation-dropping mode: a fitted selector cannot decide which rows of an unseen window to drop without breaking the weights/returns alignment.

Constructors

julia
CompleteAssetSelector() -> CompleteAssetSelector

Examples

julia
julia> rd = ReturnsResult(; nx = ["A", "B"], X = [0.1 0.2; 0.3 NaN]);

julia> PortfolioOptimisers.fit_preprocessing(CompleteAssetSelector(), rd).nx
1-element Vector{String}:
 "A"

Related

source

Selection rules

A rule turns per-asset scores into a keep-mask. ThresholdRule is literal — it compares raw scores against absolute bounds and ignores orientation, because a zero-variance filter must drop the low-variance assets. RankRule and QuantileRule are ordinal — they consult bigger_is_better and take counts (or fractions) from each tail.

Ties at a rank cut are excluded entirely, so an ordinal rule may return fewer assets than asked. If the 20th and 21st assets score equally, RankRule(; best = 20) keeps 19: the tied block is dropped rather than split arbitrarily. This is the library's "if we cannot tell them apart, trust neither" tie policy.

PortfolioOptimisers.AbstractSelectionRule Type
julia
abstract type AbstractSelectionRule <: AbstractAlgorithm

Abstract supertype for the rules that turn per-asset scores into a keep-mask.

A selection rule is an AbstractAlgorithm: it is consumed through ScoreSelector and never used on its own. Rules split into two kinds.

  • Literal: ThresholdRule compares a score against absolute bounds and ignores bigger_is_better. A threshold on a variance means what it says; reinterpreting it as "keep the better ones" would invert the intent of a zero-variance filter.

  • Ordinal: RankRule and QuantileRule sort assets from best to worst — consulting bigger_is_better, so :best is lowest risk for a risk measure and highest value for a return measure — and take counts or fractions from each tail.

Related

source
PortfolioOptimisers.ThresholdRule Type
julia
struct ThresholdRule{__T_lo, __T_hi} <: AbstractSelectionRule

Keep assets whose score falls strictly inside the band (lo, hi).

Both bounds are optional and literal: lo and hi are compared against the raw score, never reinterpreted through bigger_is_better. Omitting a bound leaves that side unbounded.

Fields

  • lo: Exclusive lower bound on the score; nothing leaves the lower side unbounded.

  • hi: Exclusive upper bound on the score; nothing leaves the upper side unbounded.

Constructors

julia
ThresholdRule(;
    lo::Option{<:Number} = nothing,
    hi::Option{<:Number} = nothing,
) -> ThresholdRule

Keywords correspond to the struct's fields.

Validation

  • At least one of lo, hi is not nothing.

  • If both are given, lo < hi.

Examples

julia
julia> ThresholdRule(; lo = 1e-12)   # drop (near-)constant assets
ThresholdRule
  lo ┼ Float64: 1.0e-12
  hi ┴ nothing

Related

source
PortfolioOptimisers.RankRule Type
julia
struct RankRule{__T_best, __T_worst, __T_action} <: AbstractSelectionRule

Take best and/or worst assets from the tails of the score ordering, then keep or drop them.

best and worst are counts taken from each end, not positions: best = 20 means twenty assets, not rank twenty. Which end is "best" comes from bigger_is_better on the score, so RankRule(; best = 20) keeps the twenty lowest-risk assets for a risk measure and the twenty highest-return assets for MeanReturn. Giving both takes both tails. action = :drop complements the whole selection, which is how "drop the five worst" is said without knowing the universe size.

Counts saturate at the number of assets: best = 50 on a 30-asset window keeps all 30 rather than throwing, so a hyperparameter search over best is never killed by its largest point.

Warning

Ties at the cut are excluded entirely, so a rule may return fewer assets than asked. If the 20th and 21st assets have equal scores, RankRule(; best = 20) keeps 19 — the tied block is dropped rather than split arbitrarily. This is the library's "if we cannot tell them apart, trust neither" tie policy, shared with find_uncorrelated_indices, which removes both assets of an exactly-tied correlated pair. A window whose scores are all equal therefore selects nothing, and fit_preprocessing throws.

Fields

  • best: Number of assets to take from the best end; nothing takes none.

  • worst: Number of assets to take from the worst end; nothing takes none.

  • action: :keep retains the taken assets, :drop retains everything else.

Constructors

julia
RankRule(;
    best::Option{<:Integer} = nothing,
    worst::Option{<:Integer} = nothing,
    action::Symbol = :keep,
) -> RankRule

Keywords correspond to the struct's fields.

Validation

  • At least one of best, worst is not nothing.

  • Any given count is >= 0, and at least one is > 0.

  • action in (:keep, :drop).

Examples

julia
julia> RankRule(; worst = 5, action = :drop)   # drop the five worst
RankRule
    best ┼ nothing
   worst ┼ Int64: 5
  action ┴ Symbol: :drop

Related

source
PortfolioOptimisers.QuantileRule Type
julia
struct QuantileRule{__T_best, __T_worst, __T_action} <: AbstractSelectionRule

RankRule with the tail sizes given as fractions of the asset universe.

best and worst are fractions in (0, 1), converted to counts as round(Int, fraction * n) on the window being fitted. Everything else — orientation via bigger_is_better, the action complement, count saturation, and the tie policy that excludes a straddling tied block — is identical to RankRule.

Fractions and counts are separate types on purpose: best = 1 (one asset) and best = 1.0 (the whole universe) would otherwise differ only by a literal's type.

Fields

  • best: Fraction of the universe to take from the best end; nothing takes none.

  • worst: Fraction of the universe to take from the worst end; nothing takes none.

  • action: :keep retains the taken assets, :drop retains everything else.

Constructors

julia
QuantileRule(;
    best::Option{<:Real} = nothing,
    worst::Option{<:Real} = nothing,
    action::Symbol = :keep,
) -> QuantileRule

Keywords correspond to the struct's fields.

Validation

  • At least one of best, worst is not nothing.

  • Any given fraction lies in (0, 1).

  • action in (:keep, :drop).

Examples

julia
julia> QuantileRule(; worst = 0.1, action = :drop)   # drop the worst decile
QuantileRule
    best ┼ nothing
   worst ┼ Float64: 0.1
  action ┴ Symbol: :drop

Related

source

Discarding redundant assets

A RedundancySelector discards assets that duplicate information already carried by others. Its alg decides what "redundant" means, and its score decides which member of a redundancy group survives.

PairwiseCorrelation is greedy: it drops one asset at a time until no surviving pair exceeds the threshold, and never chains. CorrelationComponents reads the same correlations transitively, treating a chain A ~ B ~ C as one blob and keeping a single representative — a stronger reduction, and a different answer on the same input. ClusterGroups partitions with clusterise and keeps one representative per cluster.

Leaving score as nothing falls back to the correlation algorithms' own survivor rule: the asset with the lowest summary correlation to the rest of the universe. ClusterGroups has no such fallback and requires a score.

PortfolioOptimisers.RedundancySelector Type
julia
struct RedundancySelector{__T_alg, __T_score} <: AbstractAssetSelector

Asset selector that discards assets which duplicate information already carried by others.

alg decides what "redundant" means and returns the keep-mask: greedy pairwise correlation pruning (PairwiseCorrelation), one representative per correlated blob (CorrelationComponents), or one representative per cluster (ClusterGroups).

score decides which asset survives a redundancy group — a risk measure evaluated on each asset's own return series, oriented by bigger_is_better, exactly as in ScoreSelector. Leaving it nothing falls back to the correlation algorithms' own rule: the asset with the lowest summary correlation to the rest of the universe survives. ClusterGroups has no such fallback and requires a score.

Fields

Constructors

julia
RedundancySelector(;
    alg::AbstractRedundancyAlgorithm = PairwiseCorrelation(),
    score::Option{<:AbstractBaseRiskMeasure} = nothing,
) -> RedundancySelector

Keywords correspond to the struct's fields.

Validation

  • If score is given, supports_precomputed_returns(score).

  • If requires_score(alg), score is not nothing.

Examples

julia
julia> rd = ReturnsResult(; nx = ["A", "B", "C"],
                          X = [0.10 0.10 -0.05; -0.10 -0.10 0.07; 0.05 0.05 -0.02;
                               0.02 0.02 0.09]);

julia> sel = RedundancySelector(; alg = PairwiseCorrelation(; t = 0.99), score = SCM());

julia> PortfolioOptimisers.fit_preprocessing(sel, rd).nx
1-element Vector{String}:
 "C"

A and B are identical, so neither survives — the tie policy discards both.

Related

source
PortfolioOptimisers.AbstractRedundancyAlgorithm Type
julia
abstract type AbstractRedundancyAlgorithm <: AbstractAlgorithm

Abstract supertype for the algorithms that decide which assets a RedundancySelector discards as redundant.

Each algorithm answers the same question — given the data and, optionally, a per-asset score, which columns survive? — and returns a keep-mask. The keep-mask, not a partition into groups, is the seam: PairwiseCorrelation drops one asset at a time and may keep two members of the same correlated blob, which "partition, then keep the best of each group" cannot express.

Two algorithms do partition, and share groups_argbest:

Interfaces

Concrete redundancy algorithms must implement:

  • redundancy_keep(alg::MyAlgorithm, rd, scores, bib) -> BitVector.

  • requires_score(::MyAlgorithm) -> Bool, if the algorithm cannot pick a survivor without one.

Related

source
PortfolioOptimisers.PairwiseCorrelation Type
julia
struct PairwiseCorrelation{__T_ce, __T_t, __T_absolute, __T_measure} <: AbstractRedundancyAlgorithm

Greedy pairwise correlation pruning: drop assets until no surviving pair exceeds t.

Correlated pairs are visited from most to least correlated, and the worse asset of each pair is removed. "Worse" means the higher drop score: the RedundancySelector's score when it has one, otherwise each asset's summary correlation to the rest of the universe — so the asset that is redundant with most of the universe goes first.

This algorithm never chains. If ρ(A, B) = 0.97 and ρ(B, C) = 0.97 but ρ(A, C) = 0.10, it drops B and keeps both A and C, honouring the literal promise that no surviving pair exceeds t. CorrelationComponents reads the same inputs transitively and keeps only one of the three.

Delegates to find_uncorrelated_indices.

Fields

  • ce: Covariance estimator supplying the correlation matrix.

  • t: Correlation at or above which two assets are considered redundant.

  • absolute: Whether to compare the absolute value of the correlation.

  • measure: Reducer producing the fallback drop score from each column of the correlation matrix; ignored when the selector carries a score.

Constructors

julia
PairwiseCorrelation(;
    ce::StatsBase.CovarianceEstimator = PortfolioOptimisersCovariance(),
    t::Number = 0.95,
    absolute::Bool = false,
    measure::Num_VecToScaM = MeanValue(),
) -> PairwiseCorrelation

Keywords correspond to the struct's fields.

Related

source
PortfolioOptimisers.CorrelationComponents Type
julia
struct CorrelationComponents{__T_ce, __T_t, __T_absolute, __T_measure} <: AbstractRedundancyAlgorithm

Group assets by connected component of the over-threshold correlation graph, and keep the best-scoring member of each.

Two assets share an edge when their (absolute) correlation is at or above t. Components are transitive, so this reads a chain A ~ B ~ C as one redundant blob even when A and C are uncorrelated, and keeps a single asset from it. That is a stronger claim than PairwiseCorrelation's, and a stronger reduction; choose it when you want one representative per correlated blob rather than a guarantee about surviving pairs.

A component whose best score is tied keeps nobody (see groups_argbest).

Fields

  • ce: Covariance estimator supplying the correlation matrix.

  • t: Correlation at or above which two assets share an edge.

  • absolute: Whether to compare the absolute value of the correlation.

  • measure: Reducer producing the fallback score from each column of the correlation matrix; ignored when the selector carries a score. Lower is better, so the surviving representative is the least redundant member of its component.

Constructors

julia
CorrelationComponents(;
    ce::StatsBase.CovarianceEstimator = PortfolioOptimisersCovariance(),
    t::Number = 0.95,
    absolute::Bool = false,
    measure::Num_VecToScaM = MeanValue(),
) -> CorrelationComponents

Keywords correspond to the struct's fields.

Related

source
PortfolioOptimisers.ClusterGroups Type
julia
struct ClusterGroups{__T_cle} <: AbstractRedundancyAlgorithm

Group assets by clustering them, and keep the best-scoring member of each cluster.

Clusters come from clusterise, so the whole clustering family — hierarchical linkage, DBHT, the non-hierarchical algorithms, and the optimal-number-of-clusters estimators — is available for deciding what "redundant" means. Unlike the correlation algorithms there is no natural fallback survivor rule, so a RedundancySelector using ClusterGroups must carry a score.

A cluster whose best score is tied keeps nobody (see groups_argbest).

Fields

Constructors

julia
ClusterGroups(;
    cle::AbstractClustersEstimator = ClustersEstimator(),
) -> ClusterGroups

Keywords correspond to the struct's fields.

Related

source

Internals

PortfolioOptimisers.asset_scores Function
julia
asset_scores(
    score::AbstractBaseRiskMeasure,
    X::AbstractMatrix{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}}
) -> Any

Evaluate score on every asset column of X.

Columns are passed as views: a risk-measure functor reads its argument and never writes to it.

Validation

  • Every score is finite. A NaN score (a drawdown measure on a constant series, say) would make the ordering meaningless, so it throws rather than sorting arbitrarily.

Related

source
PortfolioOptimisers.rule_keep Function
julia
rule_keep(
    rule::ThresholdRule,
    scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
    _::Bool
) -> BitVector

Turn per-asset scores into a keep-mask under a selection rule.

bib is the bigger_is_better flag of the score that produced them; ThresholdRule ignores it.

Related

source
PortfolioOptimisers.tail_mask Function
julia
tail_mask(
    scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
    k::Integer,
    bib::Bool,
    tail::Symbol
) -> BitVector

Return the mask of the k assets furthest into the tail end of the score ordering.

tail is :best or :worst; which raw direction that is comes from bib, the bigger_is_better flag of the score. The count saturates at length(scores).

Assets tied with the k-th score are included only when the whole tied block fits within k. Otherwise the block straddles the cut and is excluded — the "trust neither" tie policy, which is why the returned mask may hold fewer than k assets.

Related

source
PortfolioOptimisers.tail_action_mask Function
julia
tail_action_mask(
    best::Union{Nothing, Integer},
    worst::Union{Nothing, Integer},
    action::Symbol,
    scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
    bib::Bool
) -> BitVector

Union the two tail masks and apply the rule's action.

Related

source
PortfolioOptimisers.groups_argbest Function
julia
groups_argbest(
    groups,
    scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
    bib::Bool
) -> BitVector

Keep the single best-scoring member of each redundancy group.

A group whose best score is tied keeps nobody: under the library's "if we cannot tell them apart, trust neither" policy, two indistinguishable assets are both discarded — the same stance find_uncorrelated_indices takes on an exactly-tied correlated pair. A singleton group is trivially unambiguous and always survives.

Arguments

  • groups: Vector of index vectors partitioning the assets.

  • scores: Per-asset scores.

  • bib: Whether a larger score is better.

Returns

  • keep::BitVector: One survivor per unambiguous group.

Related

source
PortfolioOptimisers.correlation_components Function
julia
correlation_components(
    rho::AbstractMatrix{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
    t::Number
) -> Vector{Vector{Int64}}

Return the connected components of the graph whose edges are the pairs of rho at or above t.

A union-find pass over the strict lower triangle, so components are transitive and every asset lands in exactly one of them (a singleton when it has no over-threshold partner).

Returns

  • groups::Vector{Vector{Int}}: The components, each a vector of asset indices.

Related

source
PortfolioOptimisers.drop_scores Function
julia
drop_scores(
    scores::AbstractVector{<:Union{var"#s29", var"#s28"} where {var"#s29"<:Number, var"#s28"<:AbstractJuMPScalar}},
    bib::Bool
) -> Any

Convert per-asset scores into drop scores, where higher means "discard me first".

A risk measure with bigger_is_better == false (lower risk is better) already reads as a drop score; one with bigger_is_better == true is negated.

Related

source
PortfolioOptimisers.redundancy_keep Function
julia
redundancy_keep(alg::AbstractRedundancyAlgorithm, rd, scores, bib) -> BitVector

Return the keep-mask a redundancy algorithm admits.

scores is nothing when the RedundancySelector carries no score; otherwise it is the per-asset score vector and bib is the score's bigger_is_better flag.

Related

source
PortfolioOptimisers.requires_score Function
julia
requires_score(_::AbstractRedundancyAlgorithm) -> Bool

Return whether a redundancy algorithm needs a score to pick the survivor of a redundancy group.

Correlation-based algorithms fall back on each asset's summary correlation to the rest of the universe when no score is given, so they return false. ClusterGroups has no such fallback and returns true.

Related

source
PortfolioOptimisers.assert_scoreable Function
julia
assert_scoreable(score::AbstractBaseRiskMeasure)

Validate that score can be evaluated on a single asset's return series.

Scoring asset i is score(X[:, i]), which is exactly the precomputed-returns path supports_precomputed_returns governs. A WeightsInput measure — Variance and StandardDeviation among them — consumes portfolio weights instead, and cannot score an asset.

Related

source
PortfolioOptimisers.assert_selection_action Function
julia
assert_selection_action(action::Symbol)

Validate the action field shared by the ordinal selection rules.

Related

source
PortfolioOptimisers.assert_tail_counts Function
julia
assert_tail_counts(
    best::Union{Nothing, Integer},
    worst::Union{Nothing, Integer},
    name::Symbol
)

Validate the best/worst tail sizes shared by the ordinal selection rules.

Related

source
PortfolioOptimisers.assert_correlation_threshold Function
julia
assert_correlation_threshold(t::Number)

Validate a correlation threshold.

Related

source