GitHub

Base optimisation: private API

PortfolioOptimisers.TD — Type
const TD{X} = Union{<:TimeDependent, X}

Alias for a required optimiser field that accepts a static value of type X or a per-fold TimeDependent schedule, but not nothing.

The problem-definition fields that always carry a value — the prior estimator, the returns model, the scalariser, the clustering estimator, the weight finaliser — are time-dependent through this alias rather than TD_Option, so nothing stays inadmissible where it was never a legal static value. Such a field still has a static default, so a schedule in one resets to that default on a fold-less solve, unlike the optimiser-valued fields (see TD_OptE_Opt).

Related

source
PortfolioOptimisers.TD_OptE_Opt — Type
const TD_OptE_Opt = Union{TimeDependent{<:AbstractVector{<:OptE_Opt}},
                          TimeDependent{<:TimeDependentOptimiserCallable},
                          TimeDependent{<:PreviousWeightsFunction},
                          TimeDependent{<:Base.Callable}}

The TimeDependent forms admissible in an optimiser-valued field — where the scheduled thing is the optimiser itself, not one of its inputs.

Two of the four are statically checked: a vector schedule whose entries are all OptE_Opt (an optimiser or a precomputed result — a mixed schedule is allowed, fold i optimising or predicting depending on what entry i is), and a TimeDependentOptimiserCallable, which declares its output kind in its type. The other two — a bare ctx -> optimiser and a PreviousWeightsFunction wrapping one — cannot be checked before they run, so their output is checked when the fold loop swaps it into the field, by the host's own keyword constructor.

Because an optimiser-valued field is required, a schedule in one has no static default to reset to on a fold-less solve and must supply default (see NoDefault, TimeDependentDefaultError).

Related

source
PortfolioOptimisers.OptE_TD — Type
const OptE_TD = Union{<:NonFiniteAllocationOptimisationEstimator, <:TD_OptE_Opt}

Alias for an optimisation estimator, or a TimeDependent schedule standing in its place.

This is the entry-point type of the cross-validation fold loops that fit: a schedule handed straight to cross_val_predict is the optimiser, and fold i runs entry i. Precomputed results are excluded because a bare result takes the predict-only path, which has no fold loop to resolve a schedule against — but a schedule whose entries are results is admissible here, and each such entry takes the predict-only path per fold (see OptE_Opt_TD).

Related

source
PortfolioOptimisers.OptE_Opt_TD — Type
const OptE_Opt_TD = Union{<:OptE_Opt, <:TD_OptE_Opt}

Alias for an optimisation estimator or a precomputed result, or a TimeDependent schedule standing in their place.

The entry-point type of the fold loops that accept a precomputed result as well as an estimator. A schedule's entries are OptE_Opt, so a mixed schedule is admissible: fold i optimises when entry i is an estimator and predicts when it is a result, which the single-fold fit_and_predict methods already distinguish by dispatch.

Related

source
PortfolioOptimisers.VecOptE_Opt_TD — Type
const VecOptE_Opt_TD = AbstractVector{<:OptE_Opt_TD}

Alias for a vector of optimisation estimators or results in which individual elements may be TimeDependent schedules.

This is the element-level admission of schedules, needed where a vector-valued field's elements are themselves optimiser positions consumed by a fold loop one at a time — Stacking.opti, whose inner cross-validation is entered per candidate. It is a superset of VecOptE_Opt, so every method taking it continues to accept plain vectors.

Related

source
PortfolioOptimisers.TD_VecOptE_Opt — Type
const TD_VecOptE_Opt = Union{TimeDependent{<:AbstractVector{<:VecOptE_Opt_TD}},
                             TimeDependent{<:TimeDependentOptimiserCallable},
                             TimeDependent{<:PreviousWeightsFunction},
                             TimeDependent{<:Base.Callable}}

The TimeDependent forms admissible in a vector-of-optimisers field (Stacking.opti): a vector schedule whose entries are per-fold optimiser vectors, or a callable returning the fold's vector.

Entry i is fold i's complete vector of candidates, so a field-level schedule varies the whole candidate set per fold; an entry's own elements may in turn be schedules (a VecOptE_Opt_TD), which the consuming host's inner fold loop resolves as usual. Only bind = :outermost is admissible at the field level — see the host's constructor for why.

Related

source
PortfolioOptimisers.TDO_OptE_Opt — Type
const TDO_OptE_Opt = Union{<:TD_OptE_Opt,
                           <:TimeDependent{<:AbstractVector{<:Option{<:OptE_Opt}}}}

The TimeDependent forms admissible in an optional optimiser-valued field (a fallback): every TD_OptE_Opt form, plus a vector schedule whose entries may be nothing.

nothing was always a legal static value of an optional field, and the TimeDependent contract says a vector entry may be nothing, giving the field nothing for that fold — so an optional optimiser field admits TimeDependent([mr, nothing]), a fallback switched off on some folds. A required optimiser position (the optimiser itself) never admits nothing, statically or per fold, so it stays on the strict TD_OptE_Opt bound.

Related

source
PortfolioOptimisers.OptE_Opt — Type
const OptE_Opt = Union{<:NonFiniteAllocationOptimisationEstimator,
                       <:NonFiniteAllocationOptimisationResult}

Alias for a non-finite allocation optimisation estimator or result.

Matches either a NonFiniteAllocationOptimisationEstimator (specifying an optimiser configuration) or a NonFiniteAllocationOptimisationResult (a pre-computed result). Used for dispatch in cross-validation and optimisation workflows that accept either form.

Related

source
PortfolioOptimisers.FbChain — Type
const FbChain = AbstractVector{<:Tuple{<:OptimisationEstimator, <:OptimisationResult}}

Alias for a fallback chain: the (estimator, result) pair of every attempt that failed before the result that carries it, in the order the attempts ran.

optimise pushes one pair each time an attempt fails and its estimator names a fallback, and hands the vector to factory(res, fb) once an attempt succeeds or the chain runs out. A result whose fb is a chain was therefore answered by a fallback: fb[1][1] is the estimator that was asked first, and fb[end][2] is the last failure before the answer. A result whose fb is nothing was answered by the estimator it was asked of.

Related

source
PortfolioOptimisers.pipe_route — Function
pipe_route(x, ::Val{target}, v)

Absorb a Routing Target's value into an optimiser, returning the rebuilt optimiser.

This is the optimiser-owned half of the Pipeline seam: inject_context fans a PipelineContext slot out into routing targets and delivers each one here, knowing nothing about where it lands.

Targets are named after the field they land in — :pe, :cle, :wb, :lcse, :ple — because those names are this package's shared vocabulary (see field_dict) rather than any one optimiser's private layout. The default method therefore is the routing rule: a target lands in the like-named field of any optimiser that has one. Nothing is declared per type, so nothing can drift.

Two targets are exceptions, because they carry validation policy and name no plain field: :mu_ucs (requires an ArithmeticReturn, lands in ret.ucs) and :sigma_ucs (lands in the UncertaintySetVariance measures of r, see @pipe_route_sigma_ucs).

Optimisers holding their configuration in a field rather than carrying the target fields themselves declare @pipe_delegates.

The lookup is hasfield rather than hasproperty, because routing rebuilds the object through the field: a name reachable only as a forwarded property could be read but not set.

A target with no home falls through to unroutable_target, which ignores the optional ones and throws for the rest.

Internal machinery — not part of the user-facing API.

Arguments

  • x: The optimiser or optimiser configuration.
  • ::Val{target}: One of PIPELINE_ROUTING_TARGETS.
  • v: The computed result to absorb.

Returns

  • x′: The rebuilt optimiser.

Related

source
pipe_route(cfg::JuMPOptimiser, _::Val{:mu_ucs}, v) -> Any

Route a mean uncertainty set into a JuMPOptimiser's return estimator.

One of the two Routing Targets that names no plain field: the set lands in ret.ucs, and only an ArithmeticReturn can bound expected returns, so any other return estimator is an error rather than a silent drop.

A vector of return terms is refused for the same reason, from the other side: one set is a neighbourhood of one quantity, so broadcasting it across k terms would apply a ball fitted on one fit to every other one — the very defect #277 removed. Name the set on the term it belongs to instead.

Internal machinery — not part of the user-facing API.

Related

source
PortfolioOptimisers.@pipe_delegates — Macro
@pipe_delegates T field

Declare that optimiser type T forwards every Routing Target to the configuration held in field.

Emits pipe_config_field plus forwarding pipe_route and pipe_accepts methods. Targets the configuration has no home for reach its own unroutable_target, so the resulting error names the configuration — matching the pre-inversion messages.

A type that absorbs a target itself rather than through its configuration declares that target on the concrete type (see @pipe_route_sigma_ucs), which out-specialises this forwarder.

Examples

@pipe_delegates MeanRisk opt

Related

source
PortfolioOptimisers.route_sigma_ucs — Function
route_sigma_ucs(x, sig::AbstractUncertaintySetResult) -> Any

Route a covariance uncertainty set into the UncertaintySetVariance risk measure(s) held in an optimiser's r field.

Each UncertaintySetVariance found — directly or inside a vector — has its ucs replaced with sig. An r field carrying no such measure is an error rather than a silent no-op: a computed uncertainty set that reaches no risk measure would be dropped.

Internal machinery — not part of the user-facing API.

Arguments

  • x: The optimiser, which must carry an r field.
  • sig: The covariance uncertainty set result.

Returns

  • x′: The rebuilt optimiser.

Related

source
PortfolioOptimisers.@pipe_route_sigma_ucs — Macro
@pipe_route_sigma_ucs T

Declare that optimiser type T absorbs the :sigma_ucs Routing Target into its own r field via route_sigma_ucs.

Declared per concrete type rather than on a supertype: the covariance uncertainty set lands in the estimator's risk measures while every other target is forwarded to its configuration, so this method must out-specialise the @pipe_delegates forwarder on the same type. It is opt-in because carrying a configuration does not imply carrying risk measures — RelaxedRiskBudgeting has no r field.

Related

source
PortfolioOptimisers.@pipe_route_rkb — Macro
@pipe_route_rkb T

Declare that optimiser type T absorbs the :rkb Routing Target into the rkb field of its risk-budgeting algorithm.

:rkb is the one target named after a field an optimiser does not carry directly. A risk budget belongs to the algorithm — AssetRiskBudgeting budgets assets, FactorRiskBudgeting budgets factors — so it lands one level down, at rba.rkb, and the derived hasfield rule cannot reach it.

Acceptance is not a constant: it asks the algorithm the optimiser is actually carrying. A TimeDependent schedule in rba has no rkb to write, so such an optimiser declines the target and a pipeline computing a budget for it is refused at construction rather than failing in the fold loop.

Declared per concrete type, for the same reason as @pipe_route_sigma_ucs: it must out-specialise the @pipe_delegates forwarder on the same type.

Related

source
PortfolioOptimisers.result_investable_mask — Function
result_investable_mask(res::OptimisationResult) -> Option{BitVector}

Read the Investable Mask an optimisation result reduced on.

An optimisation reduces to the Investable Mask at its entry and expands the solved weights back to the caller's universe, so the result's own w is on the full universe and the mask is the record of which assets the optimisation traded. A fold reads that record to view its test window before it scores the weights.

The mask is read through this verb rather than off a field, because the families carry it in different places: a JuMP result holds it on its processed attribute bundle, a hierarchical leaf holds it on the core it wraps, and a family that derives no mask answers nothing. A family that gains a mask must add its own method here, and so must a leaf that forwards its properties into a core, because the verb dispatches on the type and a forwarded res.imsk never reaches it. test/test_54_held_gap_filter.jl censuses the concrete results and fails when one carries a mask this verb cannot read, as a field or as a forwarded property, so the omission cannot be silent.

Arguments

  • res::OptimisationResult: Fitted optimisation result.

Returns

  • imsk::Option{BitVector}: The Investable Mask, or nothing when the optimisation reduced on nothing.

Related

source
result_investable_mask(res::HierarchicalResult)
result_investable_mask(res::HierarchicalRiskParityResult)
result_investable_mask(res::HierarchicalEqualRiskContributionResult)

Read the Investable Mask a hierarchical result reduced on.

The core carries the mask as imsk, and the two leaves carry the core as hr. A leaf answers through the core rather than through its forwarded res.imsk, because the verb dispatches on the type and a forwarded property is invisible to it: without these methods the leaf falls back to nothing, the fold keeps the full weights, and a per-asset fee the result carries reduced is charged against them (#892).

Arguments

  • res: A hierarchical result, or one of its two leaves.

Returns

  • imsk::Option{BitVector}: The Investable Mask, or nothing when the optimisation reduced on nothing.

Related

source
PortfolioOptimisers.set_retcode — Function
set_retcode(res::NonFiniteAllocationOptimisationResult, retcode::OptRetCode_VecOptRetCode)

Rebuild an optimisation result with a different return code, and every other member unchanged.

A cross-validation fold that drifts a population's weights drops the members whose wealth is not positive, and it drops them by failing their entry of the result's return code. A result is an immutable record, so the drop rebuilds it. The rebuild is a per-type method that writes the constructor name once, rather than a reflection pass over the field list.

Only a result that can carry a population of weight vectors needs a method here, because only such a result can hold one return code per member. A result that reaches the fallback raises, and the message names the type that is missing its method.

Arguments

  • res: Optimisation result to rebuild.
  • retcode: Return code, or one per member of the population.

Validation

  • The type of res declares a method of its own, else an ArgumentError is raised.

Returns

  • NonFiniteAllocationOptimisationResult: The result, with the new return code.

Related

source
set_retcode(res::JuMPOptimisationResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a JuMPOptimisationResult with a different return code.

The rebuild reaches the inner constructor rather than the keyword one, because the keyword constructor expands the solver's reduced weight vector onto the caller's universe. sol is already expanded here, so a second pass through that door would expand it twice.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::MeanRiskResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a MeanRiskResult with a different return code.

retcode is not a field of this result and resolves through the JuMPOptimisationResult it embeds, so the rebuild rebuilds jr and carries every other member over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::FactorRiskContributionResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a FactorRiskContributionResult with a different return code.

retcode is not a field of this result and resolves through the JuMPOptimisationResult it embeds, so the rebuild rebuilds jr and carries every other member over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::NearOptimalCenteringResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a NearOptimalCenteringResult with a different return code.

The population's return code is the one the embedded JuMPOptimisationResult carries, because retcode is not a field of this result and resolves through jr. So the rebuild rebuilds jr, and the three return codes this result names of its own are carried over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::RiskBudgetingResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a RiskBudgetingResult with a different return code.

retcode is not a field of this result and resolves through the JuMPOptimisationResult it embeds, so the rebuild rebuilds jr and carries every other member over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::RelaxedRiskBudgetingResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a RelaxedRiskBudgetingResult with a different return code.

retcode is not a field of this result and resolves through the JuMPOptimisationResult it embeds, so the rebuild rebuilds jr and carries every other member over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::NestedClusteredResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a NestedClusteredResult with a different return code.

The result carries one return code per member of the population, so a member is dropped by failing its own entry. Every other member of the record is carried over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::StackingResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a StackingResult with a different return code.

The result carries one return code per member of the population, so a member is dropped by failing its own entry. Every other member of the record is carried over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
set_retcode(res::SubsetResamplingResult, retcode::OptRetCode_VecOptRetCode)

Rebuild a SubsetResamplingResult with a different return code.

The result carries one return code per member of the population, so a member is dropped by failing its own entry. Every other member of the record is carried over unchanged.

Arguments

  • res: Result to rebuild.
  • retcode: Return code, or one per member of the population.

Returns

Related

source
PortfolioOptimisers.assert_special_nco_requirements — Method
assert_special_nco_requirements(opt)

Assert that the optimiser meets special requirements for Nested Clustered Optimisation (NCO).

The default implementation does nothing. Overridden for estimators (e.g. Stacking) that have requirements which must be validated before NCO can proceed.

Arguments

  • opt: Optimisation estimator, result, or vector thereof.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_special_nco_requirements — Method
assert_special_nco_requirements(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}}
)

Assert special NCO requirements for each element of a vector of optimisation estimators or results.

Related

source
PortfolioOptimisers.assert_nearest_optimiser_schedule — Function
assert_nearest_optimiser_schedule(x, field::Symbol, cv, host::Symbol)

Validate a bind = :nearest TimeDependent schedule in an optimiser-valued position that a host's inner cross-validation does consume.

Two construction-time requirements, both consequences of the position's double consumer: the inner cross-validation leg resolves the schedule per fold, while the full-sample leg (the meta's wi fit, or the per-cluster optimise) always resolves it fold-lessly to its default.

  • An explicit default is required — without one, every solve would throw a TimeDependentDefaultError when the full-sample leg reaches the schedule, so the error is moved to construction. This deliberately departs from the rule that a defaultless schedule is legal at construction, for this position only.
  • cv !== nothing is required — without an inner cross-validation there is no inner fold loop, so the schedule could only ever be its default: silently inert.

No-op for anything that is not a bind = :nearest TimeDependent.

Related

source
PortfolioOptimisers.inner_fold_fields — Function
inner_fold_fields(opt)

Field names of opt that the host hands across a fold loop it opens itself, so the loop that merely reaches the host is never the nearest one for them.

The default is the empty tuple: an ordinary host opens no inner fold loop, so the loop reaching it is both outermost and nearest for every field. A meta-optimiser whose inner cross-validation consumes a field directly declares that field here (e.g. NestedClustered's opti, entered per cluster as cross_val_predict(opti, …; cols = cl)), and every generic pass — time_dependent_fields, and through it update, reset and the fold-count assertion — then leaves a bind = :nearest schedule in that field for the host's own inner loop. Without the reset leg of this rule, the fold-less reset at the top of _optimise would replace a :nearest optimiser schedule with its default before the inner cross-validation ever saw it.

Related

source
PortfolioOptimisers.time_dependent_candidate_fields — Function
time_dependent_candidate_fields(opt)

Field names of opt whose type admits a TimeDependent value — the candidate set time_dependent_fields narrows by value.

Whether a field can hold a schedule is decidable from fieldtype alone: a host built through the widened constructor signatures (see TD_Option) records a schedule in the field's type parameter, so a field that holds no schedule cannot have a type intersecting TimeDependent. The tuple is therefore computed once per host type by a generated function, and a fold-invariant scan over a wide static host such as JuMPOptimiser, whose fields number in the dozens, folds to an empty tuple at compile time rather than walking every field dynamically on every split and _optimise.

This stays derived from the field types — no hand-maintained list — so the constructor signatures remain the single source of truth for which fields may vary over folds.

Related

source
PortfolioOptimisers.time_dependent_fields — Function
time_dependent_fields(opt, all_binds::Bool = true)

Return the tuple of field names of opt whose values are TimeDependent.

The scan is generic over the host's fields, so the widened constructor signatures (see TD_Option) remain the single source of truth for which fields may vary over folds — there is no hand-maintained list. Only the fields whose type admits a schedule are visited (see time_dependent_candidate_fields); the rest are ruled out at compile time, so a static host returns an empty tuple without touching its fields.

The all_binds argument

all_binds encodes something the schedule's own bind field cannot: it is a property of the recursion position, not of the schedule. A TimeDependent's bind (:outermost / :nearest) says which fold loop the schedule wants; all_binds says whether the loop currently recursing is entitled to consume nearest-bound schedules at this depth. The second fact is not on the schedule.

Why position matters: under outer CV loop → meta → (meta's inner CV loop) → inner estimator with a :nearest field, the same :nearest field is visited by two loops. The outer loop recurses through the meta (mandatory — that recursion is how an inner estimator's :outermost field is resolved against the outer folds) and must skip the :nearest field, because it is not the nearest enclosing loop. The meta's inner CV loop drives the same estimator directly and must consume it, because it is. Same field, same bind, opposite actions — the difference is whether a nearer fold-loop boundary was crossed to reach it, which is exactly what all_binds carries.

So all_binds is true at every ordinary (outermost/standalone) fold loop — which is both outermost and nearest, and therefore takes everything remaining, including :nearest. It is forced to false only where a meta-optimiser recurses into the estimators its own inner CV owns, leaving their :nearest schedules for that inner loop. With all_binds = false, only fields with bind === :outermost are returned.

Entitlement is refined per field by inner_fold_fields: even at all_binds = true, a :nearest schedule in a field the host hands across its own inner fold loop is left alone — the host's inner loop, not the scanning one, is nearest for that field (see entitled).

Related

source
PortfolioOptimisers.assert_time_dependent_substitution — Function
assert_time_dependent_substitution(
    _::Type{T},
    args::NamedTuple,
    defaults::NamedTuple
)

Test-substitute every vector entry of the TimeDependent-valued fields in args through the keyword constructor of T.

args holds the host constructor's arguments; defaults the static defaults of the fields that may be time-dependent (see time_dependent_field_defaults). Each per-fold entry, and an explicit default, is substituted into its field — with every other time-dependent field standing at a value of its own (see time_dependent_stand_in) — and the constructor re-run, surfacing type and cross-field errors at construction time instead of mid-backtest. Substituted calls contain no TimeDependent values, so the recursion terminates.

Validation is skipped when a time-dependent field has no stand-in at all — a callable schedule in a required field, whose value only exists once a fold context does.

Related

source
PortfolioOptimisers.time_dependent_stand_in — Function
time_dependent_stand_in(
    td::TimeDependent,
    defaults::NamedTuple,
    field::Symbol
) -> Union{Nothing, Some}

Return a valid static value for a TimeDependent-valued field, wrapped in Some, or nothing if none exists.

Used by assert_time_dependent_substitution to stand the other time-dependent fields at a valid value while it test-substitutes one of them: the schedule's default, else the field's static default, else the schedule's first entry. A callable schedule in a field with no default of either kind has no stand-in — its value exists only inside a fold — so it returns nothing and validation is skipped.

Unlike time_dependent_reset_value this never throws: a schedule without a fold-less value is legitimate at construction time and only fails if it reaches a fold-less solve.

Related

source
PortfolioOptimisers.time_dependent_reset_value — Function
time_dependent_reset_value(
    td::TimeDependent,
    defaults::NamedTuple,
    field::Symbol,
    opt
) -> Any

Return the value a TimeDependent-valued field takes outside every fold loop.

The schedule's own default wins; absent one (NoDefault), the host's static default for field is used (time_dependent_field_defaults, nothing for fields it omits). Throws a TimeDependentDefaultError when neither exists — a schedule in a required field that never said what a fold-less solve should do.

Related

source
PortfolioOptimisers.assert_time_dependent_optimiser — Function
assert_time_dependent_optimiser(
    _::Union{NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult}
)

Assert that a TimeDependent schedule in an optimiser position resolved to something that can be optimised or predicted.

The vector and TimeDependentOptimiserCallable forms of a schedule declare their output kind in their type and are checked statically (see TD_OptE_Opt). The two callable forms — a bare ctx -> optimiser and a PreviousWeightsFunction wrapping one — cannot be, so their output is checked here, when the fold loop swaps it in.

Related

source
PortfolioOptimisers.assert_time_dependent_fold_count — Function
assert_time_dependent_fold_count(opt, n::Integer, all_binds::Bool = true)

Assert that every vector-valued time-dependent constraint in opt has exactly n entries.

Called by the cross-validation fold loops immediately after split, before any fold runs. The default is a no-op; hosts scan their time_dependent_fields and wrapper optimisers recurse. When all_binds is false, bind === :nearest schedules are skipped — they are validated by the nearest enclosing fold loop against its own fold count instead (see TimeDependent).

Related

source
PortfolioOptimisers.assert_time_dependent_fold_count — Function
assert_time_dependent_fold_count(
    td::Union{TimeDependent{<:AbstractVector{<:Union{Nothing, var"#s7096"} where var"#s7096"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult})}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}},
    n::Integer
)
assert_time_dependent_fold_count(
    td::Union{TimeDependent{<:AbstractVector{<:Union{Nothing, var"#s7096"} where var"#s7096"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult})}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}},
    n::Integer,
    all_binds::Bool
)

Assert that a TimeDependent schedule standing in for an optimiser has one entry per fold, and that the schedules within each entry are sized to the same fold loop.

Entry i runs at fold i of this loop, so its own :outermost schedules bind here too (see update_time_dependent_estimator) and are validated against this loop's fold count. The default is not — it runs only outside a fold loop, where its schedules reset instead.

Skipped when all_binds is false and the schedule is not :outermost-bound — the fold loop the host opens validates it against its own fold count instead.

Related

source
PortfolioOptimisers.assert_time_dependent_fold_count — Function
assert_time_dependent_fold_count(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}},
    n::Integer
)
assert_time_dependent_fold_count(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}},
    n::Integer,
    all_binds::Bool
)

Apply assert_time_dependent_fold_count element-wise to a vector of optimisation estimators or results.

Related

source
PortfolioOptimisers.rebuild_estimator — Function
rebuild_estimator(x, repl::NamedTuple) -> Pipeline

Rebuild an estimator through its keyword constructor with the fields in repl replaced.

All remaining fields are carried through unchanged. Because the rebuild goes through the validated keyword constructor, every construction invariant re-runs.

Related

source
rebuild_estimator(p::Pipeline, repl::NamedTuple)

Rebuilds a Pipeline with the fields in repl replaced, through the positional constructor: the names were built once from the steps, and a rebuild that seeds a state keeps them as they are, which the keyword constructor cannot express.

Related

source
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(opt)

Return true if the optimiser carries time-dependent constraints.

The default returns false. Hosts return true when any of their fields holds a TimeDependent (see time_dependent_fields); wrapper optimisers recurse into their inner optimiser and fallback.

Arguments

  • opt: Optimisation estimator, result, or vector thereof.

Returns

  • Bool: true if the estimator is time-dependent.

Related

source
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}}
) -> Any

Return true if any element of the vector of optimisation estimators or results is time-dependent.

Related

source
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}}
) -> Any

Return true if any element of the vector of optimisation estimators or results is time-dependent.

Related

source
PortfolioOptimisers.update_time_dependent_estimator — Function
update_time_dependent_estimator(
    opt::BaseOptimisationEstimator,
    ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
    opt::BaseOptimisationEstimator,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Resolve the time-dependent constraints of a base optimiser configuration for the fold described by ctx.

Rebuilds the configuration through its validated keyword constructor with each TimeDependent-valued field replaced by its resolved per-fold value, so the result is an ordinary static configuration. When all_binds is false, bind === :nearest fields are left in place for the nearest enclosing fold loop to consume.

Related

source
update_time_dependent_estimator(opt, ctx::TimeDependentContext, all_binds::Bool = true)

Resolve the time-dependent constraints of opt for the fold described by ctx.

The default returns the estimator unchanged. Hosts rebuild themselves through their validated keyword constructor with each TimeDependent-valued field replaced by its resolved per-fold value, so the result is an ordinary static estimator; wrapper optimisers recurse.

Arguments

  • opt: Optimisation estimator or result.
  • ctx::TimeDependentContext: The fold's context.
  • all_binds::Bool: When false, bind === :nearest schedules are skipped, leaving them for the nearest enclosing fold loop to consume. Meta-optimisers pass false when recursing into the estimators their internal fold loop processes; fold loops call with the default true.

Returns

  • Updated estimator.

Related

source
update_time_dependent_estimator(
    td::Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}},
    ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
    td::Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}},
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Resolve a TimeDependent schedule standing in for an optimiser to the optimiser of fold ctx.i.

Entry i may be an estimator or a precomputed result, so a mixed schedule optimises on some folds and predicts on others. After the swap the resolved estimator is recursed into with the same context, so its own :outermost schedules bind to this fold loop rather than going unresolved.

Returns the schedule unchanged when all_binds is false and it is not :outermost-bound — a :nearest schedule in an optimiser position is consumed by a fold loop the host itself opens, not by the loop that reached the host.

Related

source
update_time_dependent_estimator(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}},
    ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}},
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Apply update_time_dependent_estimator element-wise to a vector of optimisation estimators or results.

Related

source
update_time_dependent_estimator(
    opt::NaiveOptimisationEstimator,
    ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
    opt::NaiveOptimisationEstimator,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Resolve the time-dependent constraints of a naive optimiser for the fold described by ctx.

Rebuilds the optimiser through its validated keyword constructor with each TimeDependent-valued field replaced by its resolved per-fold value, recursing into the fallback, so the result is an ordinary static optimiser.

Related

source
update_time_dependent_estimator(
    opt::ClusteringOptimisationEstimator,
    ctx::TimeDependentContext
) -> NestedClustered
update_time_dependent_estimator(
    opt::ClusteringOptimisationEstimator,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> NestedClustered

Resolve time-dependent constraints for the fold described by ctx: the estimator's own scheduled fields (risk measures, scalarisers, fallback, …) are swapped for their per-fold values, then the inner optimiser and the (possibly just-swapped-in) fallback are recursed into with the same context.

NestedClustered overrides this with its own method resolving its own fields and inner estimators.

source
update_time_dependent_estimator(
    opt::JuMPOptimisationEstimator,
    ctx::TimeDependentContext
) -> Any
update_time_dependent_estimator(
    opt::JuMPOptimisationEstimator,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Resolve time-dependent constraints for the fold described by ctx: the estimator's own scheduled fields (risk measure, objective, warm start, fallback, …) are swapped for their per-fold values, then the inner JuMP optimiser and the (possibly just-swapped-in) fallback are recursed into with the same context.

source
update_time_dependent_estimator(
    opt::NestedClustered,
    ctx::TimeDependentContext
) -> NestedClustered
update_time_dependent_estimator(
    opt::NestedClustered,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> NestedClustered

Resolve time-dependent constraints for the fold described by ctx by recursing into the inner optimiser, outer optimiser, and fallback.

source
update_time_dependent_estimator(
    opt::Stacking,
    ctx::TimeDependentContext
) -> Stacking
update_time_dependent_estimator(
    opt::Stacking,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Stacking

Resolve time-dependent constraints for the fold described by ctx by recursing into the inner optimisers, outer optimiser, and fallback.

source
update_time_dependent_estimator(
    opt::SubsetResampling,
    ctx::TimeDependentContext
) -> SubsetResampling
update_time_dependent_estimator(
    opt::SubsetResampling,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> SubsetResampling

Resolve time-dependent constraints for the fold described by ctx by recursing into the base optimiser and fallback.

source
update_time_dependent_estimator(
    p::Pipeline,
    ctx::TimeDependentContext
) -> Pipeline
update_time_dependent_estimator(
    p::Pipeline,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Pipeline

Resolve the time-dependent steps of a Pipeline for the fold described by ctx.

The swap happens in the fold loop, outside fit entirely: it maps update_time_dependent_step over the steps (names preserved), so by the time fit runs on the fold's training window every schedule step is already a plain optimiser or precomputed result and injection (inject_context / maybe_inject_step) never sees a schedule — fit and run_step never learn about folds.

Related

source
PortfolioOptimisers.update_time_dependent_fields — Function
update_time_dependent_fields(
    opt,
    ctx::TimeDependentContext
) -> Any
update_time_dependent_fields(
    opt,
    ctx::TimeDependentContext,
    all_binds::Bool
) -> Any

Rebuild a host optimiser with each TimeDependent-valued field replaced by its per-fold value for ctx.

Shared implementation behind the hosts' update_time_dependent_estimator methods. Returns opt unchanged when no field is time-dependent.

Related

source
PortfolioOptimisers.reset_time_dependent_estimator — Method
reset_time_dependent_estimator(opt)

Replace every TimeDependent-valued field of opt with its static default, recursing through wrapper optimisers.

A time-dependent constraint is defined only over the folds of a cross-validation scheme, so a fold-less solve runs with the affected fields at their static defaults (see time_dependent_field_defaults). Called at the top of the _optimise methods; per-fold estimators produced by update_time_dependent_estimator contain no TimeDependent values, so they pass through unchanged. The default returns the estimator unchanged; hosts rebuild themselves, wrapper optimisers recurse.

Related

source
PortfolioOptimisers.reset_time_dependent_estimator — Method
reset_time_dependent_estimator(
    td::Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}
) -> Any

Return the optimiser a TimeDependent schedule takes outside every fold loop.

An optimiser position is required — there is no static default to fall back to — so the schedule must supply its own default, and one that does not throws a TimeDependentDefaultError. The fold-less optimiser is itself reset, so its own schedules resolve to their defaults too.

Related

source
PortfolioOptimisers.finalise_weight_bounds — Function
finalise_weight_bounds(wf::WeightFinaliser, wb::WeightBounds, w::VecNum)

Apply weight finalisation to enforce bounds and determine the optimisation return code.

Runs opt_weight_bounds with the given finaliser and bounds, then returns a success or failure return code based on whether all weights are finite.

Finiteness is the whole test. A vector that still violates the bounds — which an IterativeWeightFinaliser returns when it exhausts its passes — is reported as an OptimisationSuccess.

Arguments

  • wf::WeightFinaliser: Weight finaliser algorithm.
  • wb::WeightBounds: Weight bounds configuration.
  • w::VecNum: Portfolio weights to finalise.

Returns

  • (retcode, w): Tuple of return code and adjusted weights.

Related

source
PortfolioOptimisers.investable_reduction — Method
investable_reduction(
    pr::AbstractPriorResult,
    opt::AbstractOptimisationEstimator,
    rd::ReturnsResult
) -> Union{Tuple{BitVector, Union{HighOrderPrior{<:AbstractPriorResult}, LowOrderPrior{var"#s185", _A, var"#s1851", <:AbstractMatrix{var"#s137"}} where {var"#s137"<:(Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}), var"#s185"<:AbstractMatrix{var"#s137"}, _A, var"#s137"<:(Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}), var"#s1851"<:AbstractVector{var"#s137"}, var"#s137"<:(Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar})}}, Any, Union{ReturnsResult{_A, _B, _C, _D, _E, _F, _G, _H, _I, Nothing} where {_A, _B, _C, _D, _E, _F, _G, _H, _I}, ReturnsResult{_A, _B, _C, _D, _E, _F, _G, _H, _I, AssetPanel{__T_pf, __T_amsk, __T_emsk}} where {_A, _B, _C, _D, _E, _F, _G, _H, _I, __T_pf, __T_amsk, __T_emsk}}}, Tuple{Nothing, AbstractPriorResult, AbstractOptimisationEstimator, ReturnsResult}}

Derive the Investable Mask of a fitted prior, and reduce the prior, the optimiser and the returns data to the assets it keeps.

A Prior Estimator fits on the coverage universe and hands back a result on the full asset universe, in which an asset it could not estimate carries NaN in mu and on the diagonal of sigma. Nothing downstream of the fit can solve over such an asset, so the reduction happens once, at the optimiser's entry, and every constraint the caller stated over the full universe is sliced by the same index.

Every optimisation family reduces through this one verb, which is why it is bound to AbstractOptimisationEstimator and lives here rather than beside the JuMP prelude: the hierarchical, naive and meta files load before that prelude and reach it without a back reference. A meta-optimiser composes two masks, its own at its entry and each inner head's inside its own solve, and both expand.

Three methods, and the branch is dispatch rather than a condition. The first derives the mask; the nothing method is the all-investable path and returns its arguments untouched; the BitVector method takes the three views. A universe with nothing to exclude therefore costs one pass over two vectors and no allocation.

The optimiser is viewed at pr.X, the unreduced returns matrix, because port_opt_view slices a tracking estimator against it by the same asset index. The prior is reduced after, so the matrix the view reads is still the full one.

The mask rides as a BitVector because the expansion needs the length of the full universe and nothing else carries it once the prior is reduced. The three views take findall(imsk) instead, which is the integer index every other caller of port_opt_view passes.

Algorithm

  1. Derive the Investable Mask from the fitted prior with investable_mask.
  2. Return the mask, the prior, the optimiser and the returns data unchanged when the mask is nothing.
  3. Otherwise read the departed names off the unreduced rd.nx with non_investable_names, and announce them once with announce_non_investable.
  4. Take a port_opt_view of each of the three at findall(imsk).
  5. Declare the Non-Investable Axis on the viewed optimiser with non_investable_universe, and return it beside the mask and the other two views. The axis is declared after the view because the view drops one, so a name-keyed constraint stated for a departed asset resolves here and nowhere deeper.

Arguments

  • pr: Prior result.
  • opt::AbstractOptimisationEstimator: The optimisation estimator, holding every constraint estimator the caller stated over the full universe.
  • rd: The returns result to use.

Returns

  • (imsk, pr, opt, rd): The Investable Mask and the three reduced to it, or nothing and the three unchanged.

Related

source
PortfolioOptimisers.coverage_reduction — Method
coverage_reduction(
    opt::AbstractOptimisationEstimator,
    rd::ReturnsResult;
    dims
) -> Union{Tuple{BitVector, Any, Union{ReturnsResult{_A, _B, _C, _D, _E, _F, _G, _H, _I, Nothing} where {_A, _B, _C, _D, _E, _F, _G, _H, _I}, ReturnsResult{_A, _B, _C, _D, _E, _F, _G, _H, _I, AssetPanel{__T_pf, __T_amsk, __T_emsk}} where {_A, _B, _C, _D, _E, _F, _G, _H, _I, __T_pf, __T_amsk, __T_emsk}}}, Tuple{Nothing, AbstractOptimisationEstimator, ReturnsResult}}

Derive the Coverage Universe of a window, and reduce a prior-free optimiser and its returns data to the assets it keeps.

This is the twin of investable_reduction for a head that fits no prior. No Prior Result exists to derive an Investable Mask from, so the head derives the Coverage Universe of its own window instead, through the verb the priors use: an asset is kept when its return is finite and the active mask of the AssetPanel is true at every row of the window. A stale finite price during an inactive spell is therefore outside the mask, and the asset weights nothing.

The two masks are the same object downstream. The head carries the Coverage Universe as the imsk of its result, and the result's keyword constructor expands the weights through expand_investable_weights, as every other family's does, so every optimisation result of the library carries a mask and a reader has one idiom.

Three methods, and the branch is dispatch rather than a condition, as it is in investable_reduction. The first derives the mask; the nothing method is the all-covered path and returns its arguments untouched; the BitVector method takes the two views. The optimiser is viewed at rd.X, the unreduced returns matrix, because port_opt_view slices an estimator against it by the same asset index.

An all-dead window throws an IsEmptyError where the mask is derived, so the refusal is coverage_mask's and every prior-free head has it for free.

Algorithm

  1. Derive the Coverage Universe of rd.X and rd.pnl with coverage_mask.
  2. Return the mask, the optimiser and the returns data unchanged when the mask is nothing.
  3. Otherwise read the departed names with non_investable_names, announce them once with announce_non_investable, take a port_opt_view of each of the two at findall(cmsk), and declare the Non-Investable Axis on the viewed optimiser with non_investable_universe.

Arguments

  • opt::AbstractOptimisationEstimator: The optimisation estimator, holding every constraint estimator the caller stated over the full universe.
  • rd: The returns result to use.
  • dims: Dimension along which to perform the computation.

Validation

  • dims in (1, 2).
  • At least one asset must be in the Coverage Universe.

Returns

  • (cmsk, opt, rd): The Coverage Universe and the two reduced to it, or nothing and the two unchanged.

Related

source
PortfolioOptimisers.expand_investable_weights — Function
expand_investable_weights(
    _::Nothing,
    w::Union{Nothing, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}}}
) -> Union{Nothing, AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}, AbstractVector{<:AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}}}

Expand a solved weight vector from the investable subset back onto the full asset universe.

The optimiser solves over the assets the Investable Mask keeps, so its weight vector is shorter than the universe the caller stated. This puts each solved weight back at its own asset and writes a zero everywhere else, which is what a non-investable asset holds: the optimiser could not trade it.

A failed solve carries NaN at every solved position. The expansion keeps that distinction — NaN where the optimiser tried and failed, zero where it never could — rather than flattening both to zero.

The nothing mask returns the weights unchanged, so nothing is copied when every asset is investable, and a nothing weight vector stays nothing, which is what a naive head records when its finaliser gave up. The vector-of-vectors method serves the efficient-frontier route, where one weight vector is recorded per sweep point. JuMPOptimisationSolution carries its own methods beside the JuMP prelude, and they delegate to the plain weight vector here, so one length check and one message serve every family.

The keyword constructor of each optimisation result is the caller, and every family builds its result through it. The positional constructor never expands, because every return-code rebuild goes through it and a second pass would expand twice.

Arguments

  • imsk: The Investable Mask the optimisation reduced on: true at every asset whose prior moments were finite. It is nothing when every asset was investable, and that sentinel is what skips both the reduction and the expansion. investable_mask derives it once from the full-universe prior result, and the result carries it, because the reduced prior can no longer yield it.
  • w: The weights the optimisation solved over the investable universe: one weight vector, or a vector of them on the efficient-frontier route.

Validation

  • The solved weight vector must hold one weight per investable asset.

Returns

  • w: The weights, or the vector of them, on the full asset universe.

Related

source
PortfolioOptimisers.assert_special_nco_requirements — Function
assert_special_nco_requirements(opt)

Assert that the optimiser meets special requirements for Nested Clustered Optimisation (NCO).

The default implementation does nothing. Overridden for estimators (e.g. Stacking) that have requirements which must be validated before NCO can proceed.

Arguments

  • opt: Optimisation estimator, result, or vector thereof.

Returns

  • nothing.

Related

source
assert_special_nco_requirements(
    opt::AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:(Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}), var"#s7099"<:Union{TimeDependent{<:AbstractVector{<:Union{var"#s7100", var"#s7099"} where {var"#s7100"<:NonFiniteAllocationOptimisationEstimator, var"#s7099"<:NonFiniteAllocationOptimisationResult}}}, TimeDependent{<:TimeDependentOptimiserCallable}, TimeDependent{<:PreviousWeightsFunction}, TimeDependent{<:Union{Function, Type}}}}}
)

Assert special NCO requirements for each element of a vector of optimisation estimators or results.

Related

source
PortfolioOptimisers.assert_no_nearest_bind_optimiser_schedule — Method
assert_no_nearest_bind_optimiser_schedule(x, field::Symbol, host::Symbol)

Reject a bind = :nearest TimeDependent schedule in an optimiser-valued position no inner fold loop consumes.

bind picks which fold loop supplies the schedule's index. :nearest therefore says something different from :outermost only where the host opens a fold loop of its own and hands the field across it — NestedClustered.opti (its inner cross-validation is entered per cluster) and Stacking.opti[k] (entered per candidate), the positions declared by inner_fold_fields. Everywhere else the loop that reaches the host is the nearest one, and the two binds would name the same loop.

The positions this guards have no such inner loop:

  • A fallback (fb), on every host. The fallback walk is a retry chain within a single fold's solve — it has no fold indices of its own — so :nearest there is either redundant with :outermost or, behind a meta's inner cross-validation, silently wrong: it would resolve against the inner loop's fold numbers (tuning folds) instead of the backtest's periods, changing meaning with nesting depth. A per-fold fallback is fully expressible with :outermost, including nothing entries to switch it off on some folds (see TDO_OptE_Opt).
  • The outer optimisers (opto). They consume the combined inner output, once per solve.
  • SubsetResampling.opt. Its internal loop is over randomly drawn asset subsets, not time folds.

So a :nearest schedule in any of them has no nearest fold loop to bind to, and is rejected at construction rather than resolving against a loop the caller did not mean. No-op for anything that is not a TimeDependent.

Related

source
PortfolioOptimisers.entitled — Function
entitled(opt, f::Symbol, all_binds::Bool)

Return true when the recursion position described by all_binds may consume a bind = :nearest schedule in field f of opt.

Entitlement is per-field, not per-host: a loop scanning with all_binds = true takes :nearest schedules everywhere except in the fields the host hands across its own inner fold loop (see inner_fold_fields) — for those, the host's inner loop is the nearest one, whatever loop is doing the scanning. A field is taken by a pass iff entitled(opt, f, all_binds) || bind === :outermost.

Related

source
PortfolioOptimisers.extract_fees — Function
extract_fees(res::OptimisationResult) -> Any
extract_fees(
    res::OptimisationResult,
    fees::Union{Nothing, Fees}
) -> Any

Obtains the fees to use for net return calculations from an optimisation result.

An explicitly provided fees wins. Otherwise the fees are read from the fees property of res, and a result exposing no such property gives nothing.

Arguments

  • res: Optimisation result, potentially containing a fees property.
  • fees: Optional fees to use, which take precedence over res.fees if provided.

Returns

  • Option{<:Fees}: The fees to use for net return calculations, or nothing if not found.

A result's fee sits on the universe the fit solved, so a consumer that pairs it with res.w, which is on the caller's, reads both through result_investable_view rather than through this verb alone.

Related

source
PortfolioOptimisers.extract_pr — Function
extract_pr(res::OptimisationResult) -> Any
extract_pr(
    res::OptimisationResult,
    pr::Union{Nothing, AbstractPriorResult, ReturnsResult}
) -> Any

Extracts the prior result for risk calculation from an optimisation result.

An explicitly provided pr wins. Otherwise the one test is hasproperty(res, :pr), which property forwarding answers for a nested result: a JuMP leaf reaches its prior at res.jr.pa.pr, and res.pr resolves to it. A result that exposes no pr property throws.

Arguments

  • res: Optimisation result, which carries a prior result as its pr property or reaches one by property forwarding.
  • pr: Optional prior result to use for risk calculation, which takes precedence over the one found in res.

Returns

  • pr::Pr_RR: The prior result to use for risk calculation. Throws an ArgumentError when none is found.

A result's prior is on the universe the fit solved, so a consumer that pairs it with res.w, which is on the caller's, reads both through result_investable_view rather than through this verb alone.

Related

source
PortfolioOptimisers.result_investable_carrier — Function
result_investable_carrier(imsk, res::OptimisationResult, pr::Nothing)
result_investable_carrier(imsk::Nothing, res::OptimisationResult, pr::MatNum_Pr)
result_investable_carrier(imsk::BitVector, res::OptimisationResult, pr::Pr_RR)
result_investable_carrier(imsk::BitVector, res::OptimisationResult, X::MatNum)

Resolve the returns carrier a result-taking consumer reads: the result's own prior, or a caller's viewed at the result's Investable Mask.

The carrier half of result_investable_view. The result's own prior is on the universe the fit solved, so it is read through extract_pr and taken as it is, and so is that prior when a caller hands it back, expected_risk(r, res, res.pr): it is the one carrier already on the investable universe, and it is recognised by identity, never by width, because a reduced and a full-universe carrier share a width only by coincidence. Every other caller's carrier is on the caller's universe, the one res.w spans, so under a BitVector mask it is viewed at findall(imsk): a prior result or a ReturnsResult through the port_opt_view its owner writes, which is the rule fold_factor_returns applies to a caller's rd on a fold, and a bare matrix by its columns. Under a nothing mask nothing left, and a caller's carrier is returned unchanged.

Arguments

  • imsk: The Investable Mask of res, or nothing.
  • res::OptimisationResult: Fitted optimisation result.
  • pr: A caller's carrier on the universe of res.w, or nothing.

Returns

  • The carrier on the investable universe of res.

Related

source
PortfolioOptimisers.result_investable_fees — Function
result_investable_fees(imsk, res::OptimisationResult, fees::Nothing)
result_investable_fees(imsk::Nothing, res::OptimisationResult, fees::Fees)
result_investable_fees(imsk::BitVector, res::OptimisationResult, fees::Fees)

Resolve the fee a result-taking consumer charges: the result's own, or a caller's viewed at the result's Investable Mask.

The fee half of result_investable_view, and the rule fold_fees applies on a fold. The result's own fee was reduced at the fit's door, its five per-asset fields to the mask and its two liquidation carriers to the complement, and is marked with that mask, so it is read through extract_fees and taken as it is. A caller's fee is stated on the caller's universe, so it takes the same door a fee takes at the fit, investable_fees_view, which slices the per-asset fields to the mask and the carriers to its complement, and strips the carriers under a nothing mask, because nothing left. The door needs the width of the unreduced universe to derive the complement, and a result holds no full-width matrix, so the width is the mask's own length.

Arguments

  • imsk: The Investable Mask of res, or nothing.
  • res::OptimisationResult: Fitted optimisation result.
  • fees: A caller's Fees on the universe of res.w, or nothing.

Returns

  • fees::Option{<:Fees}: The fee on the two axes the result's mask leaves, or nothing.

Related

source
PortfolioOptimisers.result_investable_view — Function
result_investable_view(res::OptimisationResult, pr = nothing, fees = nothing, nx = nothing)
result_investable_view(imsk::Nothing, res::OptimisationResult, pr, fees, nx)
result_investable_view(imsk::BitVector, res::OptimisationResult, pr, fees, nx)

Pair the weights, the returns carrier, the fee and the axis names a result-taking consumer reads, on the investable universe of the result.

A result carries three things on two universes. An optimisation reduces to its Investable Mask at the entry and expands the solved weights back, so res.w is on the full universe the caller stated, while res.pr is the prior of the universe the fit solved, and res.fees was reduced at the same door, its five per-asset fields to the mask and its two liquidation carriers to the complement. A consumer that reads two of the three separately pairs a full vector with a reduced one: a per-asset fee indexed at a full-length mask raises a BoundsError, a reduced returns matrix against a full weight vector a DimensionMismatch, and a reduced mu under the caller's full names draws every bar after the gap under the name of the asset before it. This verb is the one place the pairing is made, so every result-taking arity of expected_risk, calc_net_returns, factor_attribution, performance_summary and the plotting extension reads all three through it.

The answers are on the investable universe of res, which is the universe its own prior and fee are already on. The weights are viewed at the mask through investable_weights_view. The carrier is resolved by result_investable_carrier and the fee by result_investable_fees: the result's own are taken as they are, and a caller's, stated on the universe res.w spans, is viewed at the mask, the carrier through the port_opt_view its owner writes and the fee through the door a fee takes at the fit. The axis names ride the asset axis, so they take the mask directly, and a drawn figure labels each bar with its own asset.

A result whose mask is nothing reduced on nothing, so the weights and the names are returned unchanged. A per-asset answer a consumer forms on the investable universe expands back through expand_investable_weights with the mask this verb returns first, as the value-level doors do.

Algorithm

  1. Read the Investable Mask of res with result_investable_mask.
  2. On a nothing mask, return nothing, res.w, the carrier through result_investable_carrier, the fee through result_investable_fees, and nx unchanged.
  3. On a BitVector mask, return it, the view of res.w at it, the carrier and the fee through the same two verbs, and the view of nx at it.

Arguments

  • res::OptimisationResult: Fitted optimisation result.
  • pr: A caller's returns carrier on the universe of res.w, a prior result, a ReturnsResult or a returns matrix, or nothing to read the result's own prior.
  • fees: A caller's Fees on the universe of res.w, or nothing to read the result's own.
  • nx: Asset names on the universe of res.w, or nothing.
  • imsk: The Investable Mask of res, or nothing.

Returns

  • (imsk, w, pr, fees, nx): The Investable Mask, and the weights, the carrier, the fee and the names on the investable universe of res.

Related

source
PortfolioOptimisers.synthetic_asset_weights — Function
synthetic_asset_weights(
    w::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}
) -> Any

Normalises inner weights into the convex weights that collapse real assets onto a meta-optimiser's synthetic assets.

Quantities carried alongside the returns matrix are either extensive (returns, benchmark returns) and collapse as a plain weighted sum w'x, or intensive (rates such as rd.iv and rd.ivpa) and collapse as a weighted average. A plain weighted sum scales an intensive quantity by the gross exposure sⱼ = Σᵢ|wᵢⱼ|, so a shorting or leveraged portfolio (sⱼ ≠ 1) inflates a rate that should not depend on gross exposure at all.

Normalising the weights once here makes every subsequent product a convex combination, so callers collapsing an intensive quantity need only pass their weights through this function.

Arguments

  • w: Inner weights. A vector collapses onto a single synthetic asset; a matrix (assets × synthetic assets) collapses each column independently.

Returns

  • w: abs.(w), with each column scaled to sum to one. A column summing to zero — a degenerate synthetic asset — is left as-is rather than divided, preserving the zero row it already produced.

Related

source
PortfolioOptimisers.collapse_asset_panel — Function
collapse_asset_panel(pnl::Nothing, wi::MatNum, nx) -> nothing
collapse_asset_panel(pnl::AssetPanel, wi::MatNum, nx::Option{<:VecStr}) -> AssetPanel

Aggregate an AssetPanel onto the synthetic assets a meta-optimiser builds for its outer problem.

A meta-optimiser's outer problem allocates across synthetic assets — NestedClustered's clusters, Stacking's inner portfolios — each of which is a weighted combination of the real ones. Every quantity the outer ReturnsResult carries has to be re-expressed on that universe, and the panel is no exception: without this collapse the outer optimiser has no panel at all, so a FeatureDistance there throws rather than clustering the synthetic universe.

Features are treated as intensive, exactly as iv and ivpa are: the collapse is a convex combination, obtained by pushing the inner weights through synthetic_asset_weights first. An un-normalised weighted sum would scale each synthetic asset's feature vector by its gross exposure sⱼ = Σᵢ|wᵢⱼ|, inflating it under leverage or shorting. Under the default AngularDist the normalisation is a mathematical no-op for a rectangular field — scaling one row of the result leaves every cosine unchanged — but it is not one in the square case, where the two-sided product rescales the label axis as well, and it is what keeps the collapse bounded for any sⱼ > 0. An extensive feature (a market capitalisation, a headcount) wanting a weighted sum is not supported: the divisor depends on the inner solve, so a caller cannot pre-scale their way to one.

Degenerate synthetic assets

A synthetic asset whose weights are entirely zero has sⱼ = 0; synthetic_asset_weights leaves the column alone rather than dividing, so the collapse gives that asset a zero feature vector instead of throwing. It then lands on the zero-feature-vector convention the distance kernel already implements, matching the zero returns column, iv and ivpa the same degenerate weights already produce.

Algorithm

  1. Return nothing when the carrier holds no panel.
  2. Normalise the inner weights with synthetic_asset_weights.
  3. Collapse every Panel Field with collapse_panel_field.
  4. Collapse both universe masks with collapse_panel_mask, which keeps them nothing for a static panel.

Arguments

  • pnl: The Asset Panel, or nothing.
  • wi: Inner weights, assets × synthetic assets.
  • nx: The carrier's asset names, or nothing. Read for the square case alone.

Returns

  • pnl::Option{AssetPanel}: The Asset Panel on the synthetic universe, or nothing.

Related

source
PortfolioOptimisers.collapse_panel_field — Function
collapse_panel_field(f::NumericPanelField, W, nx, syn) -> NumericPanelField
collapse_panel_field(f::CategoricalPanelField, W, nx, syn) -> TensorPanelField
collapse_panel_field(f::TensorPanelField, W, nx, syn) -> TensorPanelField

Collapse one Panel Field onto the synthetic assets a meta-optimiser builds.

The collapse acts one field at a time and returns a field, so the collapsed panel is a panel like any other and a selector written for the inner problem resolves on it unchanged.

  • A numeric field stays numeric.
  • A tensor field stays a tensor field of the same name and labels, one label at a time. When its labels are the asset names the contraction is two-sided, its labels are renamed after the synthetic assets and its groups are dropped, so the square case holds one level up.
  • A categorical field becomes a tensor field of membership fractions: the same name, the axis "level", the levels as labels, and the convex combination of its one-hot block as values. A one-hot column of the Feature Matrix is a 0/1 feature, so its convex combination is the share of the synthetic asset's weight in that level, and the collapsed panel's Feature Matrix equals the collapse of the original panel's. A convex combination of integer codes would mean nothing, and a majority level would lose the fractions and need a tie rule.

Algorithm

The method that Julia selects is the algorithm. Each kind contracts its own values and its own observed mask, through collapse_panel_numeric, collapse_panel_tensor and collapse_panel_mask.

Arguments

  • f: The Panel Field.
  • W: Normalised inner weights, assets × synthetic assets.
  • nx: The carrier's asset names, or nothing. Read for the square case alone.
  • syn: The synthetic asset names.

Returns

  • The collapsed Panel Field.

Related

source
PortfolioOptimisers.collapse_panel_numeric — Function
collapse_panel_numeric(A::AbstractVector, W::MatNum) -> Vector
collapse_panel_numeric(A::AbstractMatrix, W::MatNum) -> Matrix

Collapse one numeric Panel Field array onto the synthetic assets, as a convex combination.

W is the normalised weight matrix synthetic_asset_weights returns, assets × synthetic assets. A static array is one value per asset and a time-varying one is observations × assets, so the asset axis is the only one contracted in both.

Algorithm

The method that Julia selects is the algorithm. A vector contracts as transpose(W) * A, and a matrix as A * W.

Arguments

  • A: The values, assets or observations × assets.
  • W: Normalised inner weights, assets × synthetic assets.

Returns

  • The collapsed values, synthetic assets or observations × synthetic assets.

Related

source
PortfolioOptimisers.collapse_panel_tensor — Function
collapse_panel_tensor(A::AbstractMatrix, W::MatNum, sq::Bool) -> Matrix
collapse_panel_tensor(A::AbstractArray{<:Any, 3}, W::MatNum, sq::Bool) -> Array

Collapse one tensor Panel Field array onto the synthetic assets, as a convex combination.

A tensor array is assets × labels when static and observations × assets × labels when time-varying. The asset axis is always contracted. When sq is true the label axis is the asset axis (features_are_assets), so it is contracted too and the result is square again on the synthetic universe.

Algorithm

The method that Julia selects is the algorithm. A matrix contracts as transpose(W) * A, and again as * W when sq. A three-dimensional array does the same one observation at a time, into a preallocated result.

Arguments

  • A: The values.
  • W: Normalised inner weights, assets × synthetic assets.
  • sq: Whether the label axis is the asset axis.

Returns

  • The collapsed values, with the asset axis, and the label axis under sq, replaced by the synthetic assets.

Related

source
PortfolioOptimisers.collapse_panel_mask — Function
collapse_panel_mask(m::Nothing, W::MatNum) -> nothing
collapse_panel_mask(m::AbstractVector{Bool}, W::MatNum) -> BitVector
collapse_panel_mask(m::AbstractMatrix{Bool}, W::MatNum) -> BitMatrix

Collapse one mask onto the synthetic assets, as the support of its convex combination.

A synthetic asset is observed, active or in estimation at an observation when any member carrying weight is. So one kernel serves the values and the masks, the mask stays Bool by type, and the subset invariant between the estimation mask and the active mask survives with no second check.

Algorithm

The method that Julia selects is the algorithm. nothing stays nothing; otherwise the mask collapses through collapse_panel_numeric and the result is compared against zero.

Arguments

  • m: The mask, or nothing.
  • W: Normalised inner weights, assets × synthetic assets.

Returns

  • The collapsed mask, or nothing.

Related

source
PortfolioOptimisers.collapse_categorical_mask — Function
collapse_categorical_mask(m::Nothing, W::MatNum, nl::Integer) -> nothing
collapse_categorical_mask(m::AbstractVector{Bool}, W::MatNum, nl::Integer) -> BitMatrix
collapse_categorical_mask(m::AbstractMatrix{Bool}, W::MatNum, nl::Integer) -> BitArray

Collapse a categorical Panel Field's observed mask onto the tensor field its collapse returns.

The collapsed field carries one label per level, so its mask needs the level axis the categorical mask does not have. The asset mask is collapsed once and then repeated across the levels: a cell was observed or not for the whole label, never per level.

Algorithm

The method that Julia selects is the algorithm. nothing stays nothing; otherwise the asset mask collapses through collapse_panel_mask and is repeated over nl levels.

Arguments

  • m: The categorical field's observed mask, or nothing.
  • W: Normalised inner weights, assets × synthetic assets.
  • nl: Number of levels.

Returns

  • The collapsed mask over the level axis, or nothing.

Related

source
PortfolioOptimisers.non_investable_universe — Function
non_investable_universe(opt::AbstractOptimisationEstimator, ni::VecStr)
non_investable_universe(opt, ni::VecStr)

Mint the Non-Investable Axis on the UniverseSets an optimisation estimator carries.

A door calls this after it has viewed the estimator, and it is the only caller. Minting before the view would not survive it: port_opt_view(::UniverseSets, i) drops the axis, precisely so that a cluster of a nested optimisation cannot inherit its parent's departures and charge every one of them again. So the door reads the departed names off the unreduced returns data, takes the view, and declares the axis on what comes back.

The generic method returns opt untouched, and it is the right answer for every estimator that carries no sets: there is no axis to declare one on. A head that owns a sets field writes one method beside its own port_opt_view, and a head that reaches one through a nested optimiser forwards to that optimiser's method. Both are one line, and they are written per type rather than derived by reflection for the reason port_opt_view is: a field's meaning is the type's to state.

Arguments

  • opt: The optimisation estimator, already reduced to the Investable Mask.
  • ni: The names the mask left out, from non_investable_names.

Returns

  • opt: The estimator, carrying the Non-Investable Axis where it carries a UniverseSets.

Related

source