GitHub

Base Prior: private API

PortfolioOptimisers.AbstractLowOrderPriorEstimator — Type
abstract type AbstractLowOrderPriorEstimator <: AbstractPriorEstimator

Abstract supertype for low order prior estimators.

AbstractLowOrderPriorEstimator is the base type for estimators that compute low order moments (mean and covariance) from asset and/or factor returns. All concrete low order prior estimators should subtype this type for consistent moment estimation and integration. A member of this family returns a LowOrderPrior, never a bare tuple of moments, so every consumer reads one carrier. It does not subtype this type directly: it subtypes the one of AbstractLowOrderPriorEstimator_A, AbstractLowOrderPriorEstimator_F and AbstractLowOrderPriorEstimator_AF that names the returns it reads.

Related

source
PortfolioOptimisers.AbstractLowOrderPriorEstimator_A_AF — Type
const AbstractLowOrderPriorEstimator_A_AF = Union{<:AbstractLowOrderPriorEstimator_A,
                                                  <:AbstractLowOrderPriorEstimator_AF}

Union type for asset-only and asset-and-factor low order prior estimators.

A field typed AbstractLowOrderPriorEstimator_A_AF admits the asset-only and the optional-factor shapes, and excludes the shape that requires factor returns. That is the bound for a wrapper which fits its nested estimator on one returns matrix it supplies itself: FactorPrior and FactorBlackLittermanPrior fit pe on the factor returns alone, and AugmentedBlackLittermanPrior fits a_pe on the assets and f_pe on the factors. A nested estimator that demanded a second matrix would have nothing to be handed, so the bound refuses it at construction rather than at the call.

Related

source
PortfolioOptimisers.AbstractLowOrderPriorEstimator_F_AF — Type
const AbstractLowOrderPriorEstimator_F_AF = Union{<:AbstractLowOrderPriorEstimator_F,
                                                  <:AbstractLowOrderPriorEstimator_AF}

Union type for factor-only and asset-and-factor low order prior estimators.

A field typed AbstractLowOrderPriorEstimator_F_AF admits the factor-requiring and the optional-factor shapes, and excludes the asset-only shape. That is the bound for a wrapper which forwards both returns matrices down and needs the result to be able to carry a factor block, as HighOrderFactorPriorEstimator does. The bound constrains what the nested estimator consumes, not what its result produces: the optional-factor half may still return a result with rr === nothing, so a consumer that reads the loadings guards with assert_prior_regression.

Related

source
PortfolioOptimisers.AbstractLowOrderPriorEstimator_A_F_AF — Type
const AbstractLowOrderPriorEstimator_A_F_AF = Union{<:AbstractLowOrderPriorEstimator_A,
                                                    <:AbstractLowOrderPriorEstimator_F,
                                                    <:AbstractLowOrderPriorEstimator_AF}

Union type for asset-only, factor-only, and asset-and-factor low order prior estimators.

A field typed AbstractLowOrderPriorEstimator_A_F_AF admits all three source shapes, and excludes nothing below the low order root. That is the bound for a wrapper which passes the returns matrices it was handed straight through, so the nested estimator meets exactly the arguments the caller supplied and the shape is its own affair: EntropyPoolingPrior bounds pe this way. The union is written out rather than spelled AbstractLowOrderPriorEstimator so that the three shapes are named at every field that admits them, and so that a fourth shape added later reaches this bound only by a deliberate edit.

Related

source
PortfolioOptimisers.AbstractHighOrderPriorEstimator — Type
abstract type AbstractHighOrderPriorEstimator <: AbstractPriorEstimator

Abstract supertype for high order prior estimators.

AbstractHighOrderPriorEstimator is the base type for estimators that compute high order moments (such as coskewness and cokurtosis) from asset and/or factor returns. All concrete high order prior estimators should subtype this type to ensure a consistent interface for higher moment estimation and integration with portfolio optimisation workflows.

A member of this family returns a HighOrderPrior, which wraps the LowOrderPrior its own nested low order estimator produced. So a high order estimator adds an order rather than replacing one, and every low order name stays readable through the wrapper.

Related

source
PortfolioOptimisers.Pr_RR — Type
const Pr_RR = Union{<:AbstractPriorResult, <:ReturnsResult}

Groups the two carriers that hold an asset returns matrix X and a feature matrix Z.

Pr_RR is the bridge the clustering, phylogeny and centrality forwarders below dispatch on. Each of them reads X off its carrier and delegates to the asset-returns method, so an estimator that needs returns can be driven from a fitted prior or from the raw data with one method apiece rather than two. Where both carriers are present, returns_matrix_picker picks between them; both travel on to the estimator tree as pr and rd, so a FeatureDistance resolves its Asset Panel from them.

Related

source
PortfolioOptimisers.PrE_Pr — Type
const PrE_Pr = Union{<:AbstractPriorEstimator, <:AbstractPriorResult}

Groups a prior estimator with an already-fitted prior result.

PrE_Pr is the bound of every optimiser's pe slot, and it is what lets a caller hand an optimiser a prior it has already fitted instead of the recipe for fitting one. The two are interchangeable there because prior has a method on each: the estimator method fits, and the result method returns its argument unchanged. So the optimiser calls prior once and never branches on which kind it holds.

Related

source
PortfolioOptimisers.assert_prior_regression — Function
assert_prior_regression(pr::AbstractPriorResult; ...)
assert_prior_regression(
    pr::AbstractPriorResult,
    sym::Union{AbstractString, Symbol};
    lead
)

Assert that a prior result carries a factor block, so its loadings can be read.

Estimators whose pe field is typed AbstractLowOrderPriorEstimator_F_AF accept the AbstractLowOrderPriorEstimator_AF half of that union, whose members use factor returns only optionally. The type therefore constrains which returns an estimator consumes, not whether the result it produces carries a regression. An estimator that projects factor moments through the loadings needs the latter, and must check for it.

There is one way to arrive with pr.rr === nothing: nothing in the chain ever computed a regression (EntropyPoolingPrior(; pe = EmpiricalPrior())). Discarding one is no longer possible — every wrapping estimator forwards rr and the factor block fpr, so nesting order does not matter. Checking rr covers the whole factor block, because LowOrderPrior already requires rr and fpr to be provided together or not at all — which is why the plotting entry points that want fpr.mu or fpr.sigma check rr here rather than testing the virtual read they are about to take.

Estimators are not the only consumer: the factor-space plotting entry points need the same block, and get the same diagnosis. Only the opening sentence differs, so lead carries it and prior_regression_remedy carries the rest.

Arguments

  • pr: Prior result handed to the consumer.
  • sym: Name of the field or argument the result arrived through, used in the error message.
  • lead: Opening sentence naming what needed the loadings and what it found instead. Defaults to the wrapping-estimator case; a consumer that is not an estimator must supply its own, because the default's claim about the type not guaranteeing a regression is an estimator-field claim.

Validation

Returns

  • nothing.

Related

source
PortfolioOptimisers.prior_regression_remedy — Constant
prior_regression_remedy

The cause-and-remedy half of every "this prior carries no factor block" message, written once so the two kinds of consumer cannot drift apart on it.

Consumers differ in what they wanted the loadings for — projecting factor moments through them, or drawing them — so each supplies its own opening sentence via assert_prior_regression's lead. What none of them may restate is the diagnosis: there is exactly one way to arrive with rr === nothing, and exactly one remedy, and both are consequences of the rule that every wrapping estimator forwards rr and the factor block fpr rather than of the consumer.

Which errors carry it

Two, and they are the two ways a caller can ask for loadings that were never computed:

  • The IsNothingError that assert_prior_regression raises. This is the estimator and plotting path: an estimator whose pe slot produced a prior with rr === nothing, or a factor-space plotting entry point handed the same prior. Each supplies its own lead and appends this string unchanged.
  • The IsNothingError that constraint_space_basis raises when a factor exposure constraint has no basis for its loadings — the space states none and the prior carries none. That message opens with its own sentences about the space, then appends this string, because the way out of the prior half of the diagnosis is the same one.

Related

source
PortfolioOptimisers.prior_field_values — Function
prior_field_values(pr::AbstractPriorResult) -> NamedTuple

Return a prior result's own fields as a named tuple, keyed in declaration order.

Reads through getfield, so it sees only what the carrier stores — never a name a @forward_properties block exposes on top. That is the distinction forward_prior needs: HighOrderPrior forwards the whole of its pr, so mu and sigma are properties of it without being fields, and only a field can be patched. The field list is derived rather than written out, so adding a field to a carrier does not need an edit here.

Algorithm

  1. Read the field names of typeof(pr) into fnames, in declaration order.
  2. Read each of those fields off pr with getfield, and return them as a NamedTuple keyed by fnames.

Arguments

  • pr: Prior result.

Returns

  • vals::NamedTuple: The carrier's own fields, keyed by name in declaration order.

Related

source
PortfolioOptimisers.prior_result_property_pool — Function
prior_result_property_pool() -> Vector{Symbol}

Return every property name a prior result can answer, unioned over the carriers.

This is the candidate pool propagatable_contract_violations checks an @pprop field name against: the generated factory(x, pr::AbstractPriorResult, args...) reads getproperty(pr, :field), and the carrier that arrives is not known at the declaration.

The names of the two carriers are written out, as in reconstruct_prior; their fields are derived, so a carrier that gains a field needs no edit here. HighOrderPrior forwards the whole of the pr it wraps, so the low-order names are properties of it too without being fields — that forwarding is the reason a plain fieldnames of one carrier is not the pool.

These methods are defined here, after both carriers, because they name the concrete types.

Algorithm

  1. Concatenate the field names of LowOrderPrior and of HighOrderPrior into one vector of Symbol.
  2. Remove the duplicates in place, and return the vector. fpr is a field of both carriers, so the concatenation is not already unique.

Returns

  • pool::Vector{Symbol}: Every property name a prior result can answer, without duplicates.

Related

source
PortfolioOptimisers.bound_field_is_stale — Function
bound_field_is_stale(
    pr::AbstractPriorResult,
    sym::Symbol
) -> Bool

Return true when field sym of pr holds a value that would go stale if the field it is bound to changed without it.

A field that the carrier does not have, or holds as nothing, has nothing to go stale. Reads through getfield so a forwarded property of the same name cannot answer for a field the carrier does not own.

Algorithm

  1. Check whether typeof(pr) declares a field named sym. When it does not, the binding is inert on this carrier, so answer false without reading anything.
  2. Read that field with getfield, and answer true when the value it holds is not nothing.

Arguments

  • pr: Prior result.
  • sym: Name of the bound field to test.

Returns

  • stale::Bool: true when the carrier holds a value under sym that a change to the field it is bound to would make stale.

Related

source
PortfolioOptimisers.returns_matrix_picker — Function
returns_matrix_picker(
    pr::Union{AbstractPriorResult, ReturnsResult},
    rd::Union{Nothing, ReturnsResult},
    x_src::Symbol
) -> Any

Pick the returns matrix the clustering, phylogeny and centrality estimators read.

Two carriers can supply asset returns: the prior result and the raw returns result. x_src names which one wins — :prior takes pr.X, :data takes rd.X. When no returns result is available there is nothing to select between, so pr.X is used and x_src is inert.

Algorithm

  1. Check that x_src names one of the two carriers, with assert_source_selector.
  2. Return pr.X when there is no returns result, or when x_src is :prior. Return rd.X otherwise.

Arguments

  • pr: Prior result or returns result. Both carry the asset returns matrix X and the feature matrix Z, so either can supply them.
  • rd: The returns result to use. Consulted only when x_src is :data.
  • x_src: Which returns matrix the clustering, phylogeny and centrality estimators read: :prior takes the prior result's X, :data takes the raw returns result's X. Ignored when no returns result is available, in which case the prior result's X is used.

Validation

  • x_src in (:prior, :data).

Returns

  • X::MatNum: Asset returns matrix from the selected carrier.

Related

source
PortfolioOptimisers.investable_mask — Function
investable_mask(
    pr::AbstractPriorResult
) -> Union{Nothing, BitVector}

Derive the Investable Mask of a prior result: true at every asset whose moments are finite.

A Prior Estimator fits on the coverage universe and returns a result on the full asset universe, where an asset it could not estimate carries NaN in mu and on the diagonal of sigma. The mask is derived here and stored nowhere: no prior result carries a mask field, so a caller that needs one calls this, and a caller that reduces a result keeps the mask it was given. A reduced result can no longer yield it, because its mu is finite everywhere.

The all-investable case returns nothing rather than a mask of every true. That sentinel is what keeps a universe with nothing to exclude on the path it took before the mask existed: no reduction, no expansion, no allocation.

An off-diagonal NaN in sigma is not read. A non-investable asset may carry NaN across its whole row and column, and the diagonal alone decides, so the mask costs one pass over two vectors.

Algorithm

  1. Take the elementwise conjunction of isfinite.(pr.mu) and isfinite.(diag(pr.sigma)).
  2. Throw an IsEmptyError when the conjunction holds no true. An optimisation over no asset has no answer to give, and a zero-asset problem passed downstream fails further from its cause.
  3. Return nothing when the conjunction holds no false.
  4. Return the conjunction otherwise.

Arguments

  • pr: Prior result.

Validation

  • At least one asset must be investable.

Returns

  • imsk::Option{BitVector}: true at every investable asset, or nothing when every asset is investable.

Examples

julia> pr = prior(EmpiricalPrior(),                  ReturnsResult(; nx = ["a", "b"], X = [0.1 -0.2; -0.1 0.2; 0.05 0.1]));julia> isnothing(PortfolioOptimisers.investable_mask(pr))true

Related

source
investable_mask(
    pr::HighOrderPrior
) -> Union{Nothing, BitVector}

Derive the Investable Mask of a HighOrderPrior, which reads every order the carrier holds.

The AbstractPriorResult method reads mu and the diagonal of sigma, which under a CoveragePolicy no longer implies that the higher-order tensors are finite where it admits: the policy is set per estimator, so pe.pe may carry one while pe.ske and pe.kte do not. This method therefore ANDs the per-asset diagonals of sk and kt into it with comoment_investable, so an asset the higher orders could not estimate leaves the problem rather than reaching a spectral step that throws.

The narrowing is silent here, because a mask is derived at every optimiser entry and a derivation owes no side effect. The one warning belongs to the fit, and assert_matched_coverage raises it there.

Arguments

  • pr: Prior result.

Validation

  • At least one asset must be investable.

Returns

  • imsk::Option{BitVector}: true at every investable asset, or nothing when every asset is investable.

Related

source
PortfolioOptimisers.investable_views — Function
investable_views(pr::AbstractPriorResult, sets::Nothing) -> Tuple
investable_views(pr::AbstractPriorResult, sets::UniverseSets) -> Tuple

Derive the Investable Mask of a fitted prior, and give a view builder the universe it may write rows over.

A view is a dense linear form over the asset axis, and a departed asset carries NaN in mu and on the diagonal of sigma. A[i] == 0 does not protect a row from it, because 0 * NaN is NaN, so a view naming only live assets is poisoned exactly as thoroughly as one naming the asset that left: the row reaches the solver all NaN, and the fit fails naming something that is not the cause. Building the row on the investable columns is the whole fix, and it is the same reduction every optimiser takes at its entry — port_opt_view of the carrier at findall(imsk), and #919 measured it bit-exact against the hand-reduced oracle.

Both view-taking prior families reduce here, and what they owe afterwards is theirs, not this door's. An entropy pooling row runs over observations, so its solved probabilities carry no asset axis and nothing is expanded back — the moments come from the refit wrapped prior, which already holds the full-universe NaN frame. A Black–Litterman posterior is a moment pair over the reduced assets, so it has to be written back into a NaN frame of the full width with expand_moment before it leaves the estimator. That is the whole of the difference, and it is why this verb hands back the index rather than swallowing it.

The door also mints the Non-Investable Axis on the sets it hands the builders, with non_investable_sets after port_opt_view — after, because the view drops the axis so that a sub-problem cannot inherit its parent's departures. That is what lets a builder tell a departed name from a typo: the row is dropped whole and in silence for the first, and today's strict_diagnostic applies for the second.

sets splits by dispatch and the mask by a condition, and the asymmetry is the whole of the reason. sets is a field of a @concrete estimator, so whether it is nothing is a type fact, fixed per instantiation: the pair is static dispatch, it costs nothing, and it is what keeps the returned sets concretely a UniverseSets. A single method over Option{<:UniverseSets} would answer a value-level Union, and the view builders declare sets::UniverseSets — so JET finds no method for the Nothing half at every builder call site, none of them reachable. investable_mask, by contrast, answers a Union{Nothing, BitVector} that depends on the data: Julia union-splits a two-member Union and compiles a method pair back into this very branch, so dispatching on it would buy nothing and cost a unit in a swept file. Dispatch where the fact is a type; branch where it is a value.

sets of nothing returns early whatever the mask says: a view-taking estimator's constructor refuses nothing sets the moment any view is stated, so there is nothing to build and nothing to reduce for. The all-investable path returns its arguments untouched, so a gap-free fit pays one pass over two vectors and allocates nothing.

Algorithm

  1. Return nothing, sets and no departed names when sets is nothing, which is the method the estimator's own field type selects.
  2. Otherwise derive the Investable Mask from the fitted prior with investable_mask, and return nothing, sets and no departed names when it is nothing.
  3. Read the asset universe off sets.dict[sets.xkey], and check it against the mask.
  4. Read the departed names with non_investable_names.
  5. Take a port_opt_view of sets at findall(imsk), mint the Non-Investable Axis on it with non_investable_sets, and return the mask, the minted sets and the departed names.

Arguments

  • pr: Prior result.
  • sets: The estimator's UniverseSets, or nothing.

Validation

  • length(sets.dict[sets.xkey]) == length(imsk). A DimensionMismatch naming both counts is thrown otherwise, in place of the BoundsError the complement would raise.

Returns

  • (imsk, sets, ni): The Investable Mask or nothing, the reduced sets carrying the Non-Investable Axis, and the departed names. The mask is what investable_prior views at and what expand_moment writes back through, so a caller that reduces and expands needs nothing else.

Related

source
PortfolioOptimisers.investable_prior — Function
investable_prior(imsk::Nothing, pr::AbstractPriorResult) -> AbstractPriorResult
investable_prior(imsk::BitVector, pr::AbstractPriorResult) -> AbstractPriorResult

View a fitted prior at the Investable Mask investable_views derived.

It is separate from investable_views because a staged entropy pooling fit refits its wrapped prior between stages, once per solve, and every refit has to be viewed again before the next stage's builders read it. The mask itself does not move — a column that could not be estimated stays unestimable under any reweighting of the observations — so it is derived once and this is applied many times.

nothing is the all-investable path and hands the prior straight back, so a gap-free fit allocates nothing. The split is a method pair here and a condition inside investable_views, and the two are not in conflict: the mask arrives from a local whose Union Julia has already split at the call site, so each branch reaches this with a concrete argument. What it must not become is one method over Option{BitVector}, which would put the union back.

Arguments

  • imsk: The Investable Mask, or nothing when every asset is investable.
  • pr: Prior result.

Returns

  • pr::AbstractPriorResult: The prior over the investable assets, or the prior unchanged.

Related

source
PortfolioOptimisers.investable_universe_names — Function
investable_universe_names(sets::Nothing, imsk) -> VecStr
investable_universe_names(sets::UniverseSets, imsk::Nothing) -> VecStr
investable_universe_names(sets::UniverseSets, imsk::BitVector) -> VecStr

Name the departed assets for an estimator that reduces its asset axis while its views live on another one.

investable_views is the door for an estimator whose views resolve against xkey: it reduces the sets, mints the Non-Investable Axis on them, and the names fall out on the way. BayesianBlackLittermanPrior and FactorBlackLittermanPrior write their views on the factor axis, so they reduce their asset side and touch no view axis at all — there is nothing for them to mint, and going through that door would make them demand an asset universe they have no other use for. They still have a departure to report, and this is the least they need to report it.

Sets that are not stated at all answer an empty list rather than throwing, and an empty list is what announce_non_investable says nothing about. That is the honest outcome, and it is a real configuration: both members admit sets of nothing entirely, because a precomputed BlackLittermanViews resolves no name and needs no universe. A stated universe that does not describe this fit answers the same silence, because neither member reads an asset name for any other purpose and so nothing else has checked its length.

Arguments

  • sets: The estimator's UniverseSets, or nothing.
  • imsk: The Investable Mask, or nothing when every asset is investable.

Returns

  • ni::VecStr: The names the mask left out, or an empty vector.

Related

source
PortfolioOptimisers.scenario_fill_pairs — Function
scenario_fill_pairs(
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    imsk::BitVector
) -> Vector

Find the entries a mask-aware prior fills: the non-finite returns of an investable asset.

A non-investable asset keeps its whole NaN column, because the Investable Mask is derived from that gap and filling it would erase the mask. Only a column whose moments came back finite is filled.

Arguments

  • X: Asset returns, observations × assets.
  • imsk: The Investable Mask, true at every asset whose prior moments were finite.

Returns

  • filled::Vector{Tuple{Int, Int}}: The (observation, asset) pairs to fill, asset-major.

Related

source
PortfolioOptimisers.scenario_fill_msg — Function
scenario_fill_msg(
    filled::AbstractVector{<:Tuple{Integer, Integer}},
    wi::Integer,
    worst::Real,
    frac::Real,
    fill_limit::Union{Nothing, Real}
) -> String

Write the message a scenario fill raises.

The message names the assets, the count of filled pairs and the first observation, so a caller can find the listing that made them. It then names the worst investable column, the share of its own observations the fill invented, and the limit that share was measured against — that share is the one the fill tests, because a matrix-wide denominator scales with the universe and hides the column the notice exists to catch. The matrix-wide share is reported beside it as context, and it trips nothing.

It states the consequence at each of the four consumers that a census of the readers of pr.X found, because a caller told only about the tail will not look for the other three, and it states the tail's cost as the identity rather than the adjective: the invented zeros do not enter the tail, they inflate the denominator, so a measure at level alpha over an admitted column of coverage c reads that column's observed alpha / c level.

The limit it names is the estimator's own, after resolve_fill_limit. A fill_limit of nothing demands every observation of every investable asset, so the message says so rather than printing a share no caller chose.

Arguments

  • filled: The (observation, asset) pairs scenario_fill_pairs found.
  • wi: The index of the worst investable column, the one with the most filled entries.
  • worst: The share of its own observations that column's fill invented.
  • frac: The share of the entries of the returns matrix every filled pair is, as context.
  • fill_limit: The share of a column the fit allowed in silence, or nothing when it allowed none.

Returns

  • msg::String: The message.

Related

source
PortfolioOptimisers.resolve_fill_limit — Function
resolve_fill_limit(fill_limit::Nothing, floor::Nothing)
resolve_fill_limit(fill_limit::Nothing, floor::Real)
resolve_fill_limit(fill_limit::Real, floor::Nothing)
resolve_fill_limit(fill_limit::Real, floor::Real)

Resolve the share of an investable column a scenario_fill may invent in silence, at the fit.

fill_limit and a CoveragePolicy's min_coverage are two spellings of one number. admits reads an asset's coverage share as its own observation count over the number of observations folded, and the fill counts that column's non-finite entries over the same denominator, so filled_share == 1 - coverage_share identically and the admission test is the fill test. A limit that did not know this would fire on nearly every fold of an available-case walk-forward, naming the caller for doing precisely what they configured.

So nothing derives. Where any arm of the fitting estimator states a floor, nothing means 1 - floor, and it never fires: every admitted column satisfies it by construction. Where no arm states one — the exponentially weighted family is mask-aware without a policy, gating on min_obs, a count that says nothing about the share of a long window — nothing keeps its original meaning and names every fill, because a caller who set no floor has weighed no trade.

An explicit fill_limit overrides the derivation and must be tighter than admission. A value above 1 - floor is refused, because it is dead by construction: nothing that reaches the fill could trip it, and a knob that cannot fire is worse than no knob. What an explicit value buys is the one configuration the derivation cannot express — admit broadly and be told anyway, min_coverage = 0.3 with fill_limit = 0.5.

The four methods are dispatch rather than a branch, so a call site holding two Options finds a method for every arm of its union split.

Arguments

  • fill_limit: The estimator's own fill_limit field, or nothing to derive one.
  • floor: The binding coverage floor of the estimator's arms, or nothing when no arm states one.

Validation

  • fill_limit <= 1 - floor where both are stated, else a DomainError naming both is thrown.

Returns

  • fill_limit::Option{<:Real}: The share of an investable column the fill may invent in silence, or nothing when it may invent none.

Related

source
PortfolioOptimisers.scenario_fill — Function
scenario_fill(
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    mu::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    sigma::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    strict::Bool,
    fill_limit::Union{Nothing, Real}
) -> Any
scenario_fill(
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    mu::AbstractVector{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    sigma::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    strict::Bool,
    fill_limit::Union{Nothing, Real},
    named::Union{Nothing, AbstractSet{<:Integer}}
) -> Any

Return the returns matrix with the missing rows of every investable asset filled with zero, and say so above the fitting estimator's resolved fill_limit.

A mask-aware moment estimator answers a young asset from the observations it has, so the asset is investable and its returns column still carries a NaN at every row before it listed. Every consumer of a Prior Result reads that column — the JuMP model, the meta-optimisers and the value-level door among them — so the fill is paid once, here, on the estimator's own pass, rather than at each of them.

A non-investable asset keeps its NaN column, so investable_mask is unchanged. mu, sigma and every other block are untouched, because the estimator computed them from the rows it saw.

The share the fill tests is per asset: the worst investable column's own count of filled entries over the number of observations. A matrix-wide denominator scales with the universe, so the column the notice exists to catch disappears inside it — one asset of a hundred whose column is seven-tenths invented is seven thousandths of the matrix, under any limit a caller would set. The matrix-wide share is reported in the message as context and trips nothing.

How much of the trade passes in silence is the fitting estimator's own answer, carried in its fill_limit field and resolved at the fit by resolve_fill_limit against the coverage floor of its arms — the share is a property of one fit, not of the session, so two priors in one program may answer differently. Under strict = false the trade is silent while the worst column's share stays at or below the resolved limit, and is named through strict_diagnostic above it. Under strict = true any fill refuses, whatever the limit holds.

Two mask-aware families reach this verb. The exponentially weighted family carries no CoveragePolicy, so a fill_limit of nothing names every fill there. The plain family is mask-aware exactly where a policy is set, and a policy derives a limit that never fires, so an available-case fit names nothing while it does what it was configured to do. A plain estimator with no policy never reaches this verb at all: under the whole-window rule, an asset it could not cover leaves the Coverage Universe and is not investable.

Algorithm

  1. Return X itself when every entry of X is finite, which is every fit over a complete window.
  2. Derive the Investable Mask from mu and the diagonal of sigma, as investable_mask does, and find the pairs to fill with scenario_fill_pairs. Return X itself when there are none, which is a gap that belongs to a non-investable asset alone.
  3. Narrow the pairs to report with scenario_fill_report, which drops the assets a carry state has already named and is the identity when nothing remembers. Under strict the narrowing is skipped, because strict refuses any fill and must not depend on how often the estimator has been read out.
  4. Count the reported pairs per asset, take the worst column's share of its own observations, and report through strict_diagnostic when strict holds, when fill_limit is nothing, or when that share exceeds fill_limit. Record the assets just named with scenario_fill_remember!.
  5. Return a copy of X with zero written at each of the filled pairs — every one of them, not just the reported ones — and the NaN of every non-investable asset left where it is.

The notice fires once per asset, not once per step

named is what separates the batch call from the online one. A batch fit passes nothing, reports every fill it finds, and remembers nothing. A read-out of a folded prior passes the set its PriorCarryState carries, so a walk-forward names an asset at the step it lists and stays quiet afterwards instead of emitting the same notice at every one of two thousand steps. The fill itself is unchanged: every filled pair is written at every call, whatever the set holds. Under strict = true the first fill still throws.

Arguments

  • X: Asset returns, observations × assets.
  • mu: The expected returns the estimator answered, NaN at a non-investable asset.
  • sigma: The covariance the estimator answered, NaN on the diagonal at a non-investable asset.
  • strict: If true, any fill raises an ArgumentError; if false, a fill above the share warns.
  • fill_limit: The share of an investable column the fitting estimator accepts in silence, after resolve_fill_limit, or nothing when it accepts none.
  • named: The assets already named, written in place when a notice fires, or nothing when nothing remembers.

Validation

  • The worst investable column's filled share is at or below fill_limit, else a warning naming the assets is emitted, or an ArgumentError naming them is raised under strict, which any fill raises. A fill_limit of nothing is at or below no share, so any fill is named.

Returns

  • X::MatNum: The returns matrix whose investable columns are finite.

Related

source
PortfolioOptimisers.held_non_investable — Function
held_non_investable(imsk::BitVector, w::VecNum)
held_non_investable(imsk::BitVector, w::VecVecNum)
held_non_investable(imsk::BitVector, W::MatNum)

Find the assets a portfolio holds and the Investable Mask excludes.

A non-investable asset carries NaN in mu and on the diagonal of sigma, so no moment of it exists. A portfolio that holds none of it is reduced exactly. A portfolio that holds some of it is what a caller must be told about, and this is the one scan that finds those assets. A weight path holds an asset when any of its rows does, and a population holds one when any of its members does.

Arguments

  • imsk: The Investable Mask, true at every asset whose prior moments were finite.
  • w: Portfolio weights, a population of them, or a weight path (observations × assets).

Returns

  • held::VecInt: Indices of the held non-investable assets, empty when there are none.

Related

source
PortfolioOptimisers.investable_weights_view — Function
investable_weights_view(imsk::Nothing, w)
investable_weights_view(imsk::BitVector, w::Nothing)
investable_weights_view(imsk::BitVector, w::VecNum)
investable_weights_view(imsk::BitVector, w::VecVecNum)
investable_weights_view(imsk::BitVector, w::MatNum)

Take the view of the weights at the Investable Mask.

A weight vector is one cross-section, so the mask selects its entries. A weight path is one row of weights per observation, so the mask selects its columns and every row keeps its own observation. A population is reduced member by member. A fold's asset-return history has the shape of a path, observations × assets, and the fold-taking consumers of a HeldWeightsResult take the same view of it, so the one column selection is stated once.

Both nothing sentinels answer the argument they were handed. imsk of nothing is the all-investable universe, so there is nothing to select and the weights come back untouched. w of nothing is a caller who stated no weights at all — equilibrium_mu falls back to equal weights over whatever axis it is handed — so there is nothing to reduce, and reducing it would have to invent a length. Both are dispatch, so a caller holding neither pays nothing.

The w a prior estimator carries is per-asset configuration written against the caller's full universe, and it meets a reduced axis for the same reason a per-asset bound does. That is why this verb, first written for the weights an optimisation returns, is also the one a prior reduces its own weights with: it is one operation, and it is stated in one place.

Arguments

  • imsk: The Investable Mask, true at every asset whose prior moments were finite, or nothing when every asset is investable.
  • w: Portfolio weights, a population of them, a weight path (observations × assets), or nothing.

Returns

  • The view of w at the investable assets, or w itself when either argument is nothing.

Related

source
PortfolioOptimisers.scenario_window — Function
scenario_window(::Nothing, X::MatNum)
scenario_window(max_scenarios::Integer, X::MatNum)

Cut the returns matrix a Prior Result carries down to the last max_scenarios observations.

The Scenario Cap, applied. A prior that carries X carries it for the scenario risk measures, and a caller who wants a long window of moments and a short window of scenarios says so with one field rather than with two fits. The cap therefore touches X alone: mu and sigma are already computed when this runs, over every observation the fit read, and nothing here can or does move them.

It is deliberately the same verb in batch and online. A cap is a property of the result, not of the fold, so prior(pe, X) and the read-out of a folded pe cut the same rows off the same tail.

A window at or above the number of observations is the matrix itself, and no copy is taken: the cut is a view, so a cap that does nothing costs nothing.

Arguments

  • max_scenarios: The number of observations to carry, or nothing to carry every one.
  • X: Asset returns, observations × assets.

Returns

  • X::MatNum: The last max_scenarios rows of X, or X itself.

Related

source
PortfolioOptimisers.scenario_ens — Function
scenario_ens(::Nothing, X::MatNum)
scenario_ens(max_scenarios::Integer, X::MatNum)

State the number of observations the moments of a capped Prior Result were fitted over, or nothing when the cap cuts nothing.

The count a Scenario Cap owes its readers. scenario_window cuts the rows the result carries and leaves mu and sigma fitted over every observation, so a consumer that prices a sample size off size(pr.X, 1) — an uncertainty set's T, a calibration rule's count — would read w rows for moments fitted over t, and mis-price every count by t / w. The result therefore states t in ens exactly when the cap cuts, and every count reader takes ens before the shape. When the cap does not cut, or there is no cap, the rows carried are the observations fitted over, and ens stays nothing, so a fit without a cap is bit-identical to what it was.

It is the same verb in batch and at the folded read-out, as scenario_window is, and the two agree by construction: both read the same t off the same matrix.

Arguments

  • max_scenarios: The number of observations carried, or nothing.
  • X: Asset returns the moments were fitted over, observations × assets, before the cut.

Returns

  • ens::Option{<:Integer}: size(X, 1) when max_scenarios cuts it, nothing otherwise.

Related

source
PortfolioOptimisers.scenario_fill_report — Function
scenario_fill_report(filled, ::Nothing)
scenario_fill_report(filled, named::AbstractSet{<:Integer})

Narrow the filled pairs a scenario_fill reports to the assets it has not named before.

A batch fit calls scenario_fill once, so it reports everything it finds and keeps no memory: that is the nothing method. A walk-forward reading out at every step calls it once per step over a growing window, and every step would otherwise name the same assets again, so a caller who read the first notice learns to ignore the channel and misses the asset that lists at step 500.

So a carry state holds the set of assets it has already named, and this verb narrows the report to the pairs of assets outside it. The set is the state's memory, and scenario_fill_remember! is what writes to it — only for a notice that actually fired, so an asset whose share was under the limit at one step is still named at the step where it goes over.

Arguments

  • filled: The (observation, asset) pairs scenario_fill_pairs found.
  • named: The assets already named, or nothing when nothing remembers.

Returns

  • report::AbstractVector{<:Tuple{Integer, Integer}}: The pairs to name, which is filled itself when nothing remembers.

Related

source
PortfolioOptimisers.scenario_fill_remember! — Function
scenario_fill_remember!(::Nothing, report)
scenario_fill_remember!(named::AbstractSet{<:Integer}, report)

Record the assets a scenario_fill has just named, so a later read-out does not name them again.

The write side of scenario_fill_report. It runs after the notice, not before it, so an asset is remembered exactly when a caller was told about it: under strict the notice throws and nothing is remembered, and under a limit the notice did not trip nothing is remembered either.

The set is a field of a PriorCarryState and is written in place, because a read-out returns a Prior Result rather than the estimator, so there is no other channel by which the memory could survive the call.

Arguments

  • named: The assets already named, or nothing when nothing remembers.
  • report: The (observation, asset) pairs just named.

Returns

  • nothing.

Related

source