GitHub

Prior Scenarios: private API

A prior result carries its scenarios in X. A cap on the number of scenarios keeps the last rows of the observations. The scenario fill writes zero into each gap of an investable asset, so that a risk measure that reads the scenarios reads a complete matrix. The functions below find the entries to fill, resolve the limit on the filled share, keep the rows under the cap, cut a nested prior result to the rows that a wrapping prior keeps, put the probabilities of the scenarios on the rows that the moments read, report the filled entries, and fill them. A Cross-Sectional Factor Prior reads its scenarios from the factor returns of its regression, so two functions fit that regression and give those rows to an entropy pooling prior around it. A fit with cache_reg = true stores a cache, and six functions store it or read it, so the refit of an entropy pooling prior runs no regression. They store the cache on the block, find the block of the cache, and read its factor returns. They split the return forecast of the block, and run the steps of the assembly that read the factor moments. The last one keeps the cache of a pool in the factor slot. Three more functions count the scenarios of a prior with only the steps that drop rows, and check the rows that the exposure lag leaves.

PortfolioOptimisers.scenario_fill_pairs — Function
scenario_fill_pairs(
    X::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    imsk::BitVector
) -> Vector

Find the entries a mask-aware prior fills: the non-finite returns of an investable asset.

A non-investable asset keeps its whole NaN column, because the Investable Mask is derived from that gap and filling it would erase the mask. Only a column whose moments came back finite is filled.

Arguments

  • X: Asset returns, observations × assets.
  • imsk: The Investable Mask, true at every asset whose prior moments were finite.

Returns

  • filled::Vector{Tuple{Int, Int}}: The (observation, asset) pairs to fill, asset-major.

Related

source
PortfolioOptimisers.scenario_fill_msg — Function
scenario_fill_msg(
    filled::AbstractVector{<:Tuple{Integer, Integer}},
    wi::Integer,
    worst::Real,
    frac::Real,
    fill_limit::Union{Nothing, Real}
) -> String

Write the message a scenario fill raises.

The message names the assets, the count of filled pairs and the first observation, so a caller can find the listing that made them. It then names the worst investable column, the share of its own observations the fill invented, and the limit that share was measured against — that share is the one the fill tests, because a matrix-wide denominator scales with the universe and hides the column the notice exists to catch. The matrix-wide share is reported beside it as context, and it trips nothing.

It states the consequence at each of the four consumers that a census of the readers of pr.X found, because a caller told only about the tail will not look for the other three, and it states the tail's cost as the identity rather than the adjective: the invented zeros do not enter the tail, they inflate the denominator, so a measure at level alpha over an admitted column of coverage c reads that column's observed alpha / c level.

The limit it names is the estimator's own, after resolve_fill_limit. A fill_limit of nothing demands every observation of every investable asset, so the message says so rather than printing a share no caller chose.

Arguments

  • filled: The (observation, asset) pairs scenario_fill_pairs found.
  • wi: The index of the worst investable column, the one with the most filled entries.
  • worst: The share of its own observations that column's fill invented.
  • frac: The share of the entries of the returns matrix every filled pair is, as context.
  • fill_limit: The share of a column the fit allowed in silence, or nothing when it allowed none.

Returns

  • msg::String: The message.

Related

source
PortfolioOptimisers.resolve_fill_limit — Function
resolve_fill_limit(fill_limit::Nothing, floor::Nothing)
resolve_fill_limit(fill_limit::Nothing, floor::Real)
resolve_fill_limit(fill_limit::Real, floor::Nothing)
resolve_fill_limit(fill_limit::Real, floor::Real)

Resolve the share of an investable column a scenario_fill may invent in silence, at the fit.

fill_limit and a CoveragePolicy's min_coverage are two spellings of one number. admits reads an asset's coverage share as its own observation count over the number of observations folded, and the fill counts that column's non-finite entries over the same denominator, so filled_share == 1 - coverage_share identically and the admission test is the fill test. A limit that did not know this would fire on nearly every fold of an available-case walk-forward, naming the caller for doing precisely what they configured.

So nothing derives. Where any arm of the fitting estimator states a floor, nothing means 1 - floor, and it never fires: every admitted column satisfies it by construction. Where no arm states one — the exponentially weighted family is mask-aware without a policy, gating on min_obs, a count that says nothing about the share of a long window — nothing keeps its original meaning and names every fill, because a caller who set no floor has weighed no trade.

An explicit fill_limit overrides the derivation and must be tighter than admission. A value above 1 - floor is refused, because it is dead by construction: nothing that reaches the fill could trip it, and a knob that cannot fire is worse than no knob. What an explicit value buys is the one configuration the derivation cannot express — admit broadly and be told anyway, min_coverage = 0.3 with fill_limit = 0.5.

The four methods are dispatch rather than a branch, so a call site holding two Options finds a method for every arm of its union split.

Arguments

  • fill_limit: The estimator's own fill_limit field, or nothing to derive one.
  • floor: The binding coverage floor of the estimator's arms, or nothing when no arm states one.

Validation

  • fill_limit <= 1 - floor where both are stated, else a DomainError naming both is thrown.

Returns

  • fill_limit::Option{<:Real}: The share of an investable column the fill may invent in silence, or nothing when it may invent none.

Related

source
PortfolioOptimisers.scenario_window — Function
scenario_window(::Nothing, X::MatNum)
scenario_window(max_scenarios::Integer, X::MatNum)

Cut the returns matrix a Prior Result carries down to the last max_scenarios observations.

The Scenario Cap, applied. A prior that carries X carries it for the scenario risk measures, and a caller who wants a long window of moments and a short window of scenarios says so with one field rather than with two fits. The cap therefore touches X alone: mu and sigma are already computed when this runs, over every observation the fit read, and nothing here can or does move them.

It is deliberately the same verb in batch and online. A cap is a property of the result, not of the fold, so prior(pe, X) and prior(pe) with no data on a folded pe cut the same rows off the same tail.

A window at or above the number of observations is the matrix itself, and no copy is taken: the cut is a view, so a cap that does nothing costs nothing.

Arguments

  • max_scenarios: The number of observations to carry, or nothing to carry every one.
  • X: Asset returns, observations × assets.

Returns

  • X::MatNum: The last max_scenarios rows of X, or X itself.

Related

source
PortfolioOptimisers.scenario_tail — Function
scenario_tail(pr::LowOrderPrior, n::Integer, name::Symbol)

Cut a nested Prior Result to the last n scenarios that it carries.

AugmentedBlackLittermanPrior fits two nested priors, one on the asset returns and one on the factor returns. A Scenario Cap in one of them keeps the last rows of its own returns alone. The two blocks of the result describe the same observations, so the result keeps the rows that both nested priors keep, and this function cuts the nested prior that carries more rows. The moments of pr are fitted over every observation, so the cut changes no moment.

A Prior Result with ens = nothing states size(pr.X, 1) as the count of the observations that its moments read. The cut makes X shorter, so it writes that count in ens, as scenario_ens does for the cap itself.

Algorithm

  1. Take t = size(pr.X, 1). When n >= t, return pr.
  2. Refuse a pr that carries weights, per # Validation.
  3. Forward pr with forward_prior, with X and o_X cut by scenario_window, and with ens set to t when it is nothing.

Arguments

  • pr: The Prior Result of a nested prior.
  • n: The number of scenarios to keep.
  • name: The field of the wrapping prior that holds the nested prior, for the error message.

Validation

  • When the cut removes a row, pr.w is nothing. A weight vector states a probability for each row that pr carries, so it has no meaning on fewer rows. A DimensionMismatch that names name, both counts and the fix is thrown otherwise.

Returns

  • pr::LowOrderPrior: pr with the last n rows of X, or pr itself.

Related

source
PortfolioOptimisers.scenario_weights — Function
scenario_weights(w::Nothing, t::Integer)
scenario_weights(w::ObsWeights, t::Integer)

Put the probabilities w of the scenarios of a capped Prior Result on the t observations that its moments read.

scenario_window keeps the last rows of the observations as scenarios, and the moment estimators read every observation. A prior that reweights scenarios, such as EntropyPoolingPrior, states a probability for each scenario and none for the observations that the cap removed. Its posterior gives those observations no probability, so this function gives them a zero weight. A weighted moment over the t observations is then the moment of the scenarios under w, and the posterior mean is pr.X' * w, the mean that a view states.

Algorithm

  1. A nothing w returns nothing.
  2. Otherwise take s = length(w), and refuse s > t.
  3. When s == t, the cap removed no row: return w.
  4. Otherwise return t - s zeros followed by w, through scenario_probabilities: in the kind of w when it is a ScenarioProbabilities, and as StatsBase.AnalyticWeights otherwise.

Arguments

  • w: The probabilities of the scenarios, or nothing.
  • t: The number of observations that the moments read.

Validation

  • length(w) <= t. A DimensionMismatch is thrown otherwise.

Returns

  • w′::Option{<:ObsWeights}: The weights over the t observations, or nothing.

Related

source
PortfolioOptimisers.cross_sectional_regression_stage — Function
cross_sectional_regression_stage(pe::CrossSectionalFactorPrior, X::MatNum,
                                 F::Option{<:MatNum}, pnl::Option{<:AssetPanel};
                                 ne = nothing, E = nothing, iv = nothing,
                                 ivpa = nothing, ts = nothing) -> NamedTuple

Fit the cross-sectional regression of a CrossSectionalFactorPrior, up to the factor returns that its factor prior reads.

The returns-matrix method of prior fits pe.pe on the factor returns of this stage and assembles the result. scenario_factory and prior_probability_factory read the same factor returns, because an entropy pooling prior around pe states its probabilities on the scenarios of pe.pe. The stage reads no weight of pe.pe, so a change of pe.pe alone leaves its answer the same.

Algorithm

  1. Run the steps 1 to 9 of the returns-matrix method of prior: the exposure stage, the Neutralisation, the Factor Family Basis, the lag, the eligibility, the regression and the idiosyncratic variance history.
  2. Append the observed factors with cross_sectional_observed_append, giving ca.
  3. Mark the factors that are not empty in lv, on the axis of ca.f. An Observed Factor is never empty.

Arguments

  • pe: Cross-Sectional Factor Prior estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Optional factor returns matrix, already oriented. The stage does not read it.
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it. This estimator reads it, and refuses without it.
  • ne, E, iv, ivpa, ts: As in the returns-matrix method of prior, with E already oriented.

Validation

  • The rules of the returns-matrix method of prior, up to the factor prior.

Returns

  • ca::NamedTuple: The output of cross_sectional_observed_append. ca.f holds the factor returns, one row per fitted observation.
  • lv::BitVector: true at each factor of ca.f that is not empty.
  • fit::NamedTuple: The regression, the weights, the variance history and the rows that cross_sectional_assemble reads.
  • rde::ReturnsResult: The returns data that the estimated members read.

Related

source
PortfolioOptimisers.cross_sectional_factor_scenarios — Function
cross_sectional_factor_scenarios(pe::CrossSectionalFactorPrior, X::MatNum,
                                 F::Option{<:MatNum}, pnl::Option{<:AssetPanel};
                                 ne = nothing, E = nothing, iv = nothing,
                                 ivpa = nothing, ts = nothing, kwargs...)

Return the factor returns that the factor prior pe.pe of a CrossSectionalFactorPrior is fitted on, with the regression stage that gives them.

An entropy pooling prior around pe states its probabilities on the scenarios of pe, and these are the last rows of the factor returns of pe.pe. The Descriptor warm-up and the exposure lag drop rows, so the number of factor returns is known only after the regression. scenario_factory and prior_probability_factory read these rows, to put the probabilities on the observations that pe.pe reads. scenario_prior also reads the stage, so the refit runs the regression once.

Algorithm

  1. Fit the regression of pe with cross_sectional_regression_stage, giving the stage rs, with the factor returns rs.ca.f and the mask rs.lv of the factors that are not empty.
  2. Take the columns of rs.ca.f that rs.lv marks, as the fit of pe.pe reads them.
  3. Return them with rs, and with the keyword arguments that the stage did not read.

Arguments

  • pe: Cross-Sectional Factor Prior estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Optional factor returns matrix, already oriented.
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.
  • ne, E, iv, ivpa, ts: As in the returns-matrix method of prior, with E already oriented.
  • kwargs...: The other keyword arguments of the fit. The regression reads none of them.

Validation

Returns

  • f::Matrix: The factor returns of pe.pe, one row per fitted observation.
  • rs::NamedTuple: The output of cross_sectional_regression_stage.
  • kw: The keyword arguments in kwargs..., which the rest of the fit reads.

Related

source
PortfolioOptimisers.cross_sectional_cached_block — Function
cross_sectional_cached_block(pe::CrossSectionalFactorPrior, pr::Nothing, X::MatNum)
cross_sectional_cached_block(pe::CrossSectionalFactorPrior, pr::AbstractPriorResult,
                             X::MatNum)
cross_sectional_cached_block(pe::CrossSectionalFactorPrior, pr::LowOrderPrior, X::MatNum)

Return the block of the fit pr of a CrossSectionalFactorPrior when it holds the cache that a refit under new scenario weights reads, and nothing otherwise.

A fit of prior with cache_reg = true stores the cache. EntropyPoolingPrior and MeucciEntropyPoolingPrior fit the prior that they wrap with it, and give the result of that fit to scenario_prior and scenario_factory. The regression, the cross-sectional weights, the residual block and the Return Forecast of a refit are those of the fit, so the refit reads them from the block and runs no regression. A result with no cache gives nothing, and the refit then runs the whole fit.

Algorithm

The method that Julia selects is the algorithm.

  1. No result, or a result of any other type: return nothing.
  2. A LowOrderPrior: return its block rr when every condition holds, and nothing otherwise:
    1. rr is a CrossSectionalFactorModel with a regression csr, a recorded live-factor mask csr.lv and an active-mask history amr.
    2. rr has one row of M for each column of X, so the cache describes the asset universe of the refit.
    3. When a Calibration Slot of pe reads the Return Forecast history, as reads_forecast_history answers, and pe states a Return Forecast Estimator, rr carries that history in hist.

Arguments

  • pe: Cross-Sectional Factor Prior estimator.
  • pr: The result of the fit that the views read, or nothing.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.

Returns

  • rr::Option{<:CrossSectionalFactorModel}: The block that holds the cache, or nothing.

Related

source
PortfolioOptimisers.cross_sectional_cached_scenarios — Function
cross_sectional_cached_scenarios(rr::CrossSectionalFactorModel) -> NamedTuple

Return the factor returns that the factor prior of a CrossSectionalFactorPrior is fitted on, from the block of a fit that stores the cache.

The regression stage gives these factor returns to the fit, as cross_sectional_factor_scenarios states. The block of a fit with cache_reg = true holds the regression and its live-factor mask, so scenario_prior and scenario_factory read the same factor returns from it and run no regression.

Algorithm

  1. Take the factor returns of the block with cross_sectional_factor_returns, the estimated factors and then the observed ones, giving fa.
  2. Extend the live-factor mask rr.csr.lv of the estimated factors with true for each observed factor, giving lv. An Observed Factor carries the return the caller observed, so it is never empty.
  3. Take the columns of fa that lv marks, giving f.

Arguments

Returns

  • fa::MatNum: The factor returns on the combined reduced factor axis.
  • f::MatNum: The factor returns of the factors that are not empty, which the factor prior reads.
  • lv::BitVector: true at each factor of fa that is not empty.

Related

source
PortfolioOptimisers.cross_sectional_forecast_split — Function
cross_sectional_forecast_split(ofit::AbstractOrthogonalForecastFit, rf::Nothing,
                               csfm::CrossSectionalFactorModel,
                               cre::AbstractCrossSectionalRegressionEstimator)
cross_sectional_forecast_split(ofit::AbstractOrthogonalForecastFit,
                               rf::AbstractReturnForecastResult,
                               csfm::CrossSectionalFactorModel,
                               cre::AbstractCrossSectionalRegressionEstimator)

Split a Return Forecast Result against the latest Factor Exposures of the estimated factors of a CrossSectionalFactorModel, giving its spanned coefficients and its orthogonal part before the Orthogonal Forecast Scale.

cross_sectional_return_forecast fits the forecast and splits it here. The refit of scenario_prior reads the forecast that the block carries, and splits it here, so it fits no member.

Algorithm

  1. No Return Forecast Result: return nothing for both parts.
  2. Split rf.mu against the latest exposures of the estimated factors with cross_sectional_alpha_split, under the weights of the latest row of csfm.rw, giving g and ap. The observed factors are the trailing columns of L, and the split leaves them out, so g has one entry per column of csr.f.
  3. Scale ap with orthogonal_forecast_rescale, which changes it under OrthogonalPartCalibration alone.

Arguments

  • ofit: The Orthogonal Forecast Fit of the prior.
  • rf: The Return Forecast Result, or nothing.
  • csfm: The factor-model block.
  • cre: Cross-Sectional Regression Estimator of the split.

Validation

Returns

  • g::Option{<:VecNum}: The spanned coefficients, or nothing.
  • ap::Option{<:VecNum}: The orthogonal part before the Orthogonal Forecast Scale, one entry per asset, or nothing.

Related

source
PortfolioOptimisers.cross_sectional_moment_prior — Function
cross_sectional_moment_prior(pe::CrossSectionalFactorPrior, f_pr::NamedTuple,
                             csfm::CrossSectionalFactorModel, sp::NamedTuple, L::MatNum,
                             am::AbstractVector{Bool}, Xs::MatNum, oX::Option{<:MatNum};
                             kwargs...) -> LowOrderPrior

Build the Prior Result of a CrossSectionalFactorPrior from its block and its factor moments, with the steps of cross_sectional_assemble that read the factor moments.

The fit and the call with no data of the carry fold reach it through cross_sectional_assemble. The refit of scenario_prior under new scenario weights reaches it with the block of the fit that the views read. The block, the scenarios and the Return Forecast read no scenario weight of the factor prior, so the refit takes them from that fit, and only the steps that this function runs read the new weights.

Algorithm

  1. Take the ready factors of f_pr with cross_sectional_ready_factors and the determined assets with cross_sectional_determined. Refuse a fit that determines no asset with assert_cross_sectional_factor_moments.
  2. Resolve the Spanned Shrinkage lambda and the Orthogonal Forecast Scale c with cross_sectional_calibration, against the factor moments f_pr and the block csfm.
  3. Write b = c ap, the Return Forecast Result and the resolved lambda and c onto the block with cross_sectional_forecast_block, giving rr. Blend the spanned part into the mean of the estimated factors with cross_sectional_forecast_mu under the resolved lambda, and keep the mean of the observed factors, giving f_mu.
  4. Expand the blended factor moments onto the raw factor axis with cross_sectional_expand, with the basis of the block.
  5. Take the investable assets idx with cross_sectional_investable, from the active assets am of the latest observation, the latest idiosyncratic variances of csfm.vs, and the assets that the ready factors of the blended mean determine.
  6. Lift the reduced factor distribution onto the assets with cross_sectional_lift, and add b to the expected return it answers.
  7. Assemble a LowOrderPrior over Xs, with oX under o_X, the three lifted moments, the factor prior's w divided by its sum, its ens, kld and ow, the block under rr, and the expanded factor prior under fpr. The factor prior keeps the cache pc of a pool that answered it, so the refit of the pool reads it from fpr. The factor prior holds the factor returns of the block on the raw axis, on the last rows that the scenarios keep.

Arguments

  • pe: Cross-Sectional Factor Prior estimator.
  • f_pr: The factor moments that cross_sectional_factor_moments states over the combined reduced factor returns.
  • csfm: The factor-model block, with the regression, the variance history, the idiosyncratic covariance, the basis and the factor returns of the fit.
  • sp: The split of the Return Forecast, (; rf, g, ap, hist).
  • L: The loadings of the latest observation on the combined reduced factor axis.
  • am: The active mask of the latest observation, one entry per asset.
  • Xs: The asset return scenarios.
  • oX: The base-currency returns of the scenario rows, or nothing.
  • kwargs...: Additional keyword arguments passed to cross_sectional_lift.

Validation

  • The factor prior determines the factor moments of at least one asset. Raises an IsNonFiniteError.
  • At least one asset is investable at the latest observation. Raises an IsEmptyError.
  • The rules of every verb the algorithm names. A resolved lambda lies in [0, 1], and a resolved c is finite and >= 0.

Returns

  • pr::LowOrderPrior: The prior on the full asset universe, as prior states it.

Related

source
PortfolioOptimisers.cross_sectional_kept_cache — Function
cross_sectional_kept_cache(f_pr::NamedTuple, pf::Option{<:AbstractPriorResult}) -> NamedTuple

Keep the cache of a pool in the factor slot of a CrossSectionalFactorPrior on the factor moments of its refit.

The refit of scenario_prior fits the model of the pool under the new weights, which is pe2 of the pool and keeps no cache. So the factor prior fpr of the refit would carry no cache, and an entropy pooling prior around a nested one would run the opinions of the pool again. The fit that the views read gives the cache in pf, the factor prior of its result, and the refit keeps it.

Arguments

  • f_pr: The factor moments of the refit, as cross_sectional_factor_moments states them.
  • pf: The factor prior of the result of the fit that the views read, or nothing.

Returns

  • f_pr::NamedTuple: f_pr, with pc from pf when f_pr carries none.

Related

source
PortfolioOptimisers.cross_sectional_stored_cache — Function
cross_sectional_stored_cache(csfm::CrossSectionalFactorModel, cache_reg::Bool,
                             amr::AbstractMatrix{Bool}, hist::Option{<:MatNum})
    -> CrossSectionalFactorModel

Store the cache that the refit of an entropy pooling prior reads on the block of a CrossSectionalFactorPrior.

cross_sectional_assemble calls it after the Return Forecast. A plain fit stores nothing, and a fit with cache_reg = true stores the active mask of the fitted rows in amr and the Return Forecast history in hist, which is nothing when no Calibration Slot reads it.

Arguments

  • csfm: The factor-model block.
  • cache_reg: If true, store the cache.
  • amr: The active mask of the fitted rows.
  • hist: The Return Forecast history, or nothing.

Returns

  • csfm::CrossSectionalFactorModel: The block, with the cache when cache_reg is true.

Related

source
PortfolioOptimisers.kept_scenarios — Function
kept_scenarios(pe::AbstractPriorEstimator, T::Integer)
kept_scenarios(pe::EmpiricalPrior, T::Integer)
kept_scenarios(pe::Union{<:FactorPrior, <:FactorBlackLittermanPrior,
                         <:BlackLittermanPrior, <:BayesianBlackLittermanPrior,
                         <:EntropyPoolingPrior, <:MeucciEntropyPoolingPrior,
                         <:HighOrderPriorEstimator, <:HighOrderFactorPriorEstimator},
               T::Integer)
kept_scenarios(pe::AugmentedBlackLittermanPrior, T::Integer)
kept_scenarios(pe::OpinionPoolingPrior, T::Integer)

Return the number of scenarios that a fit of pe on T observations carries, when the configuration of pe alone states it, and nothing when the data decides it.

scenario_count reads this function first, so a prior whose count the configuration states needs no data to count its scenarios. A Scenario Cap keeps the last pe.max_scenarios rows, and a prior that wraps another carries the rows of the prior that it wraps. A CrossSectionalFactorPrior drops the rows that its Descriptors warm up over, which the data decides, so it answers nothing.

Algorithm

The method that Julia selects is the algorithm.

  1. Any other prior: return T when keeps_observations returns true for pe, and nothing otherwise.
  2. An EmpiricalPrior: return T, or pe.max_scenarios when it is set and smaller, because scenario_window keeps the last pe.max_scenarios rows.
  3. A prior whose result carries the scenarios of the prior in its pe field: return this function of pe.pe. A factor prior fits pe.pe on the factor returns, which have the rows of the asset returns.
  4. An AugmentedBlackLittermanPrior: return the smaller of this function of pe.a_pe and of pe.f_pe, because the result keeps the rows that both nested priors keep. Return nothing when either one is nothing.
  5. An OpinionPoolingPrior: return T when pe.pe1 is nothing, and this function of pe.pe1 otherwise.

Arguments

  • pe: Prior estimator.
  • T: The number of observations that the fit of pe receives.

Returns

  • n::Option{Int}: The number of scenarios of the fit, or nothing when the data decides it.

Related

source
PortfolioOptimisers.scenario_count — Function
scenario_count(pe::AbstractPriorEstimator, X::MatNum, F::Option{<:MatNum},
               pnl::Option{<:AssetPanel}; strict::Bool = false, kwargs...)
scenario_count(pe::Union{<:FactorPrior, <:FactorBlackLittermanPrior}, X::MatNum,
               F::MatNum, pnl::Option{<:AssetPanel}; strict::Bool = false, kwargs...)
scenario_count(pe::Union{<:BlackLittermanPrior, <:BayesianBlackLittermanPrior,
                         <:EntropyPoolingPrior, <:MeucciEntropyPoolingPrior,
                         <:HighOrderPriorEstimator, <:HighOrderFactorPriorEstimator},
               X::MatNum, F::Option{<:MatNum}, pnl::Option{<:AssetPanel}; kwargs...)
scenario_count(pe::AugmentedBlackLittermanPrior, X::MatNum, F::MatNum,
               pnl::Option{<:AssetPanel}; strict::Bool = false, kwargs...)
scenario_count(pe::OpinionPoolingPrior, X::MatNum, F::Option{<:MatNum},
               pnl::Option{<:AssetPanel}; kwargs...)
scenario_count(pe::CrossSectionalFactorPrior, X::MatNum, F::Option{<:MatNum},
               pnl::AssetPanel; strict::Bool = false, ne::Option{<:VecStr} = nothing,
               E::Option{<:MatNum} = nothing, iv::Option{<:MatNum} = nothing,
               ivpa::Option{<:Num_VecNum} = nothing, kwargs...)

Return the number of scenarios that the fit of pe on X, F and pnl carries, with the least work that states it.

A prior that reweights observations, such as EntropyPoolingPrior, checks a caller's prior probabilities against the scenarios of the prior that it wraps, before it fits that prior under them. A fit states the count, but the count needs only the steps that drop rows. When the configuration states the count, kept_scenarios gives it with no fit. A CrossSectionalFactorPrior drops the rows that its Descriptors warm up over, which the data decides, so its count runs its exposure stage, and not its regression, its factor moments or its assembly.

Algorithm

The method that Julia selects is the algorithm.

  1. Any other prior: return kept_scenarios of pe on size(X, 1) observations when it is not nothing. Otherwise fit pe and return the row count of its scenarios.
  2. A FactorPrior or a FactorBlackLittermanPrior: return this function of pe.pe on F alone, as the fit of pe fits pe.pe. The asset rows are cut to the rows of the factor prior.
  3. A prior whose result carries the scenarios of the prior in its pe field: return this function of pe.pe on X, F and pnl.
  4. An AugmentedBlackLittermanPrior: return the smaller of this function of pe.a_pe on X and pnl, and of pe.f_pe on F alone, because the result keeps the rows that both nested priors keep.
  5. An OpinionPoolingPrior: return size(X, 1) when pe.pe1 is nothing, and this function of pe.pe1 otherwise.
  6. A CrossSectionalFactorPrior: take the rows rw after the Descriptor warm-up with cross_sectional_exposure_stage, and check them against the exposure lag with assert_cross_sectional_lag_rows. The regression answers n = length(rw) - pe.lag factor returns. Return kept_scenarios of pe.pe on n observations when it is not nothing. Otherwise take the factor returns with cross_sectional_factor_scenarios, and return this function of pe.pe on them alone.

Arguments

  • pe: Prior estimator.
  • X: Asset returns matrix, already oriented.
  • F: Optional factor returns matrix, already oriented.
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.
  • strict: If true, throws error for missing assets; otherwise, issue warnings.
  • ne, E, iv, ivpa: As in the returns-matrix method of prior, with E already oriented.
  • kwargs...: Additional keyword arguments of the fit of pe.

Validation

  • The rules of every step of the fit that the method runs.

Returns

  • T::Int: The number of scenarios that the fit of pe carries.

Related

source
PortfolioOptimisers.assert_cross_sectional_lag_rows — Function
assert_cross_sectional_lag_rows(t::Integer, lag::Integer)

Check that the t rows after the Descriptor warm-up of a CrossSectionalFactorPrior leave at least two factor returns after the exposure lag lag.

The regression reads the exposures of the row lag observations before each return, so it answers t - lag factor returns. The factor prior states a covariance of them, and a covariance of one observation is not a number. cross_sectional_regression_stage and scenario_count both read this check, so a count refuses the rows that the fit refuses.

Arguments

  • t: The number of rows after the Descriptor warm-up.
  • lag: The exposure lag, pe.lag.

Validation

  • t > lag. Raises an ArgumentError.
  • t - lag >= 2. Raises an ArgumentError.

Returns

  • nothing.

Related

source
PortfolioOptimisers.scenario_ens — Function
scenario_ens(::Nothing, X::MatNum)
scenario_ens(max_scenarios::Integer, X::MatNum)

State the number of observations the moments of a capped Prior Result were fitted over, or nothing when the cap cuts nothing.

The count a Scenario Cap owes its readers. scenario_window cuts the rows the result carries and leaves mu and sigma fitted over every observation, so a consumer that prices a sample size off size(pr.X, 1) — an uncertainty set's T, a calibration rule's count — would read w rows for moments fitted over t, and mis-price every count by t / w. The result therefore states t in ens exactly when the cap cuts, and every count reader takes ens before the shape. When the cap does not cut, or there is no cap, the rows carried are the observations fitted over, and ens stays nothing, so a fit without a cap is bit-identical to what it was.

It is the same verb in batch and in prior(pe) with no data after a fold, as scenario_window is, and the two agree by construction: both read the same t off the same matrix.

Arguments

  • max_scenarios: The number of observations carried, or nothing.
  • X: Asset returns the moments were fitted over, observations × assets, before the cut.

Returns

  • ens::Option{<:Integer}: size(X, 1) when max_scenarios cuts it, nothing otherwise.

Related

source
PortfolioOptimisers.scenario_fill_report — Function
scenario_fill_report(filled, ::Nothing)
scenario_fill_report(filled, named::AbstractSet{<:Integer})

Narrow the filled pairs a scenario_fill reports to the assets it has not named before.

A batch fit calls scenario_fill once, so it reports everything it finds and keeps no memory: that is the nothing method. A walk-forward reading out at every step calls it once per step over a growing window, and every step would otherwise name the same assets again, so a caller who read the first notice learns to ignore the channel and misses the asset that lists at step 500.

So a carry state holds the set of assets it has already named, and this verb narrows the report to the pairs of assets outside it. The set is the state's memory, and scenario_fill_remember! is what writes to it — only for a notice that actually fired, so an asset whose share was under the limit at one step is still named at the step where it goes over.

Arguments

  • filled: The (observation, asset) pairs scenario_fill_pairs found.
  • named: The assets already named, or nothing when nothing remembers.

Returns

  • report::AbstractVector{<:Tuple{Integer, Integer}}: The pairs to name, which is filled itself when nothing remembers.

Related

source
PortfolioOptimisers.scenario_fill_remember! — Function
scenario_fill_remember!(::Nothing, report)
scenario_fill_remember!(named::AbstractSet{<:Integer}, report)

Record the assets a scenario_fill has just named, so a later call of prior(pe) with no data does not name them again.

The write side of scenario_fill_report. It runs after the notice, not before it, so an asset is remembered exactly when a caller was told about it: under strict the notice throws and nothing is remembered, and under a limit the notice did not trip nothing is remembered either.

The set is a field of a PriorCarryState and is written in place, because prior(pe) with no data returns a Prior Result rather than the estimator, so there is no other channel by which the memory could survive the call.

Arguments

  • named: The assets already named, or nothing when nothing remembers.
  • report: The (observation, asset) pairs just named.

Returns

  • nothing.

Related

source
PortfolioOptimisers.scenario_fill — Function
scenario_fill(
    X::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    mu::AbstractVector{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    sigma::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    strict::Bool,
    fill_limit::Union{Nothing, Real}
) -> Any
scenario_fill(
    X::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    mu::AbstractVector{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    sigma::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    strict::Bool,
    fill_limit::Union{Nothing, Real},
    named::Union{Nothing, AbstractSet{<:Integer}}
) -> Any

Return the returns matrix with the missing rows of every investable asset filled with zero, and say so above the fitting estimator's resolved fill_limit.

A mask-aware moment estimator answers a young asset from the observations it has, so the asset is investable and its returns column still carries a NaN at every row before it listed. Every consumer of a Prior Result reads that column — the JuMP model, the meta-optimisers and the value-level expected_* functions among them — so the fill is paid once, here, on the estimator's own pass, rather than at each of them.

A non-investable asset keeps its NaN column, so investable_mask is unchanged. mu, sigma and every other block are untouched, because the estimator computed them from the rows it saw.

The share the fill tests is per asset: the worst investable column's own count of filled entries over the number of observations. A matrix-wide denominator scales with the universe, so the column the notice exists to catch disappears inside it — one asset of a hundred whose column is seven-tenths invented is seven thousandths of the matrix, under any limit a caller would set. The matrix-wide share is reported in the message as context and trips nothing.

How much of the trade passes in silence is the fitting estimator's own answer, carried in its fill_limit field and resolved at the fit by resolve_fill_limit against the coverage floor of its arms — the share is a property of one fit, not of the session, so two priors in one program may answer differently. Under strict = false the trade is silent while the worst column's share stays at or below the resolved limit, and is named through strict_diagnostic above it. Under strict = true any fill refuses, whatever the limit holds.

Two mask-aware families reach this verb. The exponentially weighted family carries no CoveragePolicy, so a fill_limit of nothing names every fill there. The plain family is mask-aware exactly where a policy is set, and a policy derives a limit that never fires, so an available-case fit names nothing while it does what it was configured to do. A plain estimator with no policy never reaches this verb at all: under the whole-window rule, an asset it could not cover leaves the Coverage Universe and is not investable.

Algorithm

  1. Return X itself when every entry of X is finite, which is every fit over a complete window.
  2. Derive the Investable Mask from mu and the diagonal of sigma, as investable_mask does, and find the pairs to fill with scenario_fill_pairs. Return X itself when there are none, which is a gap that belongs to a non-investable asset alone.
  3. Narrow the pairs to report with scenario_fill_report, which drops the assets a carry state has already named and is the identity when nothing remembers. Under strict the narrowing is skipped, because strict refuses any fill and must not depend on how often the estimator has been read out.
  4. Count the reported pairs per asset, take the worst column's share of its own observations, and report through strict_diagnostic when strict holds, when fill_limit is nothing, or when that share exceeds fill_limit. Record the assets just named with scenario_fill_remember!.
  5. Return a copy of X with zero written at each of the filled pairs — every one of them, not just the reported ones — and the NaN of every non-investable asset left where it is.

The notice fires once per asset, not once per step

named is what separates the batch call from the online one. A batch fit passes nothing, reports every fill it finds, and remembers nothing. A call of prior(pe) with no data on a folded prior passes the set its PriorCarryState carries, so a walk-forward names an asset at the step it lists and stays quiet afterwards instead of emitting the same notice at every one of two thousand steps. The fill itself is unchanged: every filled pair is written at every call, whatever the set holds. Under strict = true the first fill still throws.

Arguments

  • X: Asset returns, observations × assets.
  • mu: The expected returns the estimator answered, NaN at a non-investable asset.
  • sigma: The covariance the estimator answered, NaN on the diagonal at a non-investable asset.
  • strict: If true, any fill raises an ArgumentError; if false, a fill above the share warns.
  • fill_limit: The share of an investable column the fitting estimator accepts in silence, after resolve_fill_limit, or nothing when it accepts none.
  • named: The assets already named, written in place when a notice fires, or nothing when nothing remembers.

Validation

  • The worst investable column's filled share is at or below fill_limit, else a warning naming the assets is emitted, or an ArgumentError naming them is raised under strict, which any fill raises. A fill_limit of nothing is at or below no share, so any fill is named.

Returns

  • X::MatNum: The returns matrix whose investable columns are finite.

Related

source