Prior partial fit (a): private API

PortfolioOptimisers.PriorCarryState — Type
struct PriorCarryState{__T_buf, __T_named} <: AbstractPartialFitState

Carries the observations that a fold-and-carry prior keeps, and the assets whose scenario fill it has already named.

A prior that carries this state folds its moments exactly, member by member, and keeps the rows for one reason. LowOrderPrior carries X for the scenario risk measures, so prior(pe) with no data copies the rows into the result and computes nothing from them. A SampleBufferState is different. An estimator that carries one has no recursion of its own, and its call with no data runs the batch verb over the rows that the buffer kept.

An Online wrapper seeds a SampleBufferState, so a wrapped prior takes the refit route and not this one. This is why the max_history of an Online windows the whole fit.

Fields

  • buf: Buffer of the observations that the prior carries, observations × assets, held as given, NaN entries included.
  • named: Indices of the assets whose scenario fill a call of prior(pe) with no data has already named. strict_diagnostic keeps no record, so without this set a walk-forward that reads out at every step names the same asset at every step. That call adds to the set in place, because it returns a Prior Result and not the estimator, and no other return value takes the set to the next step. Base.copy copies the set, and merge_states takes the union of two.

Constructors

PriorCarryState(;    buf::SampleBufferState = SampleBufferState(),    named::Set{Int} = Set{Int}()) -> PriorCarryState

Keywords correspond to the struct's fields. The default is the empty state that a first fold builds.

View parameters

port_opt_view takes the view of its fields at the selected assets:

  • buf: Sliced to the selected indices through port_opt_view.
  • named: Renumbered onto the selected indices, so an asset named before the slice is still named after it.

Related

source
PortfolioOptimisers.returns_buffer — Function
returns_buffer(state::SampleBufferState)
returns_buffer(state::PriorCarryState)
returns_buffer(state::CrossSectionalCarryState)
returns_buffer(state::AbstractPartialFitState)

Reads the buffer of asset returns out of the state that a prior carries.

Three states hold rows at the prior layer, each in a different place. A SampleBufferState is the rows. A PriorCarryState and a CrossSectionalCarryState keep them in buf. optimise(opt) with no data, and prior(pe) with no data on a prior that forwards its fold to an embedded prior, read the returns through this function. Neither reads the fields of a state, so a new state that holds rows adds one method here. Where the tree of the prior reads factor returns, the buffer holds the factor rows too, and optimise(opt) with no data takes them with factor_buffer when the fold context keeps no factor column. This function throws an ArgumentError that names the type of a state that holds no rows, such as the exact-fold state of a moment estimator.

Arguments

  • state: The state that the prior carries.

Validation

  • state holds rows. An ArgumentError is thrown otherwise.

Returns

  • buffer::SampleBufferState: The rows, the masks folded beside them, and the factor rows where the prior records them.

Related

source
PortfolioOptimisers.prior_returns_buffer — Function
prior_returns_buffer(pe::AbstractPriorEstimator)
prior_returns_buffer(pe::Union{HighOrderPriorEstimator, BlackLittermanPrior})

Reads the buffer of asset returns out of the prior that owns the folded rows.

One prior in the chain owns the rows. HighOrderPriorEstimator and BlackLittermanPrior build their result around the prior they embed and own no rows, so this function reads the buffer of the embedded prior, at any depth. Every other prior keeps its rows in its own cache. Two calls with no data use it. optimise(opt) with no data rebuilds the returns it forwarded, and prior(pe) with no data on a HighOrderPriorEstimator refits a co-moment that does not fold over the rows.

Arguments

  • pe: The prior estimator that received the observations.

Validation

Returns

  • buffer::SampleBufferState: The rows that the prior folded, and the masks beside them.

Related

source
PortfolioOptimisers.fold_carry — Function
fold_carry(cache::Option{<:PriorCarryState}, x::VecNum; kwargs...)
fold_carry(cache::Option{<:PriorCarryState}, X::MatNum; dims::Int = 1, kwargs...)

Folds observations into a PriorCarryState, and seeds an empty state where the prior carries none.

The state folds its SampleBufferState and keeps the named-asset set. The seed has no cap. The cap on the carried rows is the max_scenarios of an EmpiricalPrior, which prior(pe) with no data applies. A cap on the fit is the max_history of an Online, which puts the prior on the refit route instead.

Arguments

  • cache: The state that the prior carries, or nothing.
  • x: One observation, with one entry per asset.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which the computation runs.
  • kwargs...: Additional keyword arguments, forwarded to the fold of the buffer.

Returns

  • state::PriorCarryState: The state after the last observation.

Related

source
Base.copy — Method
copy(x::PriorCarryState) -> PriorCarryState

Copies a PriorCarryState, so that the copy shares no array and no set with the original.

The method copies the named-asset set for the reason it copies the backing matrix. prior(pe) with no data writes into the set, so a copy that shared it would record its notices in the set of the original.

Arguments

  • x: The state to copy.

Returns

  • state::PriorCarryState: A new state, equal to x.

Related

source
PortfolioOptimisers.buffer_prior — Function
buffer_prior(pe::AbstractPriorEstimator, state::SampleBufferState, pnl::Nothing; kwargs...)
buffer_prior(pe::AbstractPriorEstimator, state::SampleBufferState, pnl::AssetPanel; kwargs...)

Runs the batch verb of a prior over the rows of its sample buffer, with the masks as keywords or inside the rebuilt Asset Panel, and with the Exogenous Series and the implied volatilities that the buffer records.

This is the call with no data of the refit route. A buffer that records no Panel Field gives its masks as the keywords that sample_buffer_kwargs reads, as a matrix call of the batch verb takes them. A buffer that records Panel Fields gives the panel that sample_buffer_panel rebuilds as the third positional argument, as prior(pe, rd) gives rd.pnl. That panel holds both masks, so the call gives no mask keyword. A buffer that records the Exogenous Series gives ne and E as the keywords that exogenous_buffer_kwargs reads, as prior(pe, rd) gives rd.ne and rd.E. A buffer that records implied volatilities gives iv and the pinned ivpa as the keywords that implied_volatility_buffer_kwargs reads, as prior(pe, rd) gives rd.iv and rd.ivpa.

Algorithm

The method that Julia selects is the algorithm.

  1. pnl is nothing: call prior(pe, X, F; masks..., series..., vols..., kwargs...) over the rows of the buffer, with the keywords of sample_buffer_fit_kwargs.
  2. pnl is an Asset Panel: call prior(pe, X, F, pnl; series..., vols..., kwargs...) over the rows of the buffer, with the keywords of sample_buffer_fit_kwargs.

Arguments

  • pe: Prior estimator whose cache holds state.
  • state: The buffer to read.
  • pnl: The panel of the rows of the buffer, or nothing.
  • kwargs...: Additional keyword arguments, forwarded to the batch verb.

Returns

  • pr::AbstractPriorResult: The batch fit over the rows of the buffer.

Related

source
PortfolioOptimisers.buffer_row_key — Function
buffer_row_key(pr::AbstractPriorResult, max_history::Nothing)
buffer_row_key(pr::LowOrderPrior, max_history::Integer)
buffer_row_key(pr::HighOrderPrior, max_history::Integer)

Correct the row key of the factor model block of a prior that a sample buffer refitted.

The batch verb counts the position of each row from the first row of the buffer. A buffer with no cap holds every folded observation, so these positions are the positions among the folded observations, as the carry fold of a CrossSectionalFactorPrior counts them. A capped buffer drops its oldest observations and does not record how many it dropped, so its positions are not the positions of the folded observations. The method then removes the positions from the block and keeps the timestamps. A realised factor_attribution of a cross-validation then lines the series up at the tail and does not match the wrong rows.

Arguments

  • pr: The prior that the batch verb fitted over the rows of the buffer.
  • max_history: The cap of the buffer, or nothing.

Returns

  • pr::AbstractPriorResult: pr itself with no cap, and otherwise pr with a factor model block that carries no positions.

Related

source
PortfolioOptimisers.forget_row_positions — Function
forget_row_positions(rr::Option{<:AbstractLoadingsRegressionResult})
forget_row_positions(rr::Regression)
forget_row_positions(rr::CrossSectionalFactorModel)

Return a factor model block with no positions in its row key, and every other field unchanged.

A block that records no row key, or no block at all, is returned as it is.

Arguments

  • rr: The factor model block, or nothing.

Returns

  • rr: The block with idx = nothing, or rr itself.

Related

source
PortfolioOptimisers.needs_factor_returns — Function
needs_factor_returns(pe::AbstractHiLoOrderPriorEstimator_F) -> true
needs_factor_returns(pe::AbstractLowOrderPriorEstimator_A) -> false
needs_factor_returns(pe::AbstractLowOrderPriorEstimator_AF) -> nothing
needs_factor_returns(pe::Online)
needs_factor_returns(pe::HighOrderPriorEstimator)
needs_factor_returns(pe::BlackLittermanPrior)
needs_factor_returns(pe::EntropyPoolingPrior)
needs_factor_returns(pe::MeucciEntropyPoolingPrior)
needs_factor_returns(pe::OpinionPoolingPrior)

Answers whether the fit of a prior reads a factor matrix, from its estimator tree.

The answer depends on the tree that an estimator embeds, not on the type of the outer estimator. A prior whose factor argument is optional never reads the argument, and passes it to the prior it embeds. Only a member that requires the argument reads it. So the answer has three values, and the abstract type of the member gives each one:

  • true for a member that requires factor returns, AbstractHiLoOrderPriorEstimator_F. Its batch verb declares F::MatNum with no default, and refuses a fit without one.
  • false for a member that never reads them, AbstractLowOrderPriorEstimator_A. Its batch verb declares the argument and ignores it, so a fold drops it as the batch verb does.
  • nothing for a member whose factor argument is optional, AbstractLowOrderPriorEstimator_AF. The type does not say, so the fold takes what it receives, as the batch verb does.

Each outer estimator of the library whose factor argument is optional recurses into the prior it embeds and answers the value of the leaf. So EntropyPoolingPrior(; pe = FactorPrior()) answers true, and EntropyPoolingPrior() answers false. OpinionPoolingPrior holds several priors and combines their answers with combine_factor_answers. An Online answers for the estimator it wraps. A caller's own optional-argument subtype that embeds a prior must define the recursion. A subtype that reads F itself can keep the default, which takes what it receives.

The checks at entry that look for a missing factor matrix read this predicate. These are the ReturnsResult method of the prior, the step of the online optimiser, and the ReturnsResult methods of ucs, mu_ucs and sigma_ucs. The predicate finds a factor leaf under an outer estimator whose factor argument is optional, which an isa test on the outer estimator cannot find. It also decides what the refit route of partial_fit! does with the factor observation it receives.

Arguments

  • pe: The prior estimator, or the wrapper around one.

Returns

  • needs::Union{Bool, Nothing}: true, false or nothing, as the list above states.

Related

source
PortfolioOptimisers.combine_factor_answers — Function
combine_factor_answers(answers) -> Union{Nothing, Bool}

Combines the answers of several embedded priors into one, for an outer estimator that holds more than one.

The rule is the strong three-valued disjunction of Kleene, with nothing as the unknown value. A fit that reaches a member which requires factor returns is refused without them, so one true decides the answer. A fit whose members all drop factor returns drops them too. In every other case at least one member takes what it receives and no member requires it, so the answer is nothing.

Mathematical definition

\[\begin{align} c(a_1, \ldots, a_K) &= \begin{cases} \top & \text{if } a_k = \top \text{ for some } k\,, \\ \bot & \text{if } a_k = \bot \text{ for every } k\,, \\ \mathrm{U} & \text{otherwise}\,. \end{cases} \end{align}\]

Where:

  • $c$: Combined answer.
  • $a_k$: Answer of member $k$, which is $\top$ for true, $\bot$ for false and $\mathrm{U}$ for nothing.
  • $K$: Number of members.

The combination is commutative and associative, so neither the order nor the grouping of the members changes the answer.

Arguments

  • answers: The answers of the members, each true, false or nothing.

Returns

  • needs::Union{Bool, Nothing}: The combined answer.

Related

source
PortfolioOptimisers.reads_panel_fields — Function
reads_panel_fields(pe::AbstractPriorEstimator) -> false
reads_panel_fields(pe::CrossSectionalFactorPrior) -> true
reads_panel_fields(pe::Nothing) -> false
reads_panel_fields(pe::Online)
reads_panel_fields(pe::Union{HighOrderPriorEstimator, BlackLittermanPrior,
                             BayesianBlackLittermanPrior, HighOrderFactorPriorEstimator,
                             MeucciEntropyPoolingPrior, EntropyPoolingPrior})
reads_panel_fields(pe::AugmentedBlackLittermanPrior)
reads_panel_fields(pe::OpinionPoolingPrior)

Answers whether the fit of a prior reads the Panel Fields of its Asset Panel, from its estimator tree.

Every prior receives the Asset Panel, but most read its masks alone. A CrossSectionalFactorPrior computes its factors from the Panel Fields at each row, so for it a time-varying Panel Field is sample. The online step reads this predicate. It gives the Panel Fields to the buffer of a prior whose tree reads them, and it refuses them for any other prior. The buffer then records them, and the call with no data rebuilds the panel.

Each outer estimator of the library that embeds a prior passes the panel to it, and never reads a Panel Field itself. So each one recurses into the prior it embeds, as needs_factor_returns does. OpinionPoolingPrior answers true when any prior it holds reads them. An Online answers for the estimator it wraps. The answer is a Bool, because no prior reads the Panel Fields as an option. A caller's own prior that reads them must define a method that answers true.

Arguments

  • pe: The prior estimator, the wrapper around one, or nothing.

Returns

  • reads::Bool: true when the tree holds a prior that reads the Panel Fields.

Related

source
PortfolioOptimisers.reads_exogenous_series — Function
reads_exogenous_series(est::AbstractEstimator, rd::Option{<:ReturnsResult} = nothing) -> false
reads_exogenous_series(est::Nothing, rd::Option{<:ReturnsResult} = nothing) -> false
reads_exogenous_series(ests::AbstractVector, rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(xe::ObservedExposure, rd::Option{<:ReturnsResult} = nothing) -> true
reads_exogenous_series(xe::CurrencyExposure, rd::Nothing = nothing) -> true
reads_exogenous_series(xe::CurrencyExposure, rd::ReturnsResult) -> Bool
reads_exogenous_series(de::EWMacroSensitivity, rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(xe::CompositeExposure, rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(ds::DescriptorScores, rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(rfe::Union{FixedWeightedReturnForecast, ExpWeightedReturnForecast,
                                  TargetReturnForecast},
                       rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(pe::CrossSectionalFactorPrior, rd::Option{<:ReturnsResult} = nothing)
    -> Bool
reads_exogenous_series(pe::Online, rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(pe::Union{HighOrderPriorEstimator, BlackLittermanPrior,
                                 BayesianBlackLittermanPrior,
                                 HighOrderFactorPriorEstimator,
                                 MeucciEntropyPoolingPrior, EntropyPoolingPrior},
                       rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(pe::AugmentedBlackLittermanPrior,
                       rd::Option{<:ReturnsResult} = nothing) -> Bool
reads_exogenous_series(pe::OpinionPoolingPrior, rd::Option{<:ReturnsResult} = nothing)
    -> Bool

Answers whether the fit of an estimator reads the Exogenous Series rd.E, from its estimator tree, and on the data of a step when rd is given.

A prior that reads the series needs it at every fitted row, so for it the series is sample and not fold context. The refit route of the online step and the carry fold of a CrossSectionalFactorPrior read this predicate. Each gives the series to the buffer of a prior whose tree reads it, and refuses a step of such a prior that brings no series. The buffer then records every column of the series, and the Fold Context reads the series back from it.

The predicate is per type, so a member that reads the series answers for itself, whatever abstract type it has. An observed member, CurrencyExposure or ObservedExposure, reads the return of its factor from the series. A CurrencyExposure reads no series when every level of its Panel Field is its base currency, and the levels come from the data. So without rd it answers true, and with rd it answers whether it names a series on rd, with observed_series. Every other member answers the same with or without rd, and each node of the tree passes rd to its parts. An EWMacroSensitivity that names a column in series reads its reference return from the series, and one with series = nothing reads none. A member that holds Descriptors answers for them, and so does a Return Forecast Estimator through its DescriptorScores. A CrossSectionalFactorPrior answers for its factor list and its Return Forecast Estimator. Each outer prior answers for the prior it embeds, as reads_panel_fields does, and an Online answers for the estimator it wraps. A caller's own member that reads the series must define a method that answers true.

Arguments

  • est: An estimator of the tree, or nothing.
  • ests: A vector of estimators.
  • rd: The returns data of a step, or nothing to answer from the configuration alone.

Returns

  • reads::Bool: true when the tree holds a member that reads the Exogenous Series, on rd when it is given.

Related

source
PortfolioOptimisers.assert_factor_returns — Function
assert_factor_returns(
    pe::Union{AbstractPriorEstimator, Online},
    F::Union{Nothing, AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}, AbstractVector{<:Union{Number, AbstractJuMPScalar}}}
)

Refuses a missing factor matrix at a check at entry whose prior tree requires one.

Five checks at entry share this refusal: the ReturnsResult method of the prior, the step of the online optimiser, and the ReturnsResult methods of ucs, mu_ucs and sigma_ucs. The refit route of partial_fit! reaches it too, through fold_factor_argument. One method holds the message and the test, so the two cannot differ between callers. The test is needs_factor_returns answering true. That predicate walks the estimator tree. So this method refuses a factor leaf under an outer estimator whose factor argument is optional with an IsNothingError that names F, before the leaf meets a MethodError one call later.

A pe of nothing is an uncertainty set with no prior of its own. It reads no factor matrix, so the method checks nothing. The returns-data method refuses such a set later, with an error that names it, through ucs_prior.

Arguments

  • pe: The prior estimator that the check at entry passes the matrix to, or nothing.
  • F: The factor matrix that the ReturnsResult holds, the factor observation that a fold receives, or nothing.

Validation

  • !isnothing(F), when needs_factor_returns(pe) === true. An IsNothingError is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_prior_fold_returns — Function
assert_prior_fold_returns(rd::ReturnsResult)

Refuses returns data that the fold of a prior cannot take.

Each form of partial_fit! of a prior over a ReturnsResult calls it first, and so does step_active_mask. A fold records the returns beside every other part of the step, so a step without returns has no rows to record. The implied volatilities have a slot in the SampleBufferState beside the returns, so this check takes them. The carry fold of a CrossSectionalFactorPrior has no such slot, and it refuses them itself.

Arguments

  • rd: The returns result to use.

Validation

  • rd.X is not nothing. An IsNothingError is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.fold_prior_step — Method
fold_prior_step(pe::AbstractPriorEstimator, rd::ReturnsResult, gaps::Option{<:AbstractVector{Bool}})
fold_prior_step(pe::CrossSectionalFactorPrior, rd::ReturnsResult, gaps::Option{<:AbstractVector{Bool}})

Takes the step of the refit route of a prior: the fold of refit_prior_fold, and the work a prior adds around it.

This is the verb of every route of the step of a prior. partial_fit! over a ReturnsResult marks the Shared Series gaps of the step and calls it, and the online step of an optimiser, fold_prior, calls it with the marks of its own Shared Series. The step is a function of its own, and not a method of partial_fit!, for two reasons. The marks are a fact of the returns data, and a prior at x_src = ExcessSource() folds the excess returns, which hold no benchmark, so the caller finds them from the returns data and not the prior. And every method of this verb that reads a family state narrows the cache of its estimator, so a buffer reaches the refit route alone.

The marks gaps travel into the buffer of the prior, so every fit through it drops the marked observations with shared_series_buffer. A prior that adds no work of its own folds rd with refit_prior_fold, and applies the Choice Rule of a regression that it fits with pin_buffer_choice, over the observations that a fit keeps. A CrossSectionalFactorPrior applies its Choice Rule after the fold, with cross_sectional_pin_choice. The carry of EmpiricalPrior, the carry of CrossSectionalFactorPrior, HighOrderPriorEstimator and BlackLittermanPrior have methods of their own.

Algorithm

The method that Julia selects is the algorithm.

  1. Any prior: fold rd and gaps with refit_prior_fold, and apply the Choice Rule over the rows of the buffer with pin_buffer_choice.
  2. A CrossSectionalFactorPrior: fold rd and gaps with refit_prior_fold, and pin the dropped members under a PinnedChoice.

Arguments

  • pe: The prior whose buffer the step folds forward.
  • rd: The returns result to use.
  • gaps: One mark for each observation of rd, true at an observation with a Shared Series gap, or nothing when no observation has one. See shared_series_gaps.

Validation

Returns

  • pe: The prior, with its cache field set to the buffer after the last observation.

Related

source
PortfolioOptimisers.refit_prior_fold — Function
refit_prior_fold(
    pe::AbstractPriorEstimator,
    rd::ReturnsResult,
    gaps::Union{Nothing, AbstractVector{Bool}}
) -> Any

Folds the observations of a ReturnsResult and the marks of their Shared Series gaps into the sample buffer of a prior. This is the refit route of partial_fit!.

The generic method of fold_prior_step is this function and the Choice Rule. A prior with a step of its own, such as CrossSectionalFactorPrior, calls it for the fold and adds its own work around it. The buffer keeps every observation and records the marks, so the fits through it drop the marked observations, see shared_series_buffer.

Algorithm

  1. Refuse rd without returns with assert_prior_fold_returns.
  2. Apply the answer of needs_factor_returns to rd.F with fold_factor_argument.
  3. Read the buffer out of pe.cache with assert_sample_buffer.
  4. Read the masks and the Panel Fields of rd.pnl with refit_step_kwargs, and the Exogenous Series of rd with exogenous_step_kwargs. Fold rd.X, the factor returns, those keywords, rd.iv, rd.ivpa and gaps into the buffer through the block method of partial_fit!.
  5. Rebuild the prior with its cache set to the new buffer, and return it.

Arguments

  • pe: The prior whose buffer the function folds forward.
  • rd: The returns result to use.
  • gaps: One mark for each observation of rd, true at an observation with a Shared Series gap, or nothing.

Validation

Returns

  • pe: The prior, with its cache field set to the buffer after the last observation.

Related

source
PortfolioOptimisers.pin_buffer_choice — Function
pin_buffer_choice(pe::AbstractPriorEstimator)

Applies the Choice Rule of each regression that a prior fits over the rows of its sample buffer, after a fold of the refit route.

The two folds of the refit route call it: the matrix form of partial_fit! and fold_prior_step. It gives pin_prior_choice the data that the call with no data, buffer_prior, gives the batch verb: the rows that a fit keeps, which shared_series_buffer answers, the panel of sample_buffer_panel, and the keywords of sample_buffer_fit_kwargs. So a prior in the tree that reads a Panel Field, a mask or the Exogenous Series pins the choice of the batch fit over the same rows.

A fixed weight vector in the tree, such as the weights of the target of a regression, holds one entry for each observation of the buffer. When the buffer marks an observation for a Shared Series gap, the fit drops it, and the weight of the observation must drop too, as observed_weights states. So the function pins a copy of pe cut at the kept observations, with shared_series_buffer_fit. The step keeps its estimator for every later step, and a later buffer marks other observations, so the cut copy must not be the estimator that the step returns. adopt_pinned_choice writes the pinned choice of the copy into pe, and pe keeps its weights at the full length of the buffer.

Algorithm

  1. Return pe when pending_choice of pe is false, or when buffer_fits_none of pe.cache is true. The step then copies no row.
  2. Read the estimator and the rows that a fit keeps with shared_series_buffer_fit, giving cut and fit. cut is pe when the buffer marks no observation.
  3. Read the panel of fit with sample_buffer_panel, and the keywords of fit with sample_buffer_fit_kwargs.
  4. Apply pin_prior_choice to cut over the returns and the factor returns of fit, with no weights, the panel and the keywords, giving pinned.
  5. Return pinned when cut is pe. Otherwise write the choice of pinned into pe with adopt_pinned_choice.

Arguments

Validation

Returns

  • pe::AbstractPriorEstimator: The prior that the step returns.

Related

source
PortfolioOptimisers.pin_prior_choice — Function
pin_prior_choice(pe::Union{Nothing, AbstractPriorEstimator}, X, F, w = nothing,
                 pnl = nothing; kwargs...)
pin_prior_choice(pe::Union{FactorPrior, FactorBlackLittermanPrior,
                           AugmentedBlackLittermanPrior}, X::MatNum, F::MatNum,
                 w::Option{<:ObsWeights} = nothing,
                 pnl::Option{<:AssetPanel} = nothing; kwargs...)
pin_prior_choice(pe::Union{HighOrderPriorEstimator, HighOrderFactorPriorEstimator,
                           BlackLittermanPrior, BayesianBlackLittermanPrior,
                           EntropyPoolingPrior, MeucciEntropyPoolingPrior}, X, F,
                 w = nothing, pnl = nothing; kwargs...)
pin_prior_choice(pe::OpinionPoolingPrior, X::MatNum, F::MatNum,
                 w::Option{<:ObsWeights} = nothing,
                 pnl::Option{<:AssetPanel} = nothing; kwargs...)

Applies the Choice Rule of each regression that a prior fits, after an online step of the refit route, and in the batch fit of an entropy-pooling prior.

A StepwiseRegression or a DimensionReductionRegression under a PinnedChoice keeps the choice of its first fit. The step holds no state for it: it writes the choice into the configuration of the regression, with pin_regression_choice, over the rows of the buffer. A prior that embeds another prior fits the embedded prior on the same returns and factor returns, so the method passes the same rows to it. A prior with no regression in its tree returns unchanged.

An OpinionPoolingPrior does not give each prior the same rows. Its batch fit gives pe1 the returns of the buffer, gives each prior of pes the returns of pe1, and refits pe2 on those returns under the pooled weights. The method gives each prior the rows and the weights that the batch fit gives it, so the choice that it pins is the choice of the batch fit over the same rows. It runs pe1 and the pool only when a prior of pes or pe2 has a choice to pin, which pending_choice answers.

Every method passes pnl and kwargs to each prior in the tree, as the batch fit passes its panel and its keywords. So a prior that reads a Panel Field, a mask or the Exogenous Series runs the fit of the batch fit. pin_buffer_choice gives the panel and the keywords of the buffer.

An EntropyPoolingPrior or a MeucciEntropyPoolingPrior fits pe.pe under its prior probabilities w0, builds its views on that fit, and refits pe.pe under the posterior weights. Its batch fit applies this function to itself at the w0 fit, so every refit reads the choice that the views read. The method pins pe.pe under the weights of that fit, which embedded_pin_weights answers, so the batch fit and the step pin the same choice, and the step runs no view and no solve.

Algorithm

The method that Julia selects is the algorithm.

  1. Any other prior, nothing, or F that is nothing: return pe.
  2. A prior that fits pe.re on the returns: apply pin_regression_choice to pe.re under scenario_weights of w on the rows of F, and rebuild pe with the answer when it differs. A Scenario Cap makes w shorter than F, and the rows that the cap removed take a zero weight, as in the batch fit.
  3. A prior that embeds a prior in pe.pe: apply this function to pe.pe under the weights of embedded_pin_weights, and rebuild pe with the answer when it differs.
  4. An OpinionPoolingPrior:
    1. Return pe when F holds fewer than two rows, or when pending_choice of pe is false.
    2. Apply this function to pe.pe1 over X under w.
    3. When a prior of pe.pes or pe.pe2 has a choice to pin, read the returns X1 of pe.pe1 over X, pnl and kwargs, or take X when pe.pe1 is nothing. Apply this function to each prior of pe.pes over X1 under w.
    4. When pe.pe2 has a choice to pin, pool the opinions over X1, pnl and kwargs with opinion_pooling_weights, giving the pooled weights, and apply this function to pe.pe2 over X1 under them.
    5. Rebuild pe with the answers.

When w is not nothing, the method reads factory(pe.pe1, w) and factory(pe, w), as a batch fit of factory(pe, w) does.

Arguments

  • pe: The prior after the fold of the step.
  • X: The returns of the rows of the buffer, observations × assets.
  • F: The factor returns of the rows of the buffer, observations × factors, or nothing.
  • w: The observation weights under which the batch fit makes the choice that the method pins, or nothing when that fit is unweighted.
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.
  • kwargs: More keyword arguments.

Validation

Returns

  • pe::AbstractPriorEstimator: The prior that the step returns.

Related

source
PortfolioOptimisers.pending_choice — Function
pending_choice(est::Union{Nothing, AbstractEstimator})
pending_choice(re::StepwiseRegression{<:Any, <:Any, <:Any, <:PinnedChoice})
pending_choice(re::DimensionReductionRegression{<:Any, <:Any, <:Any, <:PinnedChoice,
                                                Nothing})
pending_choice(pe::Union{FactorPrior, FactorBlackLittermanPrior,
                         AugmentedBlackLittermanPrior})
pending_choice(pe::Union{HighOrderPriorEstimator, HighOrderFactorPriorEstimator,
                         BlackLittermanPrior, BayesianBlackLittermanPrior,
                         EntropyPoolingPrior, MeucciEntropyPoolingPrior})
pending_choice(pe::OpinionPoolingPrior)

Answers whether a regression, or a regression in the tree of a prior, holds a choice under a PinnedChoice that no fit has pinned yet.

pin_prior_choice reads it before it runs the opinions of an OpinionPoolingPrior over the rows of the buffer. That run is a batch fit of the opinions, so the step makes it only while a choice is still to pin.

Algorithm

The method that Julia selects is the algorithm.

  1. Any other estimator, or nothing: false.
  2. A StepwiseRegression under a PinnedChoice: true when re.included is nothing, or when one of its entries is nothing.
  3. A DimensionReductionRegression under a PinnedChoice whose proj is nothing: true.
  4. A prior that fits pe.re on the returns: the answer for pe.re.
  5. A prior that embeds a prior in pe.pe: the answer for pe.pe.
  6. An OpinionPoolingPrior: true when the answer for pe.pe1, for a prior of pe.pes, or for pe.pe2 is true.

Arguments

  • est, re, pe: The estimator to read.

Returns

  • flag::Bool: true when a choice is still to pin.

Related

source
PortfolioOptimisers.adopt_pinned_choice — Function
adopt_pinned_choice(est, pinned)
adopt_pinned_choice(re::StepwiseRegression{<:Any, <:Any, <:Any, <:PinnedChoice},
                    pinned::StepwiseRegression)
adopt_pinned_choice(re::DimensionReductionRegression{<:Any, <:Any, <:Any,
                                                     <:PinnedChoice, Nothing},
                    pinned::DimensionReductionRegression)
adopt_pinned_choice(pe::Union{FactorPrior, FactorBlackLittermanPrior,
                              AugmentedBlackLittermanPrior}, pinned)
adopt_pinned_choice(pe::Union{HighOrderPriorEstimator, HighOrderFactorPriorEstimator,
                              BlackLittermanPrior, BayesianBlackLittermanPrior,
                              EntropyPoolingPrior, MeucciEntropyPoolingPrior}, pinned)
adopt_pinned_choice(pe::OpinionPoolingPrior, pinned::OpinionPoolingPrior)

Write the choice that pin_prior_choice pinned into pinned into the estimator est, and keep every other field of est.

pin_buffer_choice pins a copy of the prior cut at the observations that a Shared Series gap keeps, because the fit that the pin runs drops the marked observations and their weights. The copy is the same tree as the prior, with its observation weights cut and with no cache. The pin writes only the choice fields, so this function reads the choice fields of the copy and writes them into the prior, which keeps its full weights and its buffer. The methods follow the tree of pending_choice.

Algorithm

The method that Julia selects on est is the algorithm.

  1. Any other value, nothing among them: return est. It holds no choice to pin. The method takes a value of any type, as the fallback of obs_weights_view does, because a field of a prior can hold a value that is not an estimator.
  2. A StepwiseRegression under a PinnedChoice: return re rebuilt with pinned.included.
  3. A DimensionReductionRegression under a PinnedChoice whose proj is nothing: return re when pinned.proj is nothing, and otherwise re rebuilt with pinned.proj.
  4. A prior that fits pe.re on the returns: apply this function to pe.re and pinned.re, and rebuild pe with the answer when it differs.
  5. A prior that embeds a prior in pe.pe: apply this function to pe.pe and pinned.pe, and rebuild pe with the answer when it differs.
  6. An OpinionPoolingPrior: apply this function to pe.pe1, to each prior of pe.pes and to pe.pe2, each with its counterpart in pinned, and rebuild pe with the answers.

Arguments

  • est, re, pe: The estimator that the step keeps, or another value of its tree.
  • pinned: The same estimator, cut at the kept observations, after pin_prior_choice.

Returns

  • est: The estimator with the pinned choice.

Related

source
PortfolioOptimisers.refit_step_kwargs — Function
refit_step_kwargs(pe::AbstractPriorEstimator, pnl::Nothing) -> NamedTuple
refit_step_kwargs(pe::AbstractPriorEstimator, pnl::AssetPanel{<:Any, Nothing, Nothing}) -> NamedTuple
refit_step_kwargs(pe::AbstractPriorEstimator, pnl::AssetPanel) -> NamedTuple

Reads the keywords that the refit route of a prior folds from an Asset Panel: both masks, and the Panel Fields when the prior reads them.

The refit honours everything that the batch fit reads from the panel, because its buffer records it. No panel and a static panel give no keyword. A static panel has no observation axis, so it gives no row to record.

Algorithm

The method that Julia selects is the algorithm.

  1. pnl is nothing or static: return an empty NamedTuple.
  2. pnl is time-varying: take active_mask = pnl.amsk and estimation_mask = pnl.emsk. Add panel_fields = pnl.pf when step_panel_fields answers true.

Arguments

  • pe: The prior.
  • pnl: The Asset Panel of the step, or nothing.

Validation

Returns

  • kwargs::NamedTuple: The keywords of the fold of the buffer.

Related

source
PortfolioOptimisers.exogenous_step_kwargs — Function
exogenous_step_kwargs(
    pe::AbstractPriorEstimator,
    rd::ReturnsResult
) -> Union{@NamedTuple{}, NamedTuple{(:ne, :E), <:Tuple{Any, Any}}}

Reads the keywords that the refit route of a prior, and the carry fold of a CrossSectionalFactorPrior, fold from the Exogenous Series of a step, and refuses a step that brings no series to a prior that reads it.

A prior whose tree reads the series, as reads_exogenous_series answers, owns it on the online step. Its buffer records every column of rd.E and the names rd.ne, and the first step fixes the names. A step that brings no series would leave the buffer with rows that the fit cannot read, so it is refused when the fit reads a series on the data of the step, as reads_exogenous_series(pe, rd) answers. A CurrencyExposure whose every level is its base currency reads none, so such a step gives no keyword, and the buffer records no series. The first step fixes whether the buffer records the series, and the buffer refuses a later step that disagrees. A prior that reads no series gives no keyword, and the Fold Context keeps the series. A non-finite value of the series is not refused here. The fit refuses it on the rows that it reads, and a later cap can drop such a row.

Arguments

  • pe: The prior.
  • rd: The returns result to use.

Validation

  • rd.E is not nothing when reads_exogenous_series(pe, rd) is true. An IsNothingError is thrown otherwise.

Returns

  • kwargs::NamedTuple: ne and E of rd when the prior reads the series and the step carries it, and an empty NamedTuple otherwise.

Related

source
PortfolioOptimisers.step_panel_fields — Function
step_panel_fields(
    pe::AbstractPriorEstimator,
    pnl::AssetPanel,
    route::AbstractString
) -> Bool

Answers whether a step gives Panel Fields to the fold of a prior, and refuses Panel Fields that the prior does not read.

A buffer records the Panel Fields only for a prior whose tree reads them, as reads_panel_fields answers. For any other prior the buffer has no slot that the prior reads. So the step cannot keep the Panel Fields, and optimise(opt) with no returns cannot rebuild them. The refusal names the route of the prior.

Arguments

  • pe: The prior.
  • pnl: The time-varying Asset Panel of the step.
  • route: The name of the route, for the message.

Validation

  • pnl.pf is nothing, or reads_panel_fields(pe) is true. An ArgumentError is thrown otherwise.

Returns

  • gives::Bool: true when pnl holds Panel Fields and the prior reads them.

Related

source
PortfolioOptimisers.fold_factor_argument — Function
fold_factor_argument(needs::Bool, pe::AbstractPriorEstimator, F::Option{<:VecNum_MatNum})
fold_factor_argument(::Nothing, ::AbstractPriorEstimator, F::Option{<:VecNum_MatNum})

Applies the answer of the estimator tree to the factor argument of a fold.

The method reads the three answers of needs_factor_returns as the refit route needs them. true refuses a missing F with an IsNothingError that names it, and passes a present one through. false drops F. nothing passes F through as it is. The method dispatches on the answer, so a tree whose answer follows from its type, which is true of every tree in the library, costs the fold no branch at run time.

Arguments

  • needs: The answer of needs_factor_returns.
  • pe: The prior, for the message of the refusal.
  • F: The factor argument of the fold, or nothing.

Validation

  • F is not nothing when needs is true. An IsNothingError is thrown otherwise.

Returns

  • F: The factor argument that the buffer records, or nothing.

Related

source
PortfolioOptimisers.fold_member — Function
fold_member(est, args...; kwargs...)

Folds an observation into a member of an outer estimator that carries the observations, where that member folds.

This is the fold half of the rule for an outer estimator with mixed members. An outer estimator that carries the observations folds every member that folds, and leaves the other members as they are, because it can run their batch verb over its own rows in the call with no data. supports_partial_fit answers the question from the type of the member and its cache field. So for an outer estimator of concrete members the compiler resolves the branch, and the outer estimator folds no member that must not fold.

A member that the outer estimator does not hold is nothing, and a fold of it returns nothing.

Arguments

  • est: The member to fold, or nothing.
  • args...: The observations, forwarded to partial_fit!.
  • kwargs...: Additional keyword arguments, forwarded to partial_fit!.

Returns

  • est: The member with the state after the last observation, or the member unchanged where it does not fold.

Related

source
PortfolioOptimisers.fold_carry_members — Function
fold_carry_members(pe::EmpiricalPrior, X::VecNum_MatNum; kwargs...) -> EmpiricalPrior

Folds observations into the two members of an EmpiricalPrior on its carry route.

Each step of the carry folds pe.me and pe.ce on the same observations, so the four methods of the carry fold them here. The first step of the carry also refuses a member that holds a state, because the carried rows start empty and that member would read rows that they do not hold.

Algorithm

  1. Refuse a member that holds a state at the first step, when pe.cache is nothing, with assert_cold_carry_entry.
  2. Fold pe.me and then pe.ce with X through fold_member, and return the estimator with both members rebound.

Arguments

  • pe: Empirical prior estimator on its carry route.
  • X: The observations that the members fold, one observation as a vector or one per row of a matrix.
  • kwargs...: Additional keyword arguments, forwarded to the two folds.

Validation

Returns

  • pe: The estimator, with its two members folded.

Related

source
PortfolioOptimisers.read_member — Function
read_member(
    f,
    est,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    pnl::Union{Nothing, AssetPanel};
    batch,
    shared,
    kwargs...
) -> Any

Reads the estimate of a member out of its fold, or refits the member over the rows of the outer estimator.

This half of the rule for an outer estimator with mixed members reads the estimate, and fold_member is the fold half. A member that the outer estimator folded answers from its state. A member that the outer estimator did not fold answers from the matrix that the outer estimator passes. f is the batch verb of the member, for example Statistics.mean, Statistics.cov, coskewness or cokurtosis. The one-argument method of the same verb reads the estimate from the state, and every family that folds follows this convention.

The refit calls the panel form of the verb, f(est, X, pnl), as the batch fit of the outer estimator does. pnl is the panel that buffer_asset_panel rebuilds from the buffer of the outer estimator, or nothing. The panel form reduces the rows to the Coverage Universe and expands the estimate, so a refit answers a gapped sample as the batch fit answers it, and reads the active mask that the buffer records. The plain form f(est, X) refuses a gapped sample.

The decision to fold or to refit reads est, and the refit calls the verb on batch. These differ for a raw StatsBase.CovarianceEstimator, which has no panel form. Its prior passes the GeneralCovariance of library_covariance_estimator as batch, as the batch fit does. The decision must stay on the raw member, because supports_partial_fit answers true for the wrapper, and the wrapper folded nothing.

Arguments

  • f: The batch verb of the member.
  • est: The member, or nothing.
  • X: The rows that the outer estimator carries, which are the rows the member was folded on.
  • pnl: The Asset Panel of the rows, or nothing.
  • batch: The estimator that the refit gives to the verb. It is est by default.
  • shared: Keyword arguments that reach the verb whether the member folds or not. A prior passes strict to its covariance here, so that a peel of a CoveragePolicy refuses under strict in both arms.
  • kwargs...: Additional keyword arguments, forwarded to the batch verb alone. A folded member is read with no other keyword argument, because its state is already the estimate.

Returns

  • val: The estimate of the member, in the shape that its verb returns.

Related

source
PortfolioOptimisers.fold_prior_step — Method
fold_prior_step(pe::EmpiricalPrior{<:Any, <:Any, <:Any, <:Any, <:Any,
                                   <:Option{<:PriorCarryState}}, rd::ReturnsResult,
                gaps::Option{<:AbstractVector{Bool}})

Folds the observations of a ReturnsResult into an EmpiricalPrior on its carry route.

This is the carry form of fold_prior_step, the fold of every prior that partial_fit! over a ReturnsResult takes. The members fold the observations that gaps does not mark, and the carry state keeps every observation with its mark. The carry folds pe.me and pe.ce exactly, and the exact folds of the moment layer take an active mask and no estimation mask. So the route cannot honour an estimation mask that differs from the active mask, and it refuses one by name. The refit route honours it: wrap the prior in Online. The prior reads no Panel Field, so a Panel Field is refused as the refit route refuses it for such a prior.

Algorithm

  1. Refuse rd without returns with assert_prior_fold_returns.
  2. Refuse the Panel Fields and an estimation mask that differs from the active mask with assert_carry_step_panel.
  3. Fold rd.X with the active mask that step_active_kwargs reads, with rd.iv and rd.ivpa, and with the marks gaps, through the matrix form of the carry. The carried buffer records the implied volatilities, and the call with no data gives them to a member that it refits.

Arguments

  • pe: Empirical prior estimator on its carry route.
  • rd: The returns result to use.
  • gaps: One mark for each observation of rd, true at an observation with a Shared Series gap, or nothing.

Validation

Returns

  • pe: The estimator, with its members folded and its cache field rebound.

Related

source
PortfolioOptimisers.assert_carry_step_panel — Function
assert_carry_step_panel(pe::EmpiricalPrior, pnl::Nothing) -> nothing
assert_carry_step_panel(pe::EmpiricalPrior, pnl::AssetPanel{<:Any, Nothing, Nothing}) -> nothing
assert_carry_step_panel(pe::EmpiricalPrior, pnl::AssetPanel) -> nothing

Refuses the parts of a time-varying Asset Panel that the carry of an EmpiricalPrior cannot honour.

The carry folds its moments exactly, and those folds take the active mask alone. So the route refuses an estimation mask that differs from the active mask, and it names the route and the refit that honours it. The prior reads no Panel Field, so step_panel_fields refuses a panel that holds one.

Algorithm

The method that Julia selects is the algorithm.

  1. pnl is nothing or static: return.
  2. pnl is time-varying: refuse its Panel Fields with step_panel_fields. Refuse an emsk that differs from amsk, and count the cells that differ.

Arguments

  • pe: Empirical prior estimator on its carry route.
  • pnl: The Asset Panel of the step, or nothing.

Validation

  • The rules of step_panel_fields.
  • pnl.emsk == pnl.amsk. An ArgumentError is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.step_active_kwargs — Function
step_active_kwargs(pnl::Nothing) -> NamedTuple
step_active_kwargs(pnl::AssetPanel{<:Any, Nothing, Nothing}) -> NamedTuple
step_active_kwargs(pnl::AssetPanel) -> NamedTuple

Reads the active mask of the Asset Panel of a step, as the keyword that an exact fold takes.

The carry of an EmpiricalPrior and the co-moment members of a HighOrderPriorEstimator fold exactly, and their folds take the active mask alone. No panel and a static panel give no keyword.

Algorithm

The method that Julia selects is the algorithm.

  1. pnl is nothing or static: return an empty NamedTuple.
  2. pnl is time-varying: return (; active_mask = pnl.amsk).

Arguments

  • pnl: The Asset Panel of the step, or nothing.

Returns

  • kwargs::NamedTuple: The active mask keyword, or no keyword.

Related

source