The covariance forecast evaluation: private API

PortfolioOptimisers.forecast_location — Function

forecast_location(ce::Covariance, X::MatNum; dims::Int = 1, active_mask = nothing, kwargs...)
forecast_location(ce::GeneralCovariance, X::MatNum; dims::Int = 1, kwargs...)
forecast_location(ce::Union{<:ExpWeightedCovariance, <:RegimeAdjustedExpWeightedCovariance}, X::MatNum; dims::Int = 1, active_mask = nothing, kwargs...)
forecast_location(ce::Union{<:PortfolioOptimisersCovariance, <:CorrelationCovariance}, X::MatNum; kwargs...)
forecast_location(ce::AbstractCovarianceEstimator, X::MatNum; dims::Int = 1, kwargs...)
forecast_location(ce::AbstractCovarianceEstimator, X::MatNum, pnl::Option{<:AssetPanel}; dims::Int = 1, kwargs...)
forecast_location(ce::Union{<:ExpWeightedCovariance, <:RegimeAdjustedExpWeightedCovariance}, X::MatNum, pnl::Option{<:AssetPanel}; dims::Int = 1, kwargs...)
forecast_location(ce::Covariance)
forecast_location(ce::GeneralCovariance)
forecast_location(ce::Union{<:ExpWeightedCovariance, <:RegimeAdjustedExpWeightedCovariance})
forecast_location(ce::PortfolioOptimisersCovariance)
forecast_location(ce::CorrelationCovariance)
forecast_location(ce::AbstractCovarianceEstimator)
forecast_location(pe::AbstractPriorEstimator, rd::ReturnsResult; kwargs...)
forecast_location(pe::AbstractPriorEstimator; kwargs...)

Read the location a covariance forecast is about, off the estimator that formed it.

A second moment is always of returns about some centre, $\mathbb{E}[(\boldsymbol{z} - \boldsymbol{c})(\boldsymbol{z} - \boldsymbol{c})^\intercal] = \hat{\mathbf{\Sigma}}$, and the centre is the estimator's own: the mean its centring estimator fitted, the weighted sample mean it subtracted, the exponentially weighted location its state carries, or zero for an estimator that assumed it. An evaluation that judged the forecast against the raw outer product $\boldsymbol{z}\boldsymbol{z}^\intercal$ would be judging it against a proxy of $\mathbf{\Sigma} + (\boldsymbol{\mu} - \boldsymbol{c})(\boldsymbol{\mu} - \boldsymbol{c})^\intercal$, and read a bias of $\boldsymbol{c}^\intercal \hat{\mathbf{\Sigma}}^{-1} \boldsymbol{c} / N$ into a forecast that has none. So the evaluation centres the test rows here, and there is no flag: the location is the estimator's, not the caller's.

Two arities per family. The data form answers for a batch fit over X, and the data-less form answers for the state the estimator carries after partial_fit!, so a refit fold and a stepped fold centre on the same quantity.

Covariance: the centre weighted_centre resolves from ce.me and ce.w, or state.mu. Under a CoveragePolicy the batch fit is the fold, so the data form folds the window and reads the same state.mu: the diagonal of the per-pair centre, each asset's own available-case mean. The off-diagonal products of the forecast are centred on the mean of the observations the pair shares, which is not a location the vector can carry; the diagonal is the one the per-asset ratios read.
GeneralCovariance: the sample mean, weighted by ce.w when it carries observation weights, or state.mu.
ExpWeightedCovariance and RegimeAdjustedExpWeightedCovariance: state.location, or zero for every asset when centred = true, because the estimator then assumed it. The data form runs the same pass over the window that the batch fit runs, and reads the location the pass ends on.
PortfolioOptimisersCovariance and CorrelationCovariance: the answer of the estimator they hold, because a matrix transform moves no centre.
Any other AbstractCovarianceEstimator: the window's own sample mean over the finite rows of each column. An estimator that carries a state of its own and no method is refused by name, because there is no window to read.
Under Online, whatever the family: the family's data form over the buffer's rows and masks, because a buffer means the batch verb over the buffer's rows for every read-out.
An AbstractPriorEstimator: the mu the prior publishes.

The Asset Panel form follows the moment seam of the estimator's own cov: under no policy it reduces the window to its Coverage Universe and frames the answer with NaN outside it, and under a policy, or for the two mask-aware exponentially weighted families, it hands the estimator the panel's active mask.

Arguments

ce: Covariance estimator.
pe: Prior estimator.
X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
rd: The returns result to use.
pnl: Optional AssetPanel, whose active mask the Coverage Universe of the fit is derived from. nothing makes the rule finiteness alone.
dims: Dimension along which to perform the computation.
active_mask: The active mask of the window, observations × assets, or nothing.
kwargs...: Additional keyword arguments passed to the centring estimator.

Validation

The data-less form requires a state. An ArgumentError is thrown otherwise.
The data-less fallback requires the state to be a SampleBufferState. An ArgumentError is thrown otherwise.

Returns

c::VecNum: The location, assets × 1.

Related

source

PortfolioOptimisers.forecast_state_location — Function

forecast_state_location(ce, state)

Read the location out of the state a covariance estimator carries, by the state's type.

The data-less arm of forecast_location, one method per state shape: a CovarianceState carries mu, the two exponentially weighted states carry location, a SampleBufferState carries the rows and the masks they were folded with, and answers the estimator's own data form over them, because a buffer means the batch verb over the buffer's rows for every read-out; and a composite whose own cache is nothing forwards to the estimator it holds, which carries the state. nothing is refused by name, because there is nothing to read, and so is a state of a shape the verb does not know.

Related

source

PortfolioOptimisers.forecast_coverage_policy — Function

forecast_coverage_policy(ce)

Read the CoveragePolicy that decides the Asset Panel arm of forecast_location, out of an estimator that may have no such field.

coverage_policy answers for the two families that can carry one; every other covariance estimator answers nothing, so the panel form reduces to the Coverage Universe as the moment seam does for them.

Related

source

PortfolioOptimisers.finite_column_mean — Function

finite_column_mean(X::MatNum)

Mean of each column of X over its finite entries, NaN for a column with none.

Related

forecast_location

source

PortfolioOptimisers.resolve_forecast_weights — Function

resolve_forecast_weights(w::Nothing, a::VecInt, sa::MatNum, N::Integer)
resolve_forecast_weights(w::VecNum, a::VecInt, sa::MatNum, N::Integer)
resolve_forecast_weights(w::VecVecNum, a::VecInt, sa::MatNum, N::Integer)

Resolve the test portfolios of a step over its active subset.

nothing is inverse volatility recomputed from the step's forecast, $w_i \propto 1 / \sqrt{(\hat{\mathbf{\Sigma}}_t)_{ii}}$ over the active assets, which stops the most volatile asset dominating the reading. A vector is one static portfolio and a vector of vectors several; each is cut to the active subset and renormalised to sum to one there, so a delisted asset's weight is spread over the assets that remain. No guard is placed on a subset whose weights sum to zero: the portfolio is then not a portfolio, and its ratio is not a number.

Arguments

w: The test portfolios on the full universe, or nothing.
a: The active subset.
sa: The forecast on the active subset.
N: The width of the universe.

Validation

Every portfolio has N entries. A DimensionMismatch is thrown otherwise.

Returns

ws::Vector{<:VecNum}: One weights vector per portfolio, each over a.

Related

covariance_forecast_step

source

PortfolioOptimisers.is_time_dependent — Method

is_time_dependent(est::Union{<:AbstractCovarianceEstimator, <:AbstractPriorEstimator, <:Online})
needs_previous_weights(est::Union{<:AbstractCovarianceEstimator, <:AbstractPriorEstimator, <:Online})

Answer the two per-type predicates fold_loop reads, for an estimator that is not an optimiser.

No covariance estimator and no prior holds a TimeDependent schedule — the schedule type is declared after both families, so no field of theirs can name it — and neither reads a previous fold's weights. An Online wrapper at the root answers the same, because the loop resolves it before the first fold. Both answer false, so the loop neither resolves a context nor threads weights, and a batch run of the evaluation takes the parallel arm under the sequential executor it is handed.

Related

source

PortfolioOptimisers.online_entry_state — Method

online_entry_state(o::Online)

Name the state a wrapped estimator carries at the entry of the fold loop's online arm, or answer nothing.

The Online arm of online_entry_state, for the wrapper at the root of covariance_forecast_evaluation: a wrapper holds no cache of its own, so the walk descends into the estimator it wraps and prefixes est to what it finds there, est.cache for a covariance estimator handed over already folded. The generic walk would reach the same answer through estimator_fields; this arm states it without asking the wrapper for a field it does not have.

Related

source

PortfolioOptimisers.advance_previous_fold — Method

advance_previous_fold(pws, prev, step::NamedTuple)

Hand the next fold what the last one held, when the fold's result is a step record and not a prediction.

The kernel's step record carries no weights, so there is nothing to thread and prev stays where it was. This is the arm online_folds reaches under covariance_forecast_evaluation.

Related

source

PortfolioOptimisers.forecast_moments — Function

forecast_moments(ce::AbstractCovarianceEstimator, rd::ReturnsResult, train_idx::VecInt)
forecast_moments(ce::AbstractCovarianceEstimator, rd::ReturnsResult, ::Nothing)
forecast_moments(pe::AbstractPriorEstimator, rd::ReturnsResult, train_idx::VecInt)
forecast_moments(pe::AbstractPriorEstimator, rd::ReturnsResult, ::Nothing)

Read a fold's forecast and the location it is about, by whether the fold carries a training window.

The two arms of covariance_forecast_evaluation's callback, chosen by dispatch on the fold's train exactly as fit_fold_result chooses the optimiser's. A window fits the estimator over it through the Asset Panel seam of the moment verbs, cov(ce, X, pnl) and forecast_location(ce, X, pnl), or prior(pe, rd), the refit every batch fold runs. nothing says the estimator holds its window, so the fold reads the state out, cov(ce) and forecast_location(ce), or prior(pe). A prior is read once for both moments.

Arguments

ce: Covariance estimator.
pe: Prior estimator.
rd: The carrier.
train_idx: The fold's training window, or nothing for a stepped estimator.

Returns

sigma::MatNum: The forecast, assets × assets.
c::VecNum: The location the forecast is about, assets × 1.

Related

source