The covariance forecast evaluation

struct RealisedCovariance <: AbstractRealisedTarget

Judges a covariance forecast against the realised covariance of the returns that follow it.

This is the default target. It sums the outer product of every centred return of the step, so it is a per-row statistic: under a Gaussian null the step's Mahalanobis ratio has variance $2 / (N h)$, the smaller of the two members' by a factor of $h$, which is why it is the default when the better statistic costs nothing. It asks whether the forecast describes each day's dispersion.

Mathematical definition

\[\begin{align} \mathbf{S}_t &= \sum_{s=1}^{h} \boldsymbol{z}_{t+s} \boldsymbol{z}_{t+s}^\intercal\,, \\ \mathbb{E}\left[\mathbf{S}_t \mid \mathcal{F}_t\right] &= h\, \mathbf{\Sigma}_t\,. \end{align}\]

Where:

• $\mathbf{S}_t$: Realised covariance of step $t$, formed from the centred returns that follow it.
• $h$: Horizon of a step, the number of observations the forecast is judged on.
• $\boldsymbol{z}_{t+s}$: Centred return of observation $t + s$, the return less the location the forecast is about.
• $\mathbf{\Sigma}_t$: True conditional covariance of the returns that follow step $t$.
• $\mathcal{F}_t$: Information available at step $t$.

The expectation holds when the $h$ returns are serially uncorrelated, and it makes $\mathbf{S}_t / h$ a conditionally unbiased proxy of $\mathbf{\Sigma}_t$ whatever the distribution of the returns. At $h = 1$ the proxy has rank one and equals the HorizonReturn member's. A cell whose two assets share no finite row is zero, and its count is zero.

Examples

julia> RealisedCovariance()RealisedCovariance()

Related

• AbstractRealisedTarget
• HorizonReturn
• realised_target
• covariance_forecast_evaluation

struct HorizonReturn <: AbstractRealisedTarget

Judges a covariance forecast against the outer product of the return earned over the horizon.

This is the $h$-day holder's question: the return summed over the step is what a book held for the whole horizon earns, and its outer product is a rank-one proxy of $h\, \mathbf{\Sigma}_t$. Under a Gaussian null the step's Mahalanobis ratio has variance $2 / N$, whatever $h$, so it is the noisier member at every horizon above one.

Mathematical definition

\[\begin{align} \boldsymbol{R}_t &= \sum_{s=1}^{h} \boldsymbol{z}_{t+s}\,, \\ \mathbf{S}_t &= \boldsymbol{R}_t \boldsymbol{R}_t^\intercal\,, \\ \mathbb{E}\left[\mathbf{S}_t \mid \mathcal{F}_t\right] &= h\, \mathbf{\Sigma}_t\,. \end{align}\]

Where:

• $\boldsymbol{R}_t$: Return earned over the horizon of step $t$, summed over the finite observations of each asset.
• $\mathbf{S}_t$: Realised covariance of step $t$, formed from the centred returns that follow it.
• $h$: Horizon of a step, the number of observations the forecast is judged on.
• $\boldsymbol{z}_{t+s}$: Centred return of observation $t + s$, the return less the location the forecast is about.
• $\mathbf{\Sigma}_t$: True conditional covariance of the returns that follow step $t$.
• $\mathcal{F}_t$: Information available at step $t$.

The expectation holds when the $h$ returns are serially uncorrelated. At $h = 1$ the proxy equals the RealisedCovariance member's. A non-finite cell contributes nothing to the sum, and the count of a cell is the number of rows at which both of its assets are finite.

Examples

julia> HorizonReturn()HorizonReturn()

Related

• AbstractRealisedTarget
• RealisedCovariance
• realised_target
• covariance_forecast_evaluation

struct CovarianceForecastEvaluationResult{__T_dates, __T_test_idx, __T_horizon, __T_target, __T_n_valid, __T_mahalanobis_ratio, __T_diagonal_ratio, __T_qlike, __T_frobenius, __T_standardised_return, __T_portfolio_qlike, __T_w, __T_sigma, __T_location} <: AbstractResult

The per-step diagnostics of a covariance forecast over a walk-forward, and its forecasts on request.

CovarianceForecastEvaluationResult is what covariance_forecast_evaluation returns. It keeps every diagnostic per step — three that read the forecast on its own, two per test portfolio — so the summary, the comparison and the plots are verbs over it, and it keeps the forecasts themselves only when asked, because a run of M steps over N assets pairs N² × M numbers with N × M, and the run the online form exists for is a step at every observation.

Fields

• dates: Label of each step: the timestamp of its first test row when the carrier holds one, else the row's index. One entry per step.

• test_idx: Test rows of each step, in split order. One window per step.

• horizon: Horizon of each step, the number of test rows the forecast was judged on. One entry per step; every entry is the scheme's test_size under an index walk-forward, and a calendar period's own length under a date walk-forward.

• target: Realised quantity the forecasts were judged against.

• n_valid: Number of active assets at each step: a finite forecast variance and at least one finite test return. One entry per step.

• mahalanobis_ratio: Mahalanobis ratio of each step. One entry per step; the target is one.

• diagonal_ratio: Diagonal ratio of each asset at each step, steps × assets; NaN where the asset was not active. The target is one.

• qlike: QLIKE loss of each step. One entry per step; lower is better, and only a difference between two forecasts is a reading.

• frobenius: Frobenius loss of each step. One entry per step; lower is better, and only a difference between two forecasts is a reading.

• standardised_return: Standardised return of each test portfolio at each step, steps × portfolios. Mean zero and standard deviation one under a calibrated forecast.

• portfolio_qlike: QLIKE loss of each test portfolio's variance at each step, steps × portfolios; lower is better.

• w: Test portfolios as handed to the evaluation: nothing for inverse volatility per step, a vector for one portfolio, a vector of vectors for several.

• sigma: Forecast of each step, one assets × assets matrix per step, or nothing unless store_forecasts = true.

• location: Location each forecast was centred on, one assets × 1 vector per step, or nothing unless store_forecasts = true.

Constructors

CovarianceForecastEvaluationResult(    dates, test_idx, horizon, target, n_valid, mahalanobis_ratio, diagonal_ratio,    qlike, frobenius, standardised_return, portfolio_qlike, w, sigma, location) -> CovarianceForecastEvaluationResult

Arguments correspond to the struct's fields, in the order they are declared. The type is a Result, so covariance_forecast_evaluation builds it and a caller reads it; there is no keyword constructor, and the type validates nothing of its own.

Related

• covariance_forecast_evaluation
• covariance_forecast_step
• covariance_forecast_summary
• covariance_forecast_compare
• covariance_forecast_portfolio

covariance_forecast_step(sigma::MatNum, Z::MatNum, c::VecNum, w::Option{<:VecNum_VecVecNum}, target::AbstractRealisedTarget)

Score one covariance forecast against the returns realised after it.

The per-step kernel of covariance_forecast_evaluation, on bare arrays, so a caller holding forecasts of their own builds the same Result by hand. It reads the forecast on the assets that are active at the step, centres the test rows on the location the forecast is about, forms the realised matrix through the target, and answers the diagnostics of the step: three that read the forecast without a portfolio, and two per test portfolio.

Mathematical definition

\[\begin{align} \mathbf{H}_{ij} &= \left\lvert \{ s : z_{t+s,i},\, z_{t+s,j} \text{ finite} \} \right\rvert\,, \\ m_t &= \frac{\operatorname{tr}\left(\left(\mathbf{H} \odot \hat{\mathbf{\Sigma}}_t\right)^{-1} \mathbf{S}_t\right)}{N}\,, \\ d_{t,i} &= \frac{(\mathbf{S}_t)_{ii}}{\mathbf{H}_{ii}\, (\hat{\mathbf{\Sigma}}_t)_{ii}}\,, \\ L^{\mathrm{QLIKE}}_t &= h \log\left|\hat{\mathbf{\Sigma}}_t\right| + N h\, m_t\,, \\ L^{\mathrm{F}}_t &= \sum_{i,j : \mathbf{H}_{ij} > 0} \left(\frac{(\mathbf{S}_t)_{ij}}{\mathbf{H}_{ij}} - (\hat{\mathbf{\Sigma}}_t)_{ij}\right)^2\,, \\ b_t &= \frac{\boldsymbol{w}^\intercal \boldsymbol{R}_t}{\sqrt{\boldsymbol{w}^\intercal \left(\mathbf{H} \odot \hat{\mathbf{\Sigma}}_t\right) \boldsymbol{w}}}\,, \\ L^{\mathrm{P}}_t &= \log\left(\boldsymbol{w}^\intercal \left(\mathbf{H} \odot \hat{\mathbf{\Sigma}}_t\right) \boldsymbol{w}\right) + \frac{\sum_{s=1}^{h} (\boldsymbol{w}^\intercal \boldsymbol{z}_{t+s})^2}{\boldsymbol{w}^\intercal \left(\mathbf{H} \odot \hat{\mathbf{\Sigma}}_t\right) \boldsymbol{w}}\,. \end{align}\]

Where:

• $\mathbf{H}$: Pairwise count of the finite test observations of each pair of assets.
• $\hat{\mathbf{\Sigma}}_t$: Covariance forecast formed at step $t$.
• $\mathbf{S}_t$: Realised covariance of step $t$, formed from the centred returns that follow it.
• $h$: Horizon of a step, the number of observations the forecast is judged on.
• $N$: Number of assets.
• $m_t$: Mahalanobis ratio of step $t$.
• $d_{t,i}$: Diagonal ratio of asset $i$ at step $t$.
• $L^{\mathrm{QLIKE}}_t$: QLIKE loss of step $t$.
• $L^{\mathrm{F}}_t$: Frobenius loss of step $t$.
• $b_t$: Standardised return of the test portfolio at step $t$.
• $L^{\mathrm{P}}_t$: QLIKE loss of the test portfolio's variance at step $t$.
• $\boldsymbol{R}_t$: Return earned over the horizon of step $t$, summed over the finite observations of each asset.
• $\boldsymbol{z}_{t+s}$: Centred return of observation $t + s$, zero at a cell the asset did not trade.
• $\boldsymbol{w}$: Portfolio weights vector $N \times 1$.

With no gap in the test window $\mathbf{H} = h \mathbf{1}\mathbf{1}^\intercal$, and the forms reduce to $m_t = \operatorname{tr}(\hat{\mathbf{\Sigma}}_t^{-1} \mathbf{S}_t) / (N h)$, $d_{t,i} = (\mathbf{S}_t)_{ii} / (h (\hat{\mathbf{\Sigma}}_t)_{ii})$, $L^{\mathrm{QLIKE}}_t = h \log|\hat{\mathbf{\Sigma}}_t| + \operatorname{tr}(\hat{\mathbf{\Sigma}}_t^{-1} \mathbf{S}_t)$, $L^{\mathrm{F}}_t = \lVert \mathbf{S}_t / h - \hat{\mathbf{\Sigma}}_t \rVert_F^2$ and $b_t = \boldsymbol{w}^\intercal \boldsymbol{R}_t / \sqrt{h\, \boldsymbol{w}^\intercal \hat{\mathbf{\Sigma}}_t \boldsymbol{w}}$. When the forecast is the true covariance, $\mathbb{E}[m_t] = \mathbb{E}[d_{t,i}] = 1$ and $b_t$ has mean zero and variance one, whatever the distribution of the returns; under Gaussian returns $m_t$ times its degrees of freedom is chi-squared on them (target_dof). A ratio above one is an under-prediction of the dispersion in the metric of the forecast, and one below it an over-prediction. $L^{\mathrm{QLIKE}}_t$ is $-2$ times the Gaussian log-likelihood of the step's returns under the forecast, up to a constant, and $L^{\mathrm{F}}_t$ the squared distance of the forecast from the per-cell proxy; the expected value of each is minimised by the true covariance under any conditionally unbiased proxy, so either ranks two forecasts on the proxy as it would on the truth, and neither's level is a calibration reading on its own. $L^{\mathrm{P}}_t$ is the univariate QLIKE of the portfolio's variance, and it reads the per-row portfolio returns whichever target formed $\mathbf{S}_t$.

Algorithm

1. Take the active subset a: the assets whose forecast variance and location are finite and which have at least one finite test return. Refuse an empty subset by name.
2. Centre the test rows of a on c[a], leaving a non-finite cell non-finite, giving Zc.
3. Form (S, H) through realised_target, and the effective forecast H .* sigma[a, a].
4. Factor sigma[a, a] once by Cholesky, for its log-determinant and, when H is uniform, for the whitening; a H that varies by cell factors the effective forecast a second time.
5. Read the Mahalanobis ratio, the diagonal ratio of each active asset (NaN outside a), the QLIKE loss and the Frobenius loss.
6. Resolve the test portfolios over a through resolve_forecast_weights, and read each one's standardised return and portfolio QLIKE from the column sums and the per-row portfolio returns of Zc, a non-finite cell counting zero.

Arguments

• sigma: The forecast, assets × assets; a NaN diagonal marks an inactive asset.
• Z: The test rows, observations × assets; a non-finite cell is an observation the asset did not trade.
• c: The location the forecast is about, assets × 1.
• w: The test portfolios on the full universe: nothing for inverse volatility over the active subset, a vector for one portfolio, a vector of vectors for several. Each is renormalised over the active subset.
• target: The realised quantity.

Validation

• sigma is square and Z, c and every portfolio of w have its width. A DimensionMismatch is thrown otherwise.
• At least one asset is active: a finite forecast variance, a finite location, and a finite test return. An ArgumentError is thrown otherwise.

Returns

• step::NamedTuple: n_valid, the active count; mahalanobis_ratio; diagonal_ratio, assets × 1; qlike; frobenius; standardised_return, one per portfolio; and portfolio_qlike, one per portfolio.

Related

• covariance_forecast_evaluation
• CovarianceForecastEvaluationResult
• realised_target
• resolve_forecast_weights
• forecast_location

partial_fit!(ce::AbstractCovarianceEstimator, rd::ReturnsResult)
partial_fit!(pe::AbstractPriorEstimator, rd::ReturnsResult)

Fold the observations of a carrier into a covariance estimator, or into a prior.

The carrier arity the fold loop's online arm calls, online_folds, for the two kinds of estimator covariance_forecast_evaluation threads through it. A prior takes fold_prior, the one forward the optimiser's step makes. A covariance estimator takes rd.X in its own arity, with the active mask of a time-varying Asset Panel as the keyword its step reads, through step_active_mask, which also refuses what the step cannot carry.

Arguments

• ce: Covariance estimator.
• pe: Prior estimator.
• rd: The carrier of the observations to fold, observations × assets.

Validation

• rd.X is not nothing. An IsNothingError is thrown otherwise.
• Everything step_active_mask refuses.

Returns

• est: The estimator, with the observations folded into its state.

Related

• fold_prior
• step_active_mask
• online_folds
• covariance_forecast_evaluation

covariance_forecast_evaluation(est::Union{<:AbstractCovarianceEstimator, <:AbstractPriorEstimator, <:Online},
                               rd::ReturnsResult, cv::WFCVER; w::Option{<:VecNum_VecVecNum} = nothing,
                               target::AbstractRealisedTarget = RealisedCovariance(),
                               store_forecasts::Bool = false) -> CovarianceForecastEvaluationResult

Evaluate a covariance forecast out of sample over a walk-forward, in batch or online.

One verb through the one fold loop. Per fold the callback reads the forecast and its location off the fold's estimator — a refit over the training window when the fold carries one, the threaded state when the scheme declares a Fold Fit of OnlineStep — and scores it on the test rows through covariance_forecast_step. The batch expanding, batch rolling, online expanding and online rolling forms are therefore the four compositions the walk-forward and the Online wrapper already express: expand_train = true, expand_train = false, ff = OnlineStep(), and ff = OnlineStep() with Online(est; max_history = w). The verb reads none of them, so the online run reaches the batch expanding run's rows fold for fold, to the tolerance of the estimator's own fold, and the date form, the purge and a listing or delisting come for free.

The estimator is threaded as the configuration alone. Under OnlineStep() the loop warms one estimator up on the first training window and folds each fold's new rows into it, so an estimator entering with a state, or one that cannot fold, is refused at the door by name; wrap such an estimator in Online to fold it from a buffer, and the read-out is then a batch fit over the buffer's rows.

Arguments

• est: The covariance estimator, or the prior estimator whose sigma is the forecast; either may be wrapped in Online under ff = OnlineStep(), to fold from a buffer and, with max_history, over a rolling window.
• rd: The returns result to use.
• cv: The walk-forward, index or date form, or its split.
• w: The test portfolios on the full universe: nothing for inverse volatility recomputed from each step's forecast, a vector for one static portfolio, a vector of vectors for several.
• target: The realised quantity the forecasts are judged against.
• store_forecasts: Whether to keep every step's forecast and location in the Result, for covariance_forecast_portfolio.

Validation

• rd.X is not nothing. An IsNothingError is thrown otherwise.
• The scheme's folds are not shuffled, through assert_unshuffled_folds.
• An Online at the root is handed a scheme that declares a Fold Fit. An ArgumentError is thrown otherwise: a batch fold would never seed the buffer.
• Everything covariance_forecast_step refuses at a step, and everything the fold loop's online arm refuses at entry under OnlineStep().

Returns

• cfer::CovarianceForecastEvaluationResult: The per-step diagnostics, and the forecasts when asked.

Related

• CovarianceForecastEvaluationResult
• covariance_forecast_step
• covariance_forecast_summary
• covariance_forecast_compare
• covariance_forecast_portfolio
• forecast_location
• IndexWalkForward
• DateWalkForward
• OnlineStep
• Online

abstract type AbstractRealisedTarget <: AbstractAlgorithm

Abstract supertype of the realised quantities a covariance forecast is judged against.

A covariance forecast $\hat{\mathbf{\Sigma}}_t$ is formed at a step and judged on the $h$ returns that follow it. The member of this family says which statistic of those returns stands in for the covariance they were drawn from: their realised covariance, or the outer product of their sum. Both have the same expectation, $h\, \mathbf{\Sigma}_t$ when the returns are serially uncorrelated, and both coincide at $h = 1$, so the member decides the variance of the per-step diagnostic and the question it answers, never its target.

All concrete subtypes should subtype AbstractRealisedTarget.

Interfaces

In order to implement a new concrete type that works seamlessly with the library, subtype AbstractRealisedTarget and implement the following methods:

Required method name

• realised_target(target::AbstractRealisedTarget, Zc::MatNum) -> Tuple{MatNum, AbstractMatrix{<:Integer}}: Return the realised matrix of the centred test rows, and the count of rows over which each of its cells was formed.
• target_dof(target::AbstractRealisedTarget, n::Integer, h::Integer) -> Integer: Return the degrees of freedom of the step's Mahalanobis statistic under a Gaussian null, for n active assets and a horizon of h.
• target_step_dof(target::AbstractRealisedTarget, h::Integer) -> Integer: Return the degrees of freedom of one asset's, or one portfolio's, step ratio under a Gaussian null.

Arguments

• target: The concrete subtype instance.
• Zc: The centred test rows, observations × assets; a cell an asset did not trade is non-finite.
• n: The number of active assets at the step.
• h: The horizon of the step.

Returns

• S::MatNum: The realised matrix, assets × assets.
• H::AbstractMatrix{<:Integer}: The count of rows at which both assets of each cell are finite.
• dof::Integer: The degrees of freedom.

Examples

julia> RealisedCovariance()RealisedCovariance()

Related

• RealisedCovariance
• HorizonReturn
• realised_target
• covariance_forecast_step
• covariance_forecast_evaluation

realised_target(target::RealisedCovariance, Zc::MatNum)
realised_target(target::HorizonReturn, Zc::MatNum)

Form the realised matrix of a step and the count of rows behind each of its cells.

The two arms of the AbstractRealisedTarget interface the library ships. A non-finite cell of Zc is an observation the asset did not trade, so it contributes nothing to the sum and is not counted; the count H is the pairwise number of finite rows, and it is what scales the forecast cell by cell, H ⊙ Σ̂, so that a gap in the test window moves the target and not the reading.

Arguments

• target: The realised quantity.
• Zc: The centred test rows, observations × assets.

Returns

• S::MatNum: The realised matrix, assets × assets.
• H::AbstractMatrix{<:Integer}: The pairwise count of finite rows.

Related

• AbstractRealisedTarget
• RealisedCovariance
• HorizonReturn
• covariance_forecast_step

target_dof(target::RealisedCovariance, n::Integer, h::Integer)
target_dof(target::HorizonReturn, n::Integer, h::Integer)

Degrees of freedom of a step's Mahalanobis statistic under a Gaussian null, by realised quantity.

The statistic is the trace of the whitened realised matrix: n * h under RealisedCovariance, because every one of the h returns contributes n whitened coordinates, and n under HorizonReturn, because the horizon return is one draw whatever h. The summary reads it to weight the steps of a run and to width the band on the mean ratio, and the exceedance rate reads it for its chi-squared threshold.

Arguments

• target: The realised quantity.
• n: The number of active assets at the step.
• h: The horizon of the step.

Returns

• dof::Integer: The degrees of freedom.

Related

• AbstractRealisedTarget
• target_step_dof
• covariance_forecast_summary

target_step_dof(target::RealisedCovariance, h::Integer)
target_step_dof(target::HorizonReturn, h::Integer)

Degrees of freedom of one asset's, or one portfolio's, step ratio under a Gaussian null, by realised quantity.

h under RealisedCovariance, one squared whitened coordinate per return, and 1 under HorizonReturn, one draw whatever h. The summary reads it to weight the steps of the diagonal ratio and to width its band.

Arguments

• target: The realised quantity.
• h: The horizon of the step.

Returns

• dof::Integer: The degrees of freedom.

Related

• AbstractRealisedTarget
• target_dof
• covariance_forecast_summary