Regime adjusted exp weighted covariance (a)

Types

PortfolioOptimisers.RegimeAdjustedTarget — Type
abstract type RegimeAdjustedTarget <: AbstractAlgorithm

Names the regime statistic that RegimeAdjustedExpWeightedCovariance reads from one observation.

The statistic compares the realised risk of the observation with the risk that the covariance before it predicts, so the target names which part of the covariance the regime multiplier calibrates. The cross-sectional bias statistic of [79] reads the volatilities alone, and it is DiagonalTarget. The Short-Term Volatility Updating of [80] reads the whole standardised vector, and it is MahalanobisTarget.

Interfaces

To implement a new regime-adjustment target, subtype RegimeAdjustedTarget and implement regime_statistic. Implement min_active_assets where the statistic needs more than one scored asset. Implement regime_denom to use the target with FirstMomentRegimeAdjusted, and regime_kappa to use it with LogRegimeAdjusted. A new target takes no bias correction, under every debias rule.

regime_statistic

  • regime_statistic(target::MyTarget, X::VecNum, C::MatNum, idx::AbstractVector{<:Integer}, min_val::Number) -> Option{<:VecNum}: One statistic for each direction that the target reads, from the deviations X of the scored assets, their bias-corrected covariance block C before the observation, and their index idx in the universe. Under correct calibration the statistic has the mean length(X). nothing skips the observation.

min_active_assets

  • min_active_assets(target::MyTarget) -> Int: The smallest count of scored assets at which the target gives a statistic. It is one by default.

regime_denom

  • regime_denom(method::FirstMomentRegimeAdjusted, target::MyTarget, n::Integer) -> Number: The expectation of the root of the statistic of n scored assets under correct calibration.

regime_kappa

  • regime_kappa(method::LogRegimeAdjusted, target::MyTarget, n::Integer) -> Number: The expectation of the log of the statistic of n scored assets under correct calibration.

Examples

julia> struct MyTarget <: PortfolioOptimisers.RegimeAdjustedTarget endjulia> function PortfolioOptimisers.regime_statistic(::MyTarget, X::AbstractVector,                                                     C::AbstractMatrix,                                                     ::AbstractVector{<:Integer}, min_val::Number)           return [sum(abs2, X ./ sqrt.(max.(LinearAlgebra.diag(C), min_val)))]       end;julia> PortfolioOptimisers.min_active_assets(MyTarget())1julia> X = [0.01 -0.02; -0.015 0.03; 0.02 -0.01; -0.005 0.012; 0.008 -0.02; -0.02 0.03];julia> kw = (; decay = 0.9, min_obs = 2, regime_min_obs = 1, debias = nothing,             regime_method = RootMeanSquaredAdjusted());julia> cov(RegimeAdjustedExpWeightedCovariance(; regime_target = MyTarget(), kw...), X) ≈       cov(RegimeAdjustedExpWeightedCovariance(; regime_target = DiagonalTarget(), kw...), X)true

Related

References

  • [79] J. Menchero, D. J. Orr and J. Wang. The Barra US Equity Model (USE4): Methodology Notes. Model Insight, MSCI (2011). Section 4.3, Equation 4.3.
  • [80] G. A. Paleologo. The Elements of Quantitative Investing. Wiley (2025). Section 6.3.3, Model 6.8.
source
PortfolioOptimisers.MahalanobisTarget — Type
struct MahalanobisTarget{__T_recursion} <: RegimeAdjustedTarget

Scores an observation by its squared Mahalanobis distance against the covariance that stands before it.

The distance reads every eigen-direction of the block of the scored assets, so the target calibrates the whole covariance structure. A direction with a small estimated variance takes as much weight as the market direction. The target needs two scored assets.

The squared distance is the $\lVert \boldsymbol{u}_{t} \rVert^{2}$ of the Short-Term Volatility Updating of [80], with $\boldsymbol{u}_{t} = \mathbf{C}_{t}^{-1/2} \mathbf{V}_{t}^{-1} \boldsymbol{f}_{t}$. There, $\mathbf{C}_{t}$ is the correlation, $\mathbf{V}_{t}$ the diagonal of the volatilities and $\boldsymbol{f}_{t}$ the returns of the model. Under Model 6.8 it is $e^{x_{t}}$ times a $\chi^{2}_{n}$ variate. The book defines $\kappa$ as the expectation of the log of that variate and prints no closed form. At the default law, regime_kappa is that expectation, $\psi(n/2) + \ln 2$. The book reads the covariance of the model, and this target reads the estimate before the observation.

The statistic reads the inverse of an estimated block, and the inverse of an estimate is too large on average (Jensen's inequality). At 12 assets and a half-life of 10, the squared distance of a correctly calibrated return is 1.6 times its dimension. Before the block has more than n + 1 observations the mean is not finite, and before it has more than n + 3 the variance is not finite, so one observation can hold the smoothed regime state. Where the debias field of RegimeAdjustedExpWeightedCovariance is ExactDebias or LawDebias, the target skips those observations. It divides the other statistics by the bias that its regime method reads, so the statistic has the calibration that the method assumes.

Mathematical definition

\[\begin{align} d^{2} &= \frac{u^{\top} \hat{C}^{-1} u}{\beta}\,, & u^{\top} \hat{C}^{-1} u &= \chi^{2}_{n}\, R\,, & R &= \frac{1}{S}\,, & w_{j} &= \frac{(1 - \lambda)\, \lambda^{j}}{1 - \lambda^{K}}\,. \end{align}\]

Where:

  • $u$: Returns of the $n$ contributing assets at the observation.
  • $\hat{C}$: Bias-corrected covariance block of those assets, from the observations before it.
  • $K$: Smallest count of observations among the contributing assets. The target skips the update while $K \le n + 3$. With a HAC adjustment, it skips the update while the effective count of the banded weight matrix is $n + 1$ or less.
  • $\lambda$: cor_decay where the separate correlation path runs, else decay.
  • $S$: Schur complement of one direction in $W = \sum_{j} w_{j} z_{j} z_{j}^{\top}$, $z_{j} \sim N(0, I_{n})$. $R$ is independent of the $\chi^{2}_{n}$ factor.
  • $\beta$: Bias factor of the regime method without a HAC adjustment, from mahalanobis_regime_bias!, times the variance factor of variance_noise_bias! on the separate correlation path: $\mathbb{E}[R]$ for RootMeanSquaredAdjusted, $\mathbb{E}[\sqrt{R}]^{2}$ for FirstMomentRegimeAdjusted, and $\exp(\mathbb{E}[\ln R])$ for LogRegimeAdjusted. With equal weights $R = K / \chi^{2}_{K - n + 1}$ exactly. On exponential weights the factor comes from a recursion over the $n - 1$ other directions. The recursion is exact at one asset and at equal weights, and it is within 0.15 % of a Monte Carlo at half-lives of 10 and more.

With debias = nothing, $\beta = 1$, and the target scores every observation above min_obs. A block that is not positive definite then takes the ridge of safe_regime_cholesky. Its statistic can be about $10^{12}$, and that one value holds the regime state for many half-lives.

The factor assumes one shared history of returns, and the weights of the recursion. On iid Normal returns at 12 assets and a half-life of 10, the squared multiplier of each of the three methods is then 1.000 within 0.002.

A HAC estimate reads the same recursion on the spectrum of its banded weight matrix, and it skips a block with too few effective observations. At two lags, 12 assets and a half-life of 10, the three methods read 1.0035, 1.0031 and 1.0023 within 0.004 over 8 seeds. With the fixed point of the mean they read 0.984, 0.952 and 0.920.

On the separate correlation path the factor reads cor_decay. variance_noise_bias! adds the noise of the variance at decay, to the mean and to the spread of each method. At a correlation half-life of 20, the three methods read 0.998, 0.997 and 0.996 over 8 seeds, and 1.032, 1.030 and 1.027 at cor_decay alone. At two HAC lags, each correlation row divides by the volatility before its update, and the tables at cor_decay read the kernel of hac_row_kernel. The three methods then read 1.001, 1.001 and 1.000 over 8 seeds. RootMeanSquaredAdjusted reads 1.090 at cor_decay alone, and 1.026 with the rule after the update.

Fields

  • recursion: The rule that computes the bias factor of the Mahalanobis statistic from the level recursion over its directions. MeanLevelRecursion, the default, takes the mean log-determinant of each level inside the exponential, which reads 0.2 % to 1.3 % low near the gate. GaussianLevelRecursion carries the covariance of the log-determinant of each level too, which reads within the noise of a Monte Carlo of the pre-centred law. Its nodes cost 2 s to 5.4 s for each count of assets that a fit meets, where the default costs 0.3 s to 0.9 s, and it may be removed if its gain on the forecasts of the estimator does not pay that cost.

Constructors

MahalanobisTarget(;    recursion::AbstractLevelRecursion = MeanLevelRecursion()) -> MahalanobisTarget

Keywords correspond to the struct's fields.

Examples

julia> MahalanobisTarget()MahalanobisTarget  recursion ┴ MeanLevelRecursion()

Related

References

  • [80] G. A. Paleologo. The Elements of Quantitative Investing. Wiley (2025). Section 6.3.3, pages 122 and 123, Model 6.8 and the estimate of the state.
source
PortfolioOptimisers.DiagonalTarget — Type
struct DiagonalTarget <: RegimeAdjustedTarget

Targets the diagonal of a covariance matrix, in a regime adjustment and in a geodesic shrinkage.

In a regime adjustment, the statistic divides each return by its own volatility and sums the squares, so it calibrates the volatilities and reads no correlation. This is the cross-sectional bias statistic of Equation 4.3 of [79], times the count of the assets. In a GeodesicShrinkageCovariance, the target matrix is the diagonal of the matrix that the estimator shrinks. It keeps the variances and removes every correlation.

Each term of the regime statistic reads one estimated variance, and the inverse of an estimate is too large on average. Where the estimator has debias = ExactDebias(), regime_target_statistic divides each term by the mean of that inverse at the count of its asset, so the statistic has the mean $n$ at every correlation. The root and the log of the statistic read its law, which the correlation of the assets and the noise of each estimate set, so for the first-moment and the log methods it then divides the sum by diagonal_law_factor.

Mathematical definition

\[\begin{align} d_{t}^{2} &= \frac{1}{\gamma} \sum_{i \in \mathcal{A}_{t}} \frac{u_{i,t}^{2}}{f_{i}\, \hat{C}_{ii}}\,. \end{align}\]

Where:

  • $d_{t}^{2}$: Regime statistic of observation $t$.
  • $\mathcal{A}_{t}$, $n_{t}$: Scored assets of observation $t$, the assets whose returns the regime statistic reads, and their count.
  • $u_{i,t}$: Deviation of the return of asset $i$ at observation $t$.
  • $\hat{C}_{ii}$: Bias-corrected variance of asset $i$ before observation $t$.
  • $f_{i}$: Mean of the inverse of that estimated variance, from regime_bias_table, or one under debias = nothing.
  • $\gamma$: Factor of diagonal_law_factor for the first-moment and the log methods, and one for every other method and under debias = nothing.

With $f_{i} = \gamma = 1$, the root-mean-squared method reads $d_{t}^{2} / n_{t}$, the square of the bias statistic of [79].

Related

References

  • [79] J. Menchero, D. J. Orr and J. Wang. The Barra US Equity Model (USE4): Methodology Notes. Model Insight, MSCI (2011). Section 4.3, Equation 4.3.
source
PortfolioOptimisers.PortfolioTarget — Type
struct PortfolioTarget{__T_w, __T_indefinite} <: RegimeAdjustedTarget

Scores an observation by its squared return along each portfolio direction over the variance of that direction.

The target calibrates the covariance along the directions that a portfolio holds, and reads no other direction. Each direction gives one statistic, and the regime method averages the transformed statistics.

The Short-Term Volatility Updating of [80] reads the whole standardised vector. Under its Model 6.8, the return along a fixed direction $\boldsymbol{p}$ is $e^{x_{t}/2} \sqrt{\boldsymbol{p}^{\intercal} \boldsymbol{\Omega} \boldsymbol{p}}\, z_{t}$, with the covariance $\boldsymbol{\Omega}$ of the model and a standard normal $z_{t}$. So each direction reads the same latent $x_{t}$, as $e^{x_{t}}$ times a $\chi^{2}_{1}$ variate. This target is that case of one dimension, times $n_{t}$ so that its mean is $n_{t}$. The inverse-volatility direction of w = nothing is a construction of this library.

Each direction reads one estimated variance, and the inverse of an estimate is too large on average. Where the estimator has debias = ExactDebias(), regime_target_statistic divides each direction by the bias that the regime method reads. The factor is exact for fixed weights. The inverse-volatility direction of w = nothing is built from the same estimate, so a smaller bias remains, which that function states.

A direction reads the block as a covariance. A HAC block in the warm-up, and a block of assets with different histories, need not be positive definite. A direction with a small positive variance on such a block scores a large statistic. At 8 assets, a half-life of 10 and two lags, one row of a block with the smallest eigenvalue $-0.31$ had $r^{2} = 0.51$ against a direction variance of 0.0057, and it scored 51. The indefinite field names what the statistic does with such a block.

Mathematical definition

\[\begin{align} d_{k,t}^{2} &= \frac{n_{t}\, (\boldsymbol{p}_{k}^{\intercal} \boldsymbol{u}_{t})^{2}} {f_{k}\, \boldsymbol{p}_{k}^{\intercal} \hat{\mathbf{C}} \boldsymbol{p}_{k}}\,. \end{align}\]

Where:

  • $d_{k,t}^{2}$: Regime statistic of direction $k$ at observation $t$.
  • $\mathcal{A}_{t}$, $n_{t}$: Scored assets of observation $t$, the assets whose returns the regime statistic reads, and their count.
  • $\boldsymbol{u}_{t}$: Deviations of the returns of the assets of $\mathcal{A}_{t}$.
  • $\boldsymbol{p}_{k}$: Direction $k$. Each row of w with a positive sum over $\mathcal{A}_{t}$ gives one direction, its entries on $\mathcal{A}_{t}$ divided by that sum. Where w is nothing, the one direction is $p_{i} \propto 1 / \sqrt{\hat{C}_{ii}}$, with a unit sum.
  • $\hat{\mathbf{C}}$: Bias-corrected covariance block of $\mathcal{A}_{t}$ before observation $t$.
  • $f_{k}$: Bias factor of direction $k$, which regime_target_statistic states, or one under debias = nothing.

The target does not score an observation whose direction has a variance at or below min_val. It does not score an observation whose block the rule of indefinite refuses either, under every debias. The default rule refuses a young block with a negative eigenvalue.

Fields

  • w: Optional portfolio weights for the portfolio target, as one vector over the assets or as a matrix whose rows are portfolios. If nothing, the target uses inverse-volatility weights, and it makes them again at each observation from the bias-corrected covariance block before it.
  • indefinite: The rule for a covariance block that is not positive definite. A HAC block in the warm-up, and a block of assets with different histories, need not be one, and a direction with a small positive variance on it scores a large statistic. RefuseIndefiniteWarmUp, the default, refuses a block with a negative eigenvalue while the block is young, and scores it once the gate that the Mahalanobis target reads under ExactDebias() opens for the same assets, under every debias. ClipIndefiniteBlock clips the eigenvalues of its correlation at zero and keeps the variances. ScoreIndefiniteBlock scores the block as it is.

Constructors

PortfolioTarget(;    w::Option{<:Union{VecNum, MatNum}} = nothing,    indefinite::AbstractIndefiniteBlockRule = RefuseIndefiniteWarmUp()) -> PortfolioTarget

Keywords correspond to the struct's fields. The bound is the pair of shapes the fit can honour, because Statistics.cov(ce::RegimeAdjustedExpWeightedCovariance, X::MatNum; dims::Int = 1, estimation_mask::Option{<:AbstractMatrix{<:Bool}} = nothing, active_mask::Option{<:AbstractMatrix{<:Bool}} = nothing, kwargs...) reads a bare matrix and carries no asset names. The count of assets, the sign of each weight and the sum of each row are checked at the fit, where the universe is known.

Validation

  • If w is not nothing, !isempty(w).

Examples

julia> PortfolioTarget()PortfolioTarget           w ┼ nothing  indefinite ┴ RefuseIndefiniteWarmUp()

Related

References

  • [80] G. A. Paleologo. The Elements of Quantitative Investing. Wiley (2025). Section 6.3.3, page 123, Model 6.8.
source
PortfolioOptimisers.RegimeAdjustedExpWeightedCovariance — Type
struct RegimeAdjustedExpWeightedCovariance{__T_decay, __T_cor_decay, __T_min_obs, __T_hac_lags, __T_hac_floor, __T_hac_vol_before, __T_cor_vol, __T_regime_method, __T_regime_decay, __T_regime_min_obs, __T_regime_target, __T_regime_lohi_mult, __T_min_val, __T_centring, __T_debias, __T_cache} <: AbstractCovarianceEstimator

Online exponentially weighted covariance estimator with regime-state adjustment.

At each observation it updates a running exponentially weighted covariance. It also makes a one-step-ahead statistic, which compares the realised risk of that observation with the risk that the state predicted before it. regime_decay smooths the statistic into a scalar regime state, and the estimator multiplies the covariance by the square of the multiplier of that state.

The estimator is the covariance twin of RegimeAdjustedExpWeightedVariance. It shares the RegimeAdjustedMethod family of that type, and it adds a RegimeAdjustedTarget. The target states what the statistic measures: one portfolio direction, the volatilities alone, or the whole covariance structure.

The regime adjustment is the Volatility Regime Adjustment of factor risk of [79]. It multiplies the whole matrix by the square of one multiplier, so it does not change a correlation. The log and the first-moment methods, and the Mahalanobis and the portfolio targets, are the Short-Term Volatility Updating of [80]. The separate correlation path is the exponential smoother of [81]. The HAC adjustment is the estimator of [18], applied to each term of the recursion.

The estimator reads the masks of the data. A prior fitted with it keeps a young asset investable, and scenario_fill puts a zero return in the rows where the asset had none. Every consumer of a Prior Result reads its returns matrix, so a measure of the scenarios then reads a zero return over those rows, and it gives the asset less risk than it has. The covariance stays the estimate that this recursion made from the rows that it saw. The fill is silent at or below the fill_limit field of the prior, a share of the observations of the asset. Above it the fill warns, and under strict the prior refuses every fill. fill_limit is nothing by default, and this family has no CoveragePolicy to give a limit, so the prior names every fill.

A regime_method of nothing turns the adjustment off. Then no regime state advances, the multiplier stays at one, and the estimator is the plain exponentially weighted recursion.

The statistic divides a realised square by an estimated variance, and the inverse of an estimate is too large on average (Jensen's inequality). With debias = ExactDebias(), the default, the statistic skips an estimate too young for a finite variance. It divides the other statistics by the known size of the bias, as each target states. debias = nothing scores the raw statistic.

With centring = EstimatedCentring(), the default, each deviation is taken from the normalised location of the returns of its asset before it. Each product is divided by its exact factor, centring_pair_factor, plus centring_lag_factor under HAC. The covariance is then unbiased from the second valid return of each asset, whatever the mean. The regime statistic reads each deviation divided by the root of its own factor. This is exact on the diagonal, and where the assets share one history. centring = PreCentred() takes the returns as deviations from zero.

Mathematical definition

Write $\lambda$ for decay and $\lambda_c$ for cor_decay. Where cor_decay is nothing, or equal to decay (see has_separate_cor_decay), one recursion carries the whole matrix:

\[\begin{align} S_{ij,t} &= \lambda S_{ij,t-1} + (1-\lambda) u_{i,t} u_{j,t}\,, \\ W_{ij,t} &= \lambda W_{ij,t-1} + (1-\lambda)\,, \end{align}\]

on the pairs of which both assets are valid at $t$. Every other entry holds.

Where:

  • $S_{ij,t}$: Raw exponentially weighted covariance state at time $t$, seeded at zero.
  • $W_{ij,t}$: Weight that the pair holds in $S$, seeded at zero. It is $1 - \lambda^{n_{ij}}$, with $n_{ij}$ the count of the common valid observations of the pair.
  • $u_{i,t} u_{j,t}$: The product of the deviations of observation $t$, HAC-adjusted where hac_lags is not nothing, and divided by its exact factor. Under EstimatedCentring a pair steps only where both assets have a valid return before $t$.

Where cor_decay is not nothing and differs from decay, the variance and the correlation run at their own decays, and the report combines them again:

\[\begin{align} v_{i,t} &= \lambda v_{i,t-1} + (1-\lambda) u_{i,t}^{2}\,, \\ Q_{ij,t} &= \lambda_c Q_{ij,t-1} + (1-\lambda_c) \frac{u_{i,t} u_{j,t}}{\sqrt{v_{i,t} v_{j,t}}}\,, \\ W_{ij,t} &= \lambda_c W_{ij,t-1} + (1-\lambda_c)\,, \\ \rho_{ij,t} &= \frac{Q_{ij,t} / W_{ij,t}}{\sqrt{(Q_{ii,t} / W_{ii,t})(Q_{jj,t} / W_{jj,t})}}\,. \end{align}\]

Where:

  • $v_{i,t}$: Raw exponentially weighted variance of asset $i$.
  • $Q_{ij,t}$: Raw exponentially weighted correlation state. Like $S$, it steps only on the pairs of which both assets are valid, so a holiday of asset $i$ holds every entry of $i$. Where hac_lags is not nothing and hac_vol_before is VolatilityBeforeUpdate(), the product divides by $\sqrt{v_{i,t-1} v_{j,t-1}}$, the variances before the step, because the diagonal of a HAC product can be negative and a variance that holds it amplifies it. Under cor_vol = BiasCorrectedVolatility(), the default, each variance is first divided by its weight $1 - \lambda^{K_i}$, so the first products weigh no more than the later ones.
  • $W_{ij,t}$: Weight that the pair holds in $Q$, with the step of $Q$ on a unit product: $1 - \lambda_c^{n_{ij}}$.
  • $\rho_{ij,t}$: Correlation, the weighted mean of the standardised product over the common observations of the pair, over the root of the two weighted means of the squares.

Each observation scores the assets of $\mathcal{A}_{t}$ against the covariance that stands before it. The target gives one statistic $d_{k,t}^{2}$ for each of its $D_{t}$ directions, the method transforms each one, and the regime state smooths their mean:

\[\begin{align} g_{t} &= \frac{1}{D_{t}} \sum_{k=1}^{D_{t}} \phi(d_{k,t}^{2})\,, \\ s_{t} &= \lambda_{r} s_{t-1} + (1 - \lambda_{r})\, g_{t}\,, \quad s_{t_{0}} = g_{t_{0}}\,. \end{align}\]

Where:

  • $\mathcal{A}_{t}$, $n_{t}$: Scored assets of observation $t$, the assets whose returns the regime statistic reads, and their count. An asset is scored where its return is valid, where it is in the estimation mask, and where its count of observations is at least min_obs. Under ExactDebias and LawDebias an estimate too young for a finite variance of its statistic is not scored, as each target states.
  • $d_{k,t}^{2}$, $D_{t}$: The statistic of direction $k$, and the count of the directions, which the RegimeAdjustedTarget states.
  • $\phi$: Transform of the RegimeAdjustedMethod: $d^{2} / n_{t}$ for RootMeanSquaredAdjusted, $\sqrt{d^{2}} / c_{n}$ for FirstMomentRegimeAdjusted, and $\ln \max(d^{2}, \varepsilon) - \kappa_{n}$ for LogRegimeAdjusted. $c_{n}$ and $\kappa_{n}$ are the expectations of the root and of the log of the statistic of the target under correct calibration, from regime_denom and regime_kappa, and $\varepsilon$ is min_val.
  • $g_{t}$: Regime statistic of observation $t$, which the regime state smooths. An observation that the target does not score leaves the state as it is.
  • $s_{t}$: Smoothed regime state after observation $t$. $t_{0}$ is the first observation that the target scores.
  • $\lambda_{r}$: Decay of the regime state, the regime_decay field.

A zero seed damps the state, so bias_corrected_covariance divides each pair by its weight before it reports:

\[\begin{align} \hat{\Sigma}_{ij} &= \mathrm{mult}(s_T)^{2}\, \frac{S_{ij,T}}{W_{ij,T}}\,. \end{align}\]

Where:

  • $\hat{\mathbf{\Sigma}}$: Estimated covariance matrix.
  • $\mathrm{mult}(s_T)^{2}\, S_{ij,T} / W_{ij,T}$: The entry where cor_decay is nothing. On the separate path it is $\mathrm{mult}(s_T)^{2}\, \rho_{ij,T} \sqrt{\hat{v}_{i} \hat{v}_{j}}$, with $\hat{v}_{i} = v_{i,T} / (1 - \lambda^{K_i})$ and $K_i$ the count of the terms of asset $i$.
  • $\mathrm{mult}(s_T)$: Regime multiplier of the smoothed regime state $s_T$, clamped to regime_lohi_mult where that field is not nothing. It is one while fewer than regime_min_obs observations have been scored, and where regime_method is nothing.
  • $\mathrm{mult}(s)$: Volatility multiplier of a regime state $s$. The estimator scales its variance or its covariance by the square of the multiplier.

Each entry reads its own common observations, so where the assets do not share one history, a holiday or a late listing, the matrix need not be positive semidefinite. Where the smallest eigenvalue of its correlation is below $-n\,\varepsilon$ times its largest, the report clips the negative eigenvalues to zero, restores the unit diagonal and keeps the variances. Where every pair shares one history, the division is a congruence of a sum of outer products, so the matrix is positive semidefinite and the repair changes nothing. On different histories a congruence shrinks each correlation of the pair towards zero, so the report divides each pair by its own weight instead.

Fields

  • decay: Exponential decay factor for the exponentially weighted estimator.
  • cor_decay: Exponential decay factor for the correlation smoother.
  • min_obs: Minimum number of observations required before the estimator produces a valid result.
  • hac_lags: Optional number of lags for Heteroskedasticity and Autocorrelation Consistent (HAC) kernel correction of squared returns. If nothing, no HAC correction is applied.
  • hac_floor: Whether each HAC-adjusted square is floored at zero before it enters the variance recursion. On returns with no autocorrelation the floor makes the variance too large, by 16 % at two lags, so nothing, the default, keeps the recursion an unbiased quadratic form and floors only the variance that the estimator returns. PerTermHacFloor floors each square.
  • hac_vol_before: Whether, on the separate correlation path under HAC, each row of the correlation recursion divides by the volatility before the variance step. The diagonal of a HAC row can be negative, and a divisor that holds the row damps a positive row and amplifies a negative one, so the diagonal of the correlation state is skewed down and can fall to zero. VolatilityBeforeUpdate, the default, divides by the volatility before the step: at two lags the rule after the step has 9 % to 19 % more mean squared error in the correlation on most returns, and at eight lags up to 3 % of its rows fall on a guard, where the rule before the step puts none. VolatilityAfterUpdate divides by the volatility after the step, as the plain path does. It has no effect where hac_lags or cor_decay is nothing.
  • cor_vol: The variance that standardises each product of the separate correlation path. The variance state is seeded at zero, so in the warm-up the raw state gives the first products of the correlation more weight than the bias tables of the regime statistic assume. BiasCorrectedVolatility, the default, divides the state by its weight first. nothing reads the raw state, the rule of the oracle. The steady state is the same under both. It has no effect where cor_decay is nothing.
  • regime_method: Regime adjustment method that computes the regime statistic of each observation, or nothing for no regime adjustment.
  • regime_decay: Exponential decay factor of the regime state. A value above 2^(-1/138) ≈ 0.99499, which is a regime half-life above 138 observations, makes the regime state slow to follow the variance that it corrects. After the variance changes level, the multiplier keeps correcting a mismatch that the variance has already absorbed, so it can overshoot and act erratically. The estimator gives no warning, and it computes the same recursion at every value in (0, 1).
  • regime_min_obs: Count of regime observations that the estimator needs before it applies the regime multiplier. Before that count the multiplier is one.
  • regime_target: Regime-adjustment target, which names the regime statistic of one observation and so the part of the covariance that the multiplier calibrates.
  • regime_lohi_mult: Optional (lo, hi) bounds of the regime multiplier. If nothing, no clamp runs.
  • min_val: Minimum threshold to prevent division by zero or degenerate estimates.
  • centring: The location each deviation is taken from. EstimatedCentring, the default, estimates the location of each asset and divides each term by the bias that the estimate puts in it, so the moment is unbiased from the second valid return whatever the mean. PreCentred takes the returns as deviations from a mean of zero. ZeroStartCentring takes the location of a recursion from zero that is not divided by its weight.
  • debias: Whether the regime statistic corrects the bias of the estimate it reads. The inverse of an estimated variance is too large on average (Jensen's inequality). Under ExactDebias, the default, the statistic skips an estimate of n + 3 terms or fewer, for n directions. Where hac_lags is set, it also skips an estimate whose effective count of terms is n + 1 or fewer. It divides the rest by the bias factor that its regime_method reads. The diagonal target also divides its sum by the factor of its law, which reads the correlation of the assets and the noise of each estimate, for the first-moment and the log methods. LawDebias divides by the law of the estimate alone, and takes the deviation as independent of the estimate. nothing scores the raw statistic.

Constructors

RegimeAdjustedExpWeightedCovariance(;    decay::Number                                         = exp2(-inv(40.0)),    cor_decay::Option{<:Number}                           = nothing,    min_obs::Integer                                      = round(Int, max(1, decay_half_life(decay), isnothing(cor_decay) ? 1 : decay_half_life(cor_decay, :cor_decay))),    hac_lags::Option{<:Integer}                           = nothing,    hac_floor::Option{<:AbstractHacFloor}                 = nothing,    hac_vol_before::AbstractHacVolatilityTiming           = VolatilityBeforeUpdate(),    cor_vol::Option{<:AbstractCorrelationVolatility}      = BiasCorrectedVolatility(),    regime_method::Option{<:RegimeAdjustedMethod}         = FirstMomentRegimeAdjusted(),    regime_decay::Number                                  = exp2(-2 / decay_half_life(decay)),    regime_min_obs::Integer                               = round(Int, max(1, decay_half_life(decay) / 2)),    regime_target::RegimeAdjustedTarget                   = PortfolioTarget(),    regime_lohi_mult::Option{<:Tuple{Number, Number}}     = (0.7, 1.6),    min_val::Number                                       = 1e-12,    centring::AbstractCentring                            = EstimatedCentring(),    debias::Option{<:AbstractRegimeDebias}                = ExactDebias(),    cache::Option{<:AbstractPartialFitState}              = nothing) -> RegimeAdjustedExpWeightedCovariance

Keywords correspond to the struct's fields. Where cor_decay is not nothing, the default min_obs reads the slower of the two decays.

Validation

  • 0 < decay < 1.
  • If cor_decay is not nothing, 0 < cor_decay < 1.
  • min_obs > 0.
  • If hac_lags is not nothing, hac_lags > 0.
  • 0 < regime_decay < 1.
  • regime_min_obs > 0.
  • If regime_lohi_mult is not nothing, 0 < regime_lohi_mult[1] < regime_lohi_mult[2].

Examples

julia> ce = RegimeAdjustedExpWeightedCovariance();julia> ce.decay ≈ exp2(-inv(40.0))truejulia> isnothing(ce.cor_decay)true

Related

References

  • [79] J. Menchero, D. J. Orr and J. Wang. The Barra US Equity Model (USE4): Methodology Notes. Model Insight, MSCI (2011). Section 4.3, Equations 4.3 to 4.5.
  • [80] G. A. Paleologo. The Elements of Quantitative Investing. Wiley (2025). Section 6.3.3, Model 6.8 and Equations 6.9 to 6.11.
  • [18] W. K. Newey and K. D. West. A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55, 703–708 (1987). Equation 5.
  • [81] R. Engle. Dynamic conditional correlation: A simple class of multivariate generalized autoregressive conditional heteroskedasticity models. Journal of Business & Economic Statistics 20, 339–350 (2002). Equation 17.
source

Functions

PortfolioOptimisers.min_active_assets — Function
min_active_assets(_::RegimeAdjustedTarget) -> Int64

Returns the smallest count of scored assets at which a regime-adjustment target gives a statistic.

Arguments

  • ::RegimeAdjustedTarget: Regime-adjustment target (unused by this default method).

Returns

  • 1::Int: The default is one scored asset.

Related

source
min_active_assets(_::MahalanobisTarget) -> Int64

Returns the smallest count of scored assets at which the Mahalanobis target gives a statistic.

Arguments

  • ::MahalanobisTarget: Mahalanobis regime-adjustment target (unused).

Returns

  • 2::Int: The target needs two scored assets.

Related

source
PortfolioOptimisers.regime_statistic — Function
regime_statistic(
    _::MahalanobisTarget,
    X::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    C::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    _::AbstractVector{<:Integer},
    min_val::Number
) -> Union{Nothing, Vector}

Computes the squared Mahalanobis distance of one observation against the covariance block.

The target reads every eigen-direction of the block, so it calibrates the whole covariance structure. It needs two active assets, which min_active_assets states.

Arguments

  • ::MahalanobisTarget: Mahalanobis regime-adjustment target.
  • X::VecNum: Centred returns of the assets that contribute to the statistic.
  • C::MatNum: Bias-corrected covariance block of those assets.
  • ::AbstractVector{<:Integer}: Ignored index of those assets.
  • min_val::Number: Scale of the ridge that safe_regime_cholesky applies.

Returns

  • stats::Option{<:VecNum}: One statistic, or nothing where the block does not factorise.

Related

source
regime_statistic(
    _::DiagonalTarget,
    X::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    C::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    _::AbstractVector{<:Integer},
    min_val::Number
) -> Vector

Computes the squared standardised Euclidean distance of one observation against the marginal volatilities of the covariance block.

The target divides each return by its own volatility and sums the squares, so it calibrates the diagonal risk scale and reads no correlation.

Arguments

  • ::DiagonalTarget: Diagonal regime-adjustment target.
  • X::VecNum: Centred returns of the assets that contribute to the statistic.
  • C::MatNum: Bias-corrected covariance block of those assets.
  • ::AbstractVector{<:Integer}: Ignored index of those assets.
  • min_val::Number: Floor applied to each variance before its root is taken.

Returns

  • stats::VecNum: One statistic.

Related

source
regime_statistic(
    target::PortfolioTarget,
    X::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    C::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    idx::AbstractVector{<:Integer},
    min_val::Number
) -> Any

Computes the squared standardised return of one observation along each portfolio direction.

Each direction of portfolio_directions gives its own statistic, and the caller averages the transformed values.

Algorithm

  1. Build the directions over the contributing assets with portfolio_directions. Return nothing where no row keeps a positive weight.
  2. Return nothing where the variance of a direction is not above min_val. A HAC block in the warm-up need not be positive definite, and a floor at min_val scores the row at about $r^{2} / 10^{-12}$.
  3. Return the count of contributing assets times the squared portfolio return, divided by the portfolio variance, for each row.

Arguments

  • target::PortfolioTarget: Portfolio regime-adjustment target.
  • X::VecNum: Centred returns of the assets that contribute to the statistic.
  • C::MatNum: Bias-corrected covariance block of those assets.
  • idx::AbstractVector{<:Integer}: Index of those assets in the universe.
  • min_val::Number: Floor applied to each variance, and the smallest portfolio variance that the statistic reads.

Returns

  • stats::Option{<:VecNum}: One statistic per direction, or nothing where no row keeps a positive weight or where the variance of a direction is not above min_val.

Related

source
PortfolioOptimisers.regime_kappa — Function
regime_kappa(
    method::LogRegimeAdjusted,
    _::Union{DiagonalTarget, MahalanobisTarget},
    n::Integer
) -> Any

Computes the stationary expectation of the log regime statistic for a target that reads the whole active block.

MahalanobisTarget and DiagonalTarget both sum n standardised squares, so the statistic is a $\chi^2(n)$ variate under correct calibration and its log has expectation $\psi(x n) + \ln y$. The scalar case of RegimeAdjustedExpWeightedVariance is this expression at n = 1. The squares of the diagonal target are correlated, so its sum is not a $\chi^2(n)$ variate. So the target first divides the debiased statistic by diagonal_law_factor. Then this constant holds at the correlation of the assets and at the noise of each estimate, to the second order in that noise.

Arguments

  • method::LogRegimeAdjusted: Log regime adjustment method.
  • ::Union{MahalanobisTarget, DiagonalTarget}: Regime-adjustment target.
  • n::Integer: Count of assets that contribute to the statistic.

Returns

  • kappa::Number: digamma(method.x * n) + log(method.y).

Related

source
regime_kappa(
    method::LogRegimeAdjusted,
    _::PortfolioTarget,
    n::Integer
) -> Any

Computes the stationary expectation of the log regime statistic for the portfolio target.

PortfolioTarget reads one direction and multiplies by n, so its statistic is n times a $\chi^2(1)$ variate and the log has expectation $\ln n + \psi(x) + \ln y$.

Arguments

  • method::LogRegimeAdjusted: Log regime adjustment method.
  • ::PortfolioTarget: Portfolio regime-adjustment target.
  • n::Integer: Count of assets that contribute to the statistic.

Returns

  • kappa::Number: log(n) + digamma(method.x) + log(method.y).

Related

source
PortfolioOptimisers.regime_denom — Function
regime_denom(
    method::FirstMomentRegimeAdjusted,
    _::PortfolioTarget,
    n::Integer
) -> Any

Computes the first-moment normalisation of the regime statistic for the portfolio target.

The statistic is n times a squared standard normal, so the root has expectation $\sqrt{n}\,x$ with $x = \sqrt{2/\pi}$.

Arguments

  • method::FirstMomentRegimeAdjusted: First-moment regime adjustment method.
  • ::PortfolioTarget: Portfolio regime-adjustment target.
  • n::Integer: Count of assets that contribute to the statistic.

Returns

  • denom::Number: sqrt(n) * method.x.

Related

source
regime_denom(
    method::FirstMomentRegimeAdjusted,
    _::Union{DiagonalTarget, MahalanobisTarget},
    n::Integer
) -> Any

Computes the first-moment normalisation of the regime statistic for a target that reads the whole active block.

MahalanobisTarget and DiagonalTarget both sum n standardised squares. The Mahalanobis statistic is a $\chi^2(n)$ variate, so the root has expectation $\sqrt{2}\,\Gamma((n+1)/2)/\Gamma(n/2)$. The expression is evaluated through the log gamma function, which stays finite for a wide universe. At n = 1 it is $\sqrt{2/\pi}$, which is the constant the scalar case of RegimeAdjustedExpWeightedVariance carries.

The squares of the diagonal target are correlated, so its sum is a $\chi^2(n)$ variate only at independent returns. There the constant is exact. The root of n, which is the root of the mean of the statistic, is too large at every correlation. The raw statistic of debias = nothing divides by this constant. The target first divides the debiased statistic by diagonal_law_factor. Then this constant holds at the correlation of the assets and at the noise of each estimate, to the second order in that noise. That factor reads the constant, so the debiased statistic does not depend on it.

Arguments

  • method::FirstMomentRegimeAdjusted: First-moment regime adjustment method. The type of its field x is the type of the constant.
  • ::Union{MahalanobisTarget, DiagonalTarget}: Regime-adjustment target.
  • n::Integer: Count of assets that contribute to the statistic.

Returns

  • denom::Number: sqrt(2) * exp(loggamma((n + 1) / 2) - loggamma(n / 2)).

Related

source
PortfolioOptimisers.get_regime_state — Method
get_regime_state(
    _::RootMeanSquaredAdjusted,
    _::RegimeAdjustedTarget,
    stats::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    n::Integer,
    _
) -> Any

Transforms the root-mean-squared regime statistics into the values the regime state smooths.

Arguments

  • ::RootMeanSquaredAdjusted: Root-mean-squared regime adjustment method (unused).
  • ::RegimeAdjustedTarget: Regime-adjustment target (unused).
  • stats::VecNum: One statistic per calibration direction.
  • n::Integer: Count of assets that contribute to the statistic.
  • ::Any: Ignored minimum value argument.

Returns

  • val::VecNum: stats ./ n.

Related

source
PortfolioOptimisers.get_regime_state — Method
get_regime_state(
    method::FirstMomentRegimeAdjusted,
    target::RegimeAdjustedTarget,
    stats::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    n::Integer,
    _
) -> Any

Transforms the first-moment regime statistics into the values the regime state smooths.

Arguments

  • method::FirstMomentRegimeAdjusted: First-moment regime adjustment method.
  • target::RegimeAdjustedTarget: Regime-adjustment target, which names the normalisation.
  • stats::VecNum: One statistic per calibration direction.
  • n::Integer: Count of assets that contribute to the statistic.
  • ::Any: Ignored minimum value argument.

Returns

  • val::VecNum: sqrt.(max.(stats, 0)) ./ regime_denom(method, target, n).

Related

source
PortfolioOptimisers.get_regime_state — Method
get_regime_state(
    method::LogRegimeAdjusted,
    target::RegimeAdjustedTarget,
    stats::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    n::Integer,
    min_val::Number
) -> Any

Transforms the log regime statistics into the values the regime state smooths.

Arguments

  • method::LogRegimeAdjusted: Log regime adjustment method.
  • target::RegimeAdjustedTarget: Regime-adjustment target, which names the expectation.
  • stats::VecNum: One statistic per calibration direction.
  • n::Integer: Count of assets that contribute to the statistic.
  • min_val::Number: Floor applied before the logarithm.

Returns

  • val::VecNum: log.(max.(stats, min_val)) .- regime_kappa(method, target, n).

Related

source

References

[18]
W. K. Newey and K. D. West. A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55, 703–708 (1987).
[79]
J. Menchero, D. J. Orr and J. Wang. The Barra US Equity Model (USE4): Methodology Notes (MSCI, 2011).
[80]
G. A. Paleologo. The Elements of Quantitative Investing (Wiley, 2025).
[81]
R. Engle. Dynamic conditional correlation: A simple class of multivariate generalized autoregressive conditional heteroskedasticity models. Journal of Business & Economic Statistics 20, 339–350 (2002).