HAC row kernel: private API

Functions

PortfolioOptimisers.hac_lag_weights — Function
hac_lag_weights(
    _::Nothing,
    o::Number
) -> Vector{T} where T<:Number

Returns the weight of each lag of a HAC estimate, in the number type of o.

A count of lags names the Bartlett weights $k_{i} = 1 - i / (L + 1)$. A vector is already the weight of each lag, such as the kernel of hac_row_kernel, and it is returned as it is. Without a HAC adjustment there is no lag.

Arguments

  • hac_lags::Option{<:Union{Integer, VecNum}}: Count of HAC lags, the weight of each lag, or nothing.
  • o::Number: A number whose type the Bartlett weights take.

Returns

  • k::VecNum: The weight of each lag, empty without a HAC adjustment.

Related

source
PortfolioOptimisers.hac_damped_point! — Function
hac_damped_point!(
    buf::NamedTuple,
    sc::Number,
    mix::Tuple,
    col::Integer
) -> Any

Adds one point of the grid to the moments of the damped lag terms: the log of the Laplace transform of a HAC variance estimate, and one column of the inverse on the newest rows.

The estimate is $x^{\top} M x$ of iid standard Normal returns, newest first, with $M_{jj} = c \mu_{j}$ and $M_{j, j+i} = c \mu_{j} w_{i}$, $c = 1 - \lambda$: the newer return of each lagged product carries the weight. The row weights mix two variances, $\mu_{j} = a \lambda^{j} + b \lambda^{j - d}$ for $j \geq d$. The banded LDLᵀ factorisation of $I + s M$ adds one row at a time, and the rows of $L^{-1}$ on the newest columns follow it on the same band, so

\[\begin{align} \left[(I + s M)^{-1}\right]_{a c} &= \sum_{j} \frac{(L^{-1})_{j a} (L^{-1})_{j c}}{d_{j}}\,. \end{align}\]

A row whose weight $s c \mu_{j}$ falls below the machine epsilon enters the log-determinant as its weight, so the loop stops there, and the geometric sum of the rest closes it: the history is infinite.

Arguments

  • buf::NamedTuple: Buffers (mutated): lam, the decay; w, the Bartlett weight of each lag; lrow, lpr, dpr, upr and mpr, the current row and the last rows of the factor, their pivots, their rows of $L^{-1}$ and their weights; u, scratch; and acc, the column of the inverse on the first rows.
  • sc::Number: The point $s$ times the normalisation $c$.
  • mix::Tuple: $(a, b, d)$.
  • col::Integer: Column of the inverse, from zero for the newest return.

Returns

  • lG::Number: $-\frac{1}{2} \ln \det(I + s M)$, or $-\infty$ where a pivot is not positive: the determinant has crossed zero.

Related

source
PortfolioOptimisers.hac_damped_row! — Function
hac_damped_row!(
    buf::NamedTuple,
    mj::Number,
    j::Integer,
    col::Integer
) -> Any

Adds row $j$ to the banded LDLᵀ factorisation of hac_damped_point!, and its row of $L^{-1}$ on the newest columns to the column of the inverse.

The entry of the row at lag $i$ is $s c \mu_{j-i} w_{i}$, the weight of the newer return, so the row reads the weights of the last rows from mpr. The row of $L^{-1}$ is $u_{j} = e_{j} - \sum_{i} L_{j, j-i}\, u_{j-i}$ on the band, and it adds $u_{j a}\, u_{j c} / d_{j}$ to the column. A pivot that is not positive adds nothing.

Arguments

  • buf::NamedTuple: Buffers of hac_damped_point! (mutated).
  • mj::Number: The weight $s c \mu_{j}$ of the row.
  • j::Integer: Index of the row, from zero for the newest return.
  • col::Integer: Column of the inverse, from zero for the newest return.

Returns

  • dj::Number: The pivot of the row.

Related

source
PortfolioOptimisers.hac_damped_shift! — Function
hac_damped_shift!(
    buf::NamedTuple,
    dj::Number,
    mj::Number
) -> NamedTuple

Moves the last rows of the factorisation of hac_damped_point! one place back, and keeps the new row first: its pivot, its weight, its entries and its row of $L^{-1}$.

Arguments

  • buf::NamedTuple: Buffers of hac_damped_point! (mutated).
  • dj::Number: The pivot of the new row.
  • mj::Number: The weight of the new row.

Returns

  • buf::NamedTuple: The buffers after the move.

Related

source
PortfolioOptimisers.hac_damped_integral! — Function
hac_damped_integral!(
    buf::NamedTuple,
    sig::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    mix::Tuple,
    col::Integer
) -> Tuple{Any, Any}

Integrates the moments of hac_damped_point! over the grid, and returns $\mathbb{E}[x_{a} x_{c} / Q]$ for each $a$ on the first rows, with $\mathbb{E}[1 / Q]$.

With $Q = x^{\top} M x$, $\mathbb{E}[x_{a} x_{c} / Q] = \frac{1}{2} \int_{0}^{\infty} [(I + s M)^{-1}]_{a c} \det(I + s M)^{-1/2}\, ds$. The trapezoid rule on $\ln s$ is spectrally accurate on this integrand. Where $M$ has a negative eigenvalue the transform falls to a minimum and then rises, and the integral stops at the minimum with half weight, as hac_laplace! does: it is the moment of the positive part.

Arguments

  • buf::NamedTuple: Buffers of hac_damped_point! (mutated).
  • sig::VecNum: Grid of $s (1 - \lambda)$, uniform in its log.
  • mix::Tuple: $(a, b, d)$ of hac_damped_point!.
  • col::Integer: Column of the inverse, from zero for the newest return.

Returns

  • (m, e)::Tuple{VecNum, Number}: $\mathbb{E}[x_{a} x_{c} / Q]$ for each $a$ on the first rows, and $\mathbb{E}[1 / Q]$.

Related

source
PortfolioOptimisers.hac_row_kernel — Function
hac_row_kernel(decay::Number, hac_lags::Integer) -> Any

Computes the HAC kernel whose raw estimate has the noise of the separate correlation path, where each row divides by the volatility before its update.

Row $p$ of the correlation recursion reads $y_{p} = x_{p} / \sigma_{p-1}$ and the lagged terms $\tilde{y}_{p,l} = x_{p-l} / \sigma_{p-1}$. The variance $V_{p-1}$ already holds $x_{p-l}^{2}$ and the HAC products of $x_{p-l}$, so each lagged term is damped and correlated with the returns near it, while its mean product with $y_{p}$ stays zero. The kernel of regime_bias_table at cor_decay assumes the raw noise, so the factor over-corrects: at 12 assets, half-lives of 10 and 20 and two lags, the truth reads 2.9 % below it.

Regress each damped term on the returns divided by their own volatility before their own update, $\tilde{y}_{p,l} = \sum_{d=1}^{2L} \beta_{l,d}\, y_{p-d} + e_{p,l}$. The $y$ are uncorrelated, each of variance $\mathbb{E}[1/V]$, and the residual is small. A row of the correlation state is then a raw HAC product of the $y$ with the kernel below, whose width is $2L$, and the residual variance enters in quadrature at $d = l$. Both moments come from the law of the steady-state HAC variance, the second through $1 / \sqrt{ab} = \frac{2}{\pi} \int_{0}^{\pi/2} d\theta / (a \sin^{2}\theta + b \cos^{2}\theta)$, which turns the product of two volatilities into one quadratic form.

Mathematical definition

\[\begin{align} \beta_{l,d} &= \frac{\mathbb{E}\left[x_{p-l}\, x_{p-d} / (\sigma_{p-1}\, \sigma_{p-d-1})\right]}{\mathbb{E}[1 / V]}\,, \qquad \Gamma_{l} = \frac{\mathbb{E}\left[x_{p-l}^{2} / V_{p-1}\right]}{\mathbb{E}[1 / V]}\,, \\ \omega_{d} &= \sum_{l=1}^{L} k_{l}\, \beta_{l,d}\,, \qquad \omega_{l} \leftarrow \operatorname{sign}(\omega_{l}) \sqrt{\omega_{l}^{2} + k_{l}^{2} \left(\Gamma_{l} - \sum_{d} \beta_{l,d}^{2}\right)}\,. \end{align}\]

Where:

  • $x$: iid standard Normal returns.
  • $V_{t}$: The HAC variance at decay after observation $t$, $\sigma_{t}^{2} = V_{t}$.
  • $k_{l}$: Bartlett weight $1 - l / (L + 1)$ of lag $l$.
  • $\omega_{d}$: Weight of lag $d$ of the kernel, $d = 1, \ldots, 2L$.

At a half-life of 10 and two lags the kernel is 0.613, 0.290, −0.022 and −0.003, against the Bartlett 0.667 and 0.333. With it the mean of the debiased statistic reads 0.2 % to 2.2 % high at $R = I$ over 4 to 24 assets, half-lives of 5 to 20, correlation half-lives of 20 and 40 and one to four lags, where it read 0.8 % to 9.5 % low. The kernel is that of the steady state, and the rows of the warm-up read it too. What remains is the part that every row rule has: the time modulation of the rows and their coupling with $1 / \hat{V}$, which the factor of variance_noise_bias! does not read.

Algorithm

  1. Lay the grid of $\ln s$ on $[-24, 8]$ with step $1/4$, and the composite 4-point Gauss–Legendre rule of gauss_legendre_rule on two halves of $[0, \pi/2]$.
  2. With hac_damped_integral!, find $\mathbb{E}[1/V]$ and $\Gamma_{l}$ from one variance, and each column $d$ of $\beta$ from the mixed variance at each node of the rule.
  3. Sum the kernel, and add the residual in quadrature.

Against a fine grid (step $1/20$ on $[-40, 30]$, 32 nodes) the moments agree to $2 \times 10^{-7}$ at a half-life of 10, and to $3 \times 10^{-5}$ at a half-life of 5 and four lags, where the estimate is negative in some draws and the cut of the positive part moves with the grid. Against a chain of $2 \times 10^{7}$ rows they agree to its noise, $3 \times 10^{-4}$. The kernel costs 0.1 s at a half-life of 10 and two lags, and 4 s at a half-life of 250.

Arguments

  • decay::Number: Decay of the variance.
  • hac_lags::Integer: Count of HAC lags.

Returns

  • kernel::VecNum: The weight of each of the $2L$ lags.

Related

source
PortfolioOptimisers.correlation_hac_lags! — Function
correlation_hac_lags!(
    store::Union{DiagonalBiasStore, MahalanobisBiasStore},
    ce::RegimeAdjustedExpWeightedCovariance
) -> Any

Returns the HAC weights that the bias factor of the correlation reads: the kernel of hac_row_kernel on the separate correlation path where each row divides by the volatility before its update, else ce.hac_lags. The kernel is made the first time the state needs it.

Arguments

  • store::Union{DiagonalBiasStore, MahalanobisBiasStore}: Store of the state, whose kernel fills on the first call (mutated).
  • ce::RegimeAdjustedExpWeightedCovariance: Covariance estimator configuration.

Returns

  • hac::Option{<:Union{Integer, VecNum}}: The kernel, or ce.hac_lags.

Related

source
PortfolioOptimisers.hac_laplace! — Function
hac_laplace!(
    G::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    lG::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
) -> AbstractVector{<:Union{Number, AbstractJuMPScalar}}

Forms the Laplace transform of a HAC estimate on the grid, cut at its minimum.

Where the weight matrix is positive definite, $G$ falls on the whole grid and nothing is cut. Where it is not, $G$ falls to a minimum and then rises to the first zero of the determinant, where the negative part of the estimate rules the transform. The transform is cut at the minimum, and the last point takes the weight one half of a closed trapezoid, so the error of the cut is of the second order in the step.

Arguments

  • G::VecNum: Transform on the grid (mutated).
  • lG::VecNum: $\ln G$ on the grid, $-\infty$ past the first zero of the determinant.

Returns

  • G::VecNum: The cut transform.

Related

source