Estimated Mahalanobis bias: private API

estimated_log_det_chain(
    decay::Number,
    K::Integer,
    sigma::Number,
    k::AbstractVector{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}}
) -> NTuple{5, Any}

Computes $\ln \det(I + \sigma M_{K})$ of an estimate of K terms under EstimatedCentring, its derivative in $\sigma$, and the tilted variance of the location, at one point $\sigma$.

The chain of estimated_bias_count! for the point $\sigma$ after K terms reads the points $\sigma \lambda^{K - j}$ at the count $j$. This function runs that chain alone, at the complex point $\sigma (1 + i \varepsilon)$, so each derivative is exact to the round-off. A count whose point is below the seed $\ln s_{0} = \ln \epsilon + \ln(1 - \lambda)$ reads the state at $s = 0$, as the lattice does.

Arguments

• decay::Number: Decay of the weights.
• K::Integer: Count of the terms of the estimate.
• sigma::Number: The point $\sigma$.
• k::VecNum: Weight of each lag, from hac_lag_weights.

Returns

• (alive, rho, g, vg, dvg)::Tuple: Whether the determinant is positive along the chain, the derivative $\rho_{0} = \mathrm{d} g / \mathrm{d} \sigma$, $g$, the tilted variance $P_{11}$ of the location, and its derivative in $\ln \sigma$.

Related

• estimated_bias_count!
• hac_log_det_peak
• estimated_mahalanobis_columns!

estimated_mahalanobis_columns!(
    cols::NamedTuple,
    lat::NamedTuple,
    xf::AbstractRange,
    j0::Integer,
    decay::Number,
    K::Integer
) -> NamedTuple

Fills the columns of the Mahalanobis level recursion at one count from the lattice of estimated_bias_lattice, on the lattice xf of mahalanobis_bias_grid that shares its points.

The columns are $g = \ln \det(I + \sigma M_{K})$ and $\rho_{0} = g'$, which the law reads, and $\sigma V_{g}(\sigma)$ and its derivative, which the dependence of the deviation reads: with $V_{g} = g^{\top} (I + \sigma M_{K})^{-1} g = P_{11}$, the location moves $g$ by $t\, \sigma V_{g}$ to the first order where $M_{K}$ takes $t\, g g^{\top}$.

• Below the seed: $g = \sigma \kappa_{1}$, $\rho_{0} = \kappa_{1}$, and $V_{g}$ at $s = 0$.
• From the first point whose determinant is not positive: $g = \infty$ and zero elsewhere, as hac_log_det_path! marks such a point.
• Above the top of the lattice: the continuation of mahalanobis_cut!, $g = g_{t} + N_{t} \ln(\sigma / \sigma_{t})$ with $N_{t} = \sigma_{t} \rho_{0}(\sigma_{t})$, and $\sigma V_{g}$ held. There $e^{-g / 2} < \epsilon^{3}$.

Arguments

• cols::NamedTuple: g, r, vt and rt on xf (mutated).
• lat::NamedTuple: Complex lattice after the count K.
• xf::AbstractRange: Lattice of $\ln \sigma$, whose point j0 + i is the point i of lat.
• j0::Integer: Count of the points of xf below the seed.
• decay::Number: Decay of the weights.
• K::Integer: Count of the terms of the estimate.

Returns

• cols::NamedTuple: The filled columns.

Related

• mahalanobis_level_bias
• estimated_bias_lattice

mahalanobis_level_sum!(
    grid::NamedTuple,
    gh::AbstractVector{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    rh::AbstractVector{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    peak::Union{Nothing, Tuple},
    shift::Tuple,
    n::Integer,
    bufs::NamedTuple
) -> Tuple{Any, Any, Any}

Runs the level recursion of mahalanobis_level_bias on one pair of columns, and returns the transform of the last level with the sum over the levels of the integral of each transform.

Each level $k$ of the recursion gives the Laplace transform $L_{k}$ of the Schur complement $S_{k}$ of one direction in $k + 1$ directions. The determinant of the estimate is the product of the $S_{k}$, so $\mathbb{E}[\ln \det W] = \sum_{k} \mathbb{E}[\ln S_{k}]$, and each term is $\int_{0}^{\infty} (e^{-s} - L_{k}(s))\, s^{-1}\, \mathrm{d}s$ up to a constant. The function returns $J = \sum_{k = 0}^{n - 1} \int L_{k}(s)\, s^{-1}\, \mathrm{d}s$ on the output grid, so a change of the columns moves $\mathbb{E}[\ln \det W]$ by minus the change of $J$. With a cut, each level reads the transform of mahalanobis_transform at the cut.

Arguments

• grid::NamedTuple: Grids of mahalanobis_bias_grid.
• gh::VecNum: $g$ on the lattice (mutated by mahalanobis_cut!).
• rh::VecNum: $\rho_{0}$ on the lattice (mutated by mahalanobis_cut!).
• peak::Option{<:Tuple{<:Number, <:Number}}: The peak of hac_log_det_peak, or nothing.
• shift::Tuple{<:Integer, <:Number}: The shift $m$ of the lattice and the normalisation $\tilde{c}$.
• n::Integer: Count of assets that contribute to the statistic.
• bufs::NamedTuple: Buffers dg, dr, v, e and lv of the recursion (mutated).

Returns

• (so, L, J)::Tuple: The output grid and the transform of the last level, and $J$.

Related

• mahalanobis_level_bias
• mahalanobis_level_step!
• mahalanobis_transform

estimated_mahalanobis_level_bias(
    method::RegimeAdjustedMethod,
    decay::Number,
    Ks::AbstractVector{<:Integer},
    n::Integer,
    hac_lags::Union{Nothing, Integer},
    debias::AbstractRegimeDebias
) -> Any

Computes the Mahalanobis bias factor at each count of Ks where each deviation is taken from an estimated location: the law of the estimate on its dense form and, where the debias rule reads it, the dependence of the deviation on the estimate.

The terms share their location, so the estimate of a pair with one shared history is $W = Z^{\top} M_{K} Z$ with the dense $M_{K}$ of regime_bias_table, and the recursion starts from $g = \ln \det(I + \sigma M_{K})$ in place of the banded form. The lattice of estimated_bias_lattice gives $g$ and its derivative on the points of the grid, at the step $|\ln \lambda| / m$ in place of $1/8$, and the peak of a HAC estimate takes the exact slope of estimated_log_det_chain. The deviation $u = z - Z^{\top} g$ reads the same returns as $W$, and estimated_mahalanobis_dependence gives the part of each method that this dependence moves.

The columns agree with the eigenvalues of a dense $M_{K}$ to $10^{-11}$. Against a Monte Carlo of a million draws of the statistic at half-lives of 5, 10 and 40, no lag to four lags and two to 12 assets, the three factors of ExactDebias are within 0.3 % at the steady state, and without HAC at the first scored count. Near the gate under HAC they read 0.3 % to 1.7 % low, and 4.6 % low for the root mean square at five assets and a half-life of 10, the error of the level recursion. The factors of the law alone, LawDebias, are 0.2 % to 1.4 % high. The pre-centred factors are 1.3 % to 3.1 % high under HAC, and 7.5 % at the first scored count of a half-life of 40.

Algorithm

1. Make the complex lattice from $\ln \epsilon + \ln(1 - \lambda)$ to 51, and the grid whose xf shares its points, up to the top of the output grid and one stencil past it.
2. Advance the lattice to each count of Ks in order, and fill the columns with estimated_mahalanobis_columns!.
3. Read the factor of the count with estimated_mahalanobis_factor.

Arguments

• method::RegimeAdjustedMethod: Regime adjustment method, which names the moment.
• decay::Number: Decay of the weights.
• Ks::AbstractVector{<:Integer}: Counts of observations, each above n + 1.
• n::Integer: Count of assets that contribute to the statistic, at least two.
• hac_lags::Option{<:Integer}: Count of HAC lags, or nothing.
• debias::AbstractRegimeDebias: The debias rule, read by reads_dependence.

Returns

• factors::VecNum: The factor at each count of Ks.

Related

• mahalanobis_level_bias
• estimated_mahalanobis_columns!
• estimated_mahalanobis_factor
• mahalanobis_level_sum!

estimated_mahalanobis_factor(
    method::RegimeAdjustedMethod,
    ctx::NamedTuple,
    decay::Number,
    K::Integer
) -> Any

Computes the Mahalanobis bias factor of one count under EstimatedCentring from the columns of estimated_mahalanobis_columns!.

Algorithm

1. Round the normalisation $c$ to the lattice, $\tilde{c} = e^{m h_{f}}$.
2. Find the peak with hac_log_det_peak on the slope of estimated_log_det_chain, and take the value of $g$ there from the chain too.
3. Run mahalanobis_level_sum!, read the factor of the law with regime_bias_factor, and scale it by $\tilde{c} / c$.
4. Where reads_dependence holds for the debias rule, replace it with the factor of estimated_mahalanobis_dependence.

Arguments

• method::RegimeAdjustedMethod: Regime adjustment method, which names the moment.
• ctx::NamedTuple: The grid, the columns, the buffers of the recursion, j0, the count of assets n, the weight k of each lag, and the debias rule.
• decay::Number: Decay of the weights.
• K::Integer: Count of the terms of the estimate.

Returns

• factor::Number: The factor at K.

Related

• estimated_mahalanobis_level_bias
• estimated_mahalanobis_dependence

estimated_perp_run(
    ctx::NamedTuple,
    peak::Union{Nothing, Tuple},
    shift::Tuple,
    f::Number
) -> Tuple{Any, Any, Any}

Runs the level recursion on the columns of the estimate with the direction of the deviation taken out, $W_{\perp}$, the part of estimated_mahalanobis_dependence that the first moment and the log read.

In the frame where the deviation is the first direction, $\ln \det W = \ln \det W_{\perp} + \ln S$, with $S$ the Schur complement of that direction, and $d^{2} = \|u\|^{2} / S$. $W_{\perp}$ is a plain weighted Wishart of $n - 1$ directions, whose weights are those of the history and the new return, $\tilde{M} = \operatorname{diag}(M_{K}, 0)$, compressed onto the complement of $e = (-g; 1)$. By the identity of Jacobi its log-determinant is that of $\tilde{M}$ plus $\ln (V / f)$, with $V = e^{\top} (I + \sigma \tilde{M})^{-1} e = 1 + V_{g}$ and $f = e^{\top} e$.

Mathematical definition

\[\begin{align} g_{\perp}(\sigma) &= g(\sigma) + \ln \frac{V(\sigma)}{f}\,, \quad \rho_{\perp}(\sigma) = \rho_{0}(\sigma) + \frac{V'(\sigma)}{V(\sigma)}\,, \quad V(\sigma) = 1 + \frac{\sigma V_{g}(\sigma)}{\sigma}\,. \end{align}\]

Arguments

• ctx::NamedTuple: The context of estimated_mahalanobis_factor.
• peak::Option{<:Tuple}: The peak and the values of estimated_log_det_chain there, or nothing. The cut stays at the peak of $g$.
• shift::Tuple{<:Integer, <:Number}: The shift $m$ of the lattice and $\tilde{c}$.
• f::Number: $f = 1 + g^{\top} g$, centring_factor of the deviation.

Returns

• (so, L, J)::Tuple: The run of mahalanobis_level_sum! over $n$ levels.

Related

• estimated_mahalanobis_dependence
• mahalanobis_level_sum!

estimated_mahalanobis_dependence(::RegimeAdjustedMethod, b::Number, ctx, law, peak, shift, decay, K)
estimated_mahalanobis_dependence(::RootMeanSquaredAdjusted, b::Number, ctx, law, peak, shift, decay, K)
estimated_mahalanobis_dependence(::FirstMomentRegimeAdjusted, b::Number, ctx, law, peak, shift, decay, K)
estimated_mahalanobis_dependence(::LogRegimeAdjusted, b::Number, ctx, law, peak, shift, decay, K)

Computes the Mahalanobis factor of a method with the dependence of the deviation on the estimate, from the factor of the law.

The deviation is $u = z - Z^{\top} g$ and the statistic $d^{2} = u^{\top} (c W)^{-1} u / f$. In the frame where $u$ is the first direction, $d^{2} \propto \|u\|^{2} / S$ with $S$ the Schur complement of that direction, and $\|u\|^{2} / f$ is a $\chi^{2}_{n}$.

• The root mean square reads $\mathbb{E}[u^{\top} W^{-1} u] = \mathbb{E}[\operatorname{tr} W^{-1}] + \mathbb{E}[y^{\top} W^{-1} y]$, $y = Z^{\top} g$, and the second term is the derivative in $t$ of $\mathbb{E}[\ln \det Z^{\top}(M_{K} + t\, g g^{\top}) Z]$: the columns move by $t\, \sigma V_{g}$, and a complex step of $J$ in $t$ gives it, at a fixed cut. A finite step does not: $t\, \sigma V_{g}$ grows with $\sigma$ past the range where the moved columns hold, so the step is $\sqrt{\epsilon}$ over the largest $\sigma V_{g}$, and the derivative has no cancellation.
• The log reads $\mathbb{E}[\ln d^{2}]$, and $\ln d^{2} = \ln \|u\|^{2} + \ln \det W_{\perp} - \ln \det W$ up to the scale. Each log-determinant is the sum over the levels of $\mathbb{E}[\ln S_{k}] = \int (e^{-s/2} - L_{k}) s^{-1} \mathrm{d}s$, from mahalanobis_level_sum! and estimated_perp_run.
• The first moment reads $\mathbb{E}[d]$. Given the other directions, $S$ is a quadratic form in the first, whose coordinate along $e$ is $\|u\| / \sqrt{f}$, so the mean over that coordinate is in closed form. With the mean log-determinant of each level inside the exponential, as the recursion takes it, the transform of the last level $L$ becomes $L^{n + 1} L_{\perp}^{-n}$.

Any other method returns the factor of the law.

Mathematical definition

\[\begin{align} b_{R} &= \frac{1}{f} \left(b - \frac{\operatorname{Im} J(i t)}{t\, n c}\right)\,, \\ b_{L} &= \frac{\tilde{c}}{c} \exp\left(J - J_{\perp} - h \sum_{s} e^{-s / 2}\right)\,, \\ b_{F} &= \frac{\tilde{c}}{c} \frac{1}{2 \pi} \left(h \sum_{s} s^{1/2} L(s)^{n + 1} L_{\perp}(s)^{-n}\right)^{2}\,. \end{align}\]

Where $J$ sums the levels of $W$ and $J_{\perp}$ the $n - 1$ levels of $W_{\perp}$, and the sums run over the output grid of the last level.

Arguments

• method::RegimeAdjustedMethod: Regime adjustment method.
• b::Number: The factor of the law.
• ctx::NamedTuple: The context of estimated_mahalanobis_factor.
• law::Tuple: The run of mahalanobis_level_sum! of the law: the output grid, the transform of the last level and $J$.
• peak::Option{<:Tuple}: The peak and the values of estimated_log_det_chain there, or nothing.
• shift::Tuple{<:Integer, <:Number}: The shift $m$ of the lattice and $\tilde{c}$.
• decay::Number: Decay of the weights.
• K::Integer: Count of the terms of the estimate.

Returns

• factor::Number: The factor of the method.

Related

• estimated_mahalanobis_factor
• estimated_perp_run
• mahalanobis_level_sum!