Lower Tail Dependence Covariance: private API

PortfolioOptimisers.lower_tail_dependence — Function
lower_tail_dependence(X::MatNum, alpha::Number = 0.05,
                      ex::FLoops.Transducers.Executor = FLoops.SequentialEx())

Compute the lower tail dependence matrix of a matrix of asset returns.

The lower tail dependence of two assets is the probability that the returns of the two assets are in their lower tails at the same time, at the quantile level alpha. The function estimates it for each pair of columns of X.

Mathematical definition

For a quantile level $\alpha \in (0,1)$ and $k = \lceil T \alpha \rceil$, let $\hat{q}_i$ be the empirical $\alpha$-quantile of asset $i$, which is the $k$-th order statistic. The estimate of the lower tail dependence of assets $i$ and $j$ is:

\[\begin{align} \hat{\lambda}_{ij} &= \frac{1}{k} \sum_{t=1}^{T} \mathbf{1}\left[x_{ti} \leq \hat{q}_i \text{ and } x_{tj} \leq \hat{q}_j\right]\,. \end{align}\]

Where:

  • $\hat{\lambda}_{ij}$: The estimate of the lower tail dependence of assets $i$ and $j$.
  • $T$: Number of observations.
  • $\alpha$: Significance level (left tail probability), $\alpha \in (0, 1)$.
  • $k = \lceil T \alpha \rceil$: The number of observations in the lower tail.
  • $x_{ti}$: The return of asset $i$ at time $t$.
  • $\hat{q}_i$: The empirical $\alpha$-quantile of asset $i$.

The function clamps each entry of the symmetric result to $[\sqrt{\varepsilon},\, 1]$, where $\varepsilon$ is the machine epsilon of eltype(X). The floor is not zero. A pair whose tails never coincide gets 1.4901161193847656e-8 for a Float64 input, so a caller can divide by an entry or take its logarithm.

Warning

The estimate is biased, not only noisy, and the bias decreases slowly as alpha decreases. At a fixed alpha, the joint count over $k$ converges to $C(\alpha,\, \alpha) / \alpha$ as $T$ grows, where $C$ is the copula of the pair. That ratio is the conditional probability at the level alpha, and not its limit $\lambda_L$ at zero. The multivariate normal distribution is lower tail independent, so a Gaussian copula with a correlation $\rho < 1$ has $\lambda_L = 0$. But its ratio is far from zero at the levels that a sample of returns can support:

$\rho$alpha = 0.05alpha = 0.01alpha = 0.001
0.30.1430.0560.015
0.60.3100.1880.095
0.80.4950.3770.263

A smaller alpha decreases the bias, but it also decreases $k$, so the variance of the estimate increases. The choice of $k$ is thus a variance-bias problem, and the finite-sample bias increases with the correlation. Do not read a positive estimate as evidence of tail dependence. A Gaussian sample with correlation 0.6 gives about 0.31 at alpha = 0.05. To compare a model with an estimate, compare the ratio of the model at the same alpha, not its limit.

Algorithm

  1. Set $k = \lceil T \alpha \rceil$, the number of observations in the lower tail. Allocate the result as a matrix of zeros. The allocation is zeros and not undef, so the caller gets zeros and not uninitialised memory when k is not positive.
  2. If k is not positive, return the zero matrix. The # Validation section states the cases that get to this branch.
  3. Copy X. Partially sort each column of the copy about position k. Position k of the sorted column is then the $k$-th order statistic, which is the empirical $\alpha$-quantile of that asset.
  4. Fill the boolean mask one column at a time. Entry (t, j) is true when $x_{tj}$ is at or below the quantile of step 3. The method fills all of the mask before step 5. Iteration j of the threaded loop of step 5 reads each column i <= j, so a mask that the loop fills itself can be incomplete when an iteration reads it.
  5. For each pair $(i, j)$ with $i \leq j$, count the observations at which the two mask entries are true. Divide the count by k, and clamp the ratio to $[\sqrt{\varepsilon},\, 1]$. Write the clamped value into rho[i, j] and into rho[j, i]. The executor ex runs this loop.

Arguments

  • X: The data matrix of asset returns, observations × assets.
  • alpha: The quantile level of the lower tail.
  • ex: The parallel execution strategy.

Validation

  • ceil(Int, T * alpha) > 0. If the count is not positive, the function returns a zero matrix and not an estimate. This happens when alpha <= 0 or when X has no observation. A small positive alpha does not get to this branch, because ceil of a positive real is at least 1. So alpha = nextfloat(0.0) on a matrix of one or more observations selects one observation.
  • ceil(Int, T * alpha) <= T. The function does not check it. An alpha > 1 makes k larger than T, and partialsort! raises a BoundsError.
  • LowerTailDependenceCovariance refuses an alpha outside $(0, 1)$ at construction, because assert_unit_interval requires 0 < alpha < 1. So only a direct call gets to the two cases above.

Returns

  • rho::Matrix{<:Number}: The symmetric matrix of lower tail dependence coefficients. rho[i, j] is the estimate for assets i and j. If k is positive, the diagonal is exactly 1 and each entry is in $[\sqrt{\varepsilon},\, 1]$. If k is not positive, each entry is 0.

Related

References

  • [40] R. Schmidt and U. Stadtmüller. Non-parametric estimation of tail dependence. Scandinavian Journal of Statistics 33, 307–335 (2006). Section 2, the tail independence of the multivariate normal distribution. Section 4, the variance-bias problem of the choice of $k$. Section 7.1, Table 1, the finite-sample bias.
source
PortfolioOptimisers.show_fields — Method
show_fields(
    _::LowerTailDependenceCovariance
) -> Tuple{Symbol, Symbol, Symbol}

Renders every field of a LowerTailDependenceCovariance except cache.

A cache holds the state of an incremental fit. That state is not part of the configuration, and it prints under the estimator at each site that renders the estimator. Set set_show_nothing_fields!(:LowerTailDependenceCovariance, true) to render it.

Arguments

  • ::LowerTailDependenceCovariance: The lower tail dependence covariance estimator. The method reads only its type.

Returns

  • fields::Tuple: The names of the fields to render, (:ve, :alpha, :ex).

Related

source

References

[40]
R. Schmidt and U. Stadtmüller. Non-parametric estimation of tail dependence. Scandinavian Journal of Statistics 33, 307–335 (2006).