Gerber covariance

The Gerber statistic measures how often two assets move together. An observation counts only when a return moves past a threshold, and it counts as one vote whatever the size of the move. Small moves, which are mostly noise, do not count, and one extreme move cannot dominate the result. The statistic extends Kendall's tau. It compares the votes where the two assets move past their thresholds in the same direction with the votes where they move in opposite directions [33].

Three variants are published, and the library has all three, Gerber0, Gerber1 and Gerber2. Gerber0 and Gerber1 divide the net vote by different counts, and Gerber2 normalises the whole matrix, so the three give different estimates from the same data [34].

Abstract Gerber covariance types

Subtype these to write a new Gerber covariance estimator or algorithm.

PortfolioOptimisers.BaseGerberCovariance — Type
abstract type BaseGerberCovariance <: AbstractCovarianceEstimator

Abstract supertype for all Gerber covariance estimators.

Every concrete or abstract type that implements a Gerber covariance estimator must subtype BaseGerberCovariance.

Section 2 of [33] defines the statistic. Section 2 of [34] compares its three denominators, and each GerberCovarianceAlgorithm subtype selects one of them.

Interfaces

A new subtype must follow the interface of AbstractCovarianceEstimator.

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.
  • [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2.
source
PortfolioOptimisers.GerberCovarianceAlgorithm — Type
abstract type GerberCovarianceAlgorithm <: AbstractAlgorithm

Abstract supertype for all Gerber covariance algorithm types.

Every concrete or abstract type that implements a Gerber covariance algorithm must subtype GerberCovarianceAlgorithm. A subtype goes in the alg field of a GerberCovariance estimator. It selects the denominator that comovement_ratio puts under the net co-movement count. Each method of gerber dispatches on one subtype, so a new subtype needs a new method of gerber.

[33] states the three denominators:

[33] is the final version of two earlier preprints, as its footnote 1 states. Section 2 of [34] compares the three denominators. It attributes the first to the first preprint, the third to the 2019 preprint and the second to the final version.

Interfaces

The estimator that holds a new subtype must follow the interface of AbstractCovarianceEstimator.

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Footnote 1, Section 2.1, Equation 4, Section 2.2, Equation 11, and footnote 3.
  • [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2.
source

Gerber covariance estimators and algorithms

This section has the three variants of the statistic, and the estimator that takes one of them.

PortfolioOptimisers.Gerber0 — Type
struct Gerber0 <: GerberCovarianceAlgorithm

Counts only the observations on which both assets cross their thresholds, as the original Gerber statistic does. The denominator is the number of these observations.

Mathematical definition

\[\begin{align} \rho_{i,\,j} &= \frac{n_{c} - n_{d}}{n_{c} + n_{d}}\,. \end{align}\]

Where:

  • $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
  • $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
  • $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.

An observation on which exactly one asset crosses does not change the statistic. When the two assets never cross their thresholds on the same observation, the denominator is zero and the statistic is zero.

This is Equation 4 of [33], which identifies it with Kendall's tau when the thresholds are zero. Section 2 of [34] dates it to the first preprint of the work. [33] is the final version of that preprint, as its footnote 1 states. On a random sample of four assets, Equation 4 gives the entries of the library to the last digit.

Algorithm

The branch of comovement_ratio that this tag selects runs these steps.

  1. Add the concordant and discordant counts into the denominator den.
  2. Return zero(T) when den is zero. An asset that never crosses its own threshold gives a zero den for every pair that contains it.
  3. Otherwise return (p - n) / den.

Constructors

Gerber0() -> Gerber0

Examples

julia> Gerber0()Gerber0()

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.1, Equations 3 and 4, and footnote 1.
  • [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2.
source
PortfolioOptimisers.Gerber1 — Type
struct Gerber1 <: GerberCovarianceAlgorithm

Adds the observations on which exactly one asset crosses its threshold to the denominator of the Gerber statistic.

Mathematical definition

\[\begin{align} \rho_{i,\,j} &= \frac{n_{c} - n_{d}}{n_{c} + n_{d} + n_{n}}\,. \end{align}\]

Where:

  • $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
  • $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
  • $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.
  • $n_{n}$: Neutral count of a pair, the observations on which exactly one of the two assets crossed its threshold.

The denominator is that of Gerber0 plus $n_{n}$. It is never smaller, so the magnitude of the statistic is never larger. The two statistics are equal when $n_{n} = 0$.

The sum $n_{c} + n_{d} + n_{n}$ is the number of observations on which at least one asset crosses. It equals $T - n^{NN}$, where $n^{NN}$ is the number of observations on which neither asset crosses. Equation 11 of [33] states the denominator in that form, and Equation 2 of [34] repeats it. Both identify it as the statistic of the final version of the work.

Algorithm

The branch of comovement_ratio that this tag selects runs these steps.

  1. Add the concordant, discordant and neutral counts into the denominator den.
  2. Return zero(T) when den is zero. An asset that never crosses its own threshold gives a zero den for every pair that contains it.
  3. Otherwise return (p - n) / den.

Constructors

Gerber1() -> Gerber1

Examples

julia> Gerber1()Gerber1()

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.2, Equation 11.
  • [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2, Equation 2.
source
PortfolioOptimisers.Gerber2 — Type
struct Gerber2 <: GerberCovarianceAlgorithm

Divides each net co-movement count by the geometric mean of the crossing counts of the two assets.

Mathematical definition

\[\begin{align} h_{i,\,j} &= n_{c} - n_{d}\,, \\ \rho_{i,\,j} &= \frac{h_{i,\,j}}{\sqrt{h_{i,\,i} \, h_{j,\,j}}}\,. \end{align}\]

Where:

  • $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
  • $h_{i,\,j}$: Net co-movement count of the pair, before the division.
  • $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
  • $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.

The denominator comes from the diagonal of the whole matrix, not from a count of the pair alone. Gerber0 and Gerber1 divide by a count of the pair. An asset moves in the same direction as itself at each of its crossings, so $h_{i,\,i}$ is the number of crossings of asset $i$. The diagonal of the result is therefore one by construction.

The denominator is the geometric mean of the number of observations on which asset $i$ crosses and the number on which asset $j$ crosses. Footnote 3 of [33] states this denominator as the alternative of the 2019 preprint, and states that the covariance is positive semidefinite. The footnote misprints one count of its second factor. Equation 1 of [34] states the correct count and gives the same attribution. [33] is the final version of that preprint, as its footnote 1 states. On a random sample of four assets, the footnote with the correct count gives the entries of the library to within 2.3e-16.

Algorithm

The branch of comovement_ratio and of standardise_comovement! that this tag selects runs these steps.

  1. Return the difference p - n for every pair, with no denominator.
  2. Divide the assembled matrix by the outer product of the square roots of its own diagonal. standardise_comovement! clamps each root from below at sqrt(eps(eltype(rho))), so an asset that never crosses gives a zero row and no division by zero.

Constructors

Gerber2() -> Gerber2

Examples

julia> Gerber2()Gerber2()

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.2, footnote 3, and footnote 1.
  • [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2, Equation 1.
source
PortfolioOptimisers.GerberCovariance — Type
struct GerberCovariance{__T_ve, __T_me, __T_pdm, __T_t, __T_alg, __T_cache} <: BaseGerberCovariance

Estimates a covariance matrix from how often the two assets of each pair cross their thresholds together.

Each entry of a Gerber matrix is a separate statistic of one pair, so the matrix is not positive definite in general. Equation 4 of [33] can give a matrix that is not positive semidefinite. The pdm field is the repair of the library, not of the source. It projects the result onto the nearest positive definite matrix, and pdm = nothing returns the statistic with no repair.

The threshold t is the fraction $c$ of Equation 2 of [33]. An asset crosses when its centred return is at least t standard deviations from zero. Section 2.1 of [33] calls 1/2 typical and names 7/10 and 9/10 as alternatives, and the default t = 0.5 follows it. The covariance is the correlation scaled by the standard deviations, as in Equation 9.

The library compares the centred return with the threshold. Equation 1 of [33] compares the return itself with the threshold. Footnote 4 of [34] states that the threshold is centred about zero, not about the mean of each asset. This estimator subtracts the mean that me computes before it compares. On a sample of 200 observations of five assets with unit standard deviation and a mean of 0.1, the entries differ from the uncentred statistic by up to 0.06 under Gerber0, 0.03 under Gerber1 and 0.04 under Gerber2. me = CustomValueExpectedReturns(; val = 0.0) gives the statistic of the source.

Fields

  • ve: Variance estimator.
  • me: Expected returns estimator. The estimator subtracts its mean from the returns before the threshold test.
  • pdm: Positive definite matrix estimator.
  • t: Threshold value.
  • alg: Gerber covariance algorithm.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it. factory keeps it unchanged, because a factory call resolves the configuration, not the sample. port_opt_view copies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone. obs_weights_view removes it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.

Constructors

GerberCovariance(;    ve::StatsBase.CovarianceEstimator = SimpleVariance(),    me::AbstractExpectedReturnsEstimator = SimpleExpectedReturns(),    pdm::Option{<:AbstractPosdefEstimator} = Posdef(),    t::Number = 0.5,    alg::GerberCovarianceAlgorithm = Gerber1(),    cache::Option{<:AbstractPartialFitState} = nothing) -> GerberCovariance

Keywords correspond to the struct's fields.

Validation

  • 0 <= t.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • ve: Recursively updated through factory.
  • me: Recursively updated through factory.
  • alg: Recursively updated through factory.
  • cache: Carried unchanged through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Examples

julia> GerberCovariance()GerberCovariance   ve ┼ SimpleVariance      │          me ┼ SimpleExpectedReturns      │             │   w ┴ nothing      │           w ┼ nothing      │   corrected ┴ Bool: true   me ┼ SimpleExpectedReturns      │   w ┴ nothing  pdm ┼ Posdef      │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton      │   kwargs ┴ @NamedTuple{}: NamedTuple()    t ┼ Float64: 0.5  alg ┴ Gerber1()

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2, Equations 1, 2 and 9.
  • [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Footnote 4.
source
Statistics.cov — Method
Statistics.cov(
    ce::GerberCovariance,
    X::MatNum;
    dims::Int = 1,
    kwargs...
) -> MatNum

Compute the Gerber covariance matrix with the algorithm in ce.alg.

Mathematical definition

\[\begin{align} \hat{\mathbf{\Sigma}} &= \mathrm{Diag}(\boldsymbol{\sigma}) \, \hat{\boldsymbol{\rho}} \, \mathrm{Diag}(\boldsymbol{\sigma})\,. \end{align}\]

Where:

  • $\hat{\mathbf{\Sigma}}$: Gerber covariance matrix.
  • $\hat{\boldsymbol{\rho}}$: Gerber correlation matrix of the same estimator.
  • $\boldsymbol{\sigma}$: Standard deviation vector of the assets.

The Gerber statistic gives only the correlations. The variances come from ce.ve, so the diagonal of $\hat{\mathbf{\Sigma}}$ is $\boldsymbol{\sigma}^2$. This is Equation 9 of [33], with the estimator ce.ve in place of the sample standard deviation.

Algorithm

  1. Orient X to observations × assets with dims_oriented.
  2. Compute the standard deviation vector sd with ce.ve, and raise each entry to at least eps(eltype(sd)). A constant column has a standard deviation and a demeaned residual of the same round-off order. Without the floor, the threshold marks every observation of that column as a crossing. With the floor, it marks none of them.
  3. Centre the returns with ce.me through demean_returns.
  4. Compute the Gerber correlation matrix sigma with gerber, through the branch that ce.alg selects.
  5. Rescale sigma in place to a covariance matrix with StatsBase.cor2cov! and sd.

Arguments

  • ce: Gerber covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keyword arguments for the standard deviation estimator and for demean_returns.

Validation

  • dims in (1, 2).

Returns

  • sigma::MatNum: Covariance matrix assets x assets.

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.2, Equation 9.
source
Statistics.cor — Method
Statistics.cor(
    ce::GerberCovariance,
    X::MatNum;
    dims::Int = 1,
    kwargs...
) -> MatNum

Compute the Gerber correlation matrix with the algorithm in ce.alg.

Algorithm

  1. Orient X to observations × assets with dims_oriented.
  2. Compute the standard deviation vector sd with ce.ve, and raise each entry to at least eps(eltype(sd)). A constant column has a standard deviation and a demeaned residual of the same round-off order. Without the floor, the threshold marks every observation of that column as a crossing. With the floor, it marks none of them.
  3. Centre the returns with ce.me through demean_returns.
  4. Return the Gerber correlation matrix from gerber, through the branch that ce.alg selects.

Arguments

  • ce: Gerber covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keyword arguments for the standard deviation estimator and for demean_returns.

Validation

  • dims in (1, 2).

Returns

  • rho::MatNum: Correlation matrix assets x assets.

Related

References

  • [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.
source

References

[33]
S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021).
[34]
E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023).