Gerber covariance
The Gerber statistic measures how often two assets move together. An observation counts only when a return moves past a threshold, and it counts as one vote whatever the size of the move. Small moves, which are mostly noise, do not count, and one extreme move cannot dominate the result. The statistic extends Kendall's tau. It compares the votes where the two assets move past their thresholds in the same direction with the votes where they move in opposite directions [33].
Three variants are published, and the library has all three, Gerber0, Gerber1 and Gerber2. Gerber0 and Gerber1 divide the net vote by different counts, and Gerber2 normalises the whole matrix, so the three give different estimates from the same data [34].
Abstract Gerber covariance types
Subtype these to write a new Gerber covariance estimator or algorithm.
PortfolioOptimisers.BaseGerberCovariance — Type
abstract type BaseGerberCovariance <: AbstractCovarianceEstimatorAbstract supertype for all Gerber covariance estimators.
Every concrete or abstract type that implements a Gerber covariance estimator must subtype BaseGerberCovariance.
Section 2 of [33] defines the statistic. Section 2 of [34] compares its three denominators, and each GerberCovarianceAlgorithm subtype selects one of them.
Interfaces
A new subtype must follow the interface of AbstractCovarianceEstimator.
Related
GerberCovarianceGerberCovarianceAlgorithmBaseSmythBrobyCovariance: A subtype that gives each crossing a weight instead of a count of one.BaseGerberIQCovariance: A subtype that scales the threshold for each pair and gives older observations less weight.
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.
- [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2.
PortfolioOptimisers.GerberCovarianceAlgorithm — Type
abstract type GerberCovarianceAlgorithm <: AbstractAlgorithmAbstract supertype for all Gerber covariance algorithm types.
Every concrete or abstract type that implements a Gerber covariance algorithm must subtype GerberCovarianceAlgorithm. A subtype goes in the alg field of a GerberCovariance estimator. It selects the denominator that comovement_ratio puts under the net co-movement count. Each method of gerber dispatches on one subtype, so a new subtype needs a new method of gerber.
[33] states the three denominators:
[33] is the final version of two earlier preprints, as its footnote 1 states. Section 2 of [34] compares the three denominators. It attributes the first to the first preprint, the third to the 2019 preprint and the second to the final version.
Interfaces
The estimator that holds a new subtype must follow the interface of AbstractCovarianceEstimator.
Related
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Footnote 1, Section 2.1, Equation 4, Section 2.2, Equation 11, and footnote 3.
- [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2.
Gerber covariance estimators and algorithms
This section has the three variants of the statistic, and the estimator that takes one of them.
PortfolioOptimisers.Gerber0 — Type
struct Gerber0 <: GerberCovarianceAlgorithmCounts only the observations on which both assets cross their thresholds, as the original Gerber statistic does. The denominator is the number of these observations.
Mathematical definition
\[\begin{align} \rho_{i,\,j} &= \frac{n_{c} - n_{d}}{n_{c} + n_{d}}\,. \end{align}\]
Where:
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
- $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
- $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.
An observation on which exactly one asset crosses does not change the statistic. When the two assets never cross their thresholds on the same observation, the denominator is zero and the statistic is zero.
This is Equation 4 of [33], which identifies it with Kendall's tau when the thresholds are zero. Section 2 of [34] dates it to the first preprint of the work. [33] is the final version of that preprint, as its footnote 1 states. On a random sample of four assets, Equation 4 gives the entries of the library to the last digit.
Algorithm
The branch of comovement_ratio that this tag selects runs these steps.
- Add the concordant and discordant counts into the denominator
den. - Return
zero(T)whendenis zero. An asset that never crosses its own threshold gives a zerodenfor every pair that contains it. - Otherwise return
(p - n) / den.
Constructors
Gerber0() -> Gerber0Examples
julia> Gerber0()Gerber0()Related
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.1, Equations 3 and 4, and footnote 1.
- [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2.
PortfolioOptimisers.Gerber1 — Type
struct Gerber1 <: GerberCovarianceAlgorithmAdds the observations on which exactly one asset crosses its threshold to the denominator of the Gerber statistic.
Mathematical definition
\[\begin{align} \rho_{i,\,j} &= \frac{n_{c} - n_{d}}{n_{c} + n_{d} + n_{n}}\,. \end{align}\]
Where:
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
- $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
- $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.
- $n_{n}$: Neutral count of a pair, the observations on which exactly one of the two assets crossed its threshold.
The denominator is that of Gerber0 plus $n_{n}$. It is never smaller, so the magnitude of the statistic is never larger. The two statistics are equal when $n_{n} = 0$.
The sum $n_{c} + n_{d} + n_{n}$ is the number of observations on which at least one asset crosses. It equals $T - n^{NN}$, where $n^{NN}$ is the number of observations on which neither asset crosses. Equation 11 of [33] states the denominator in that form, and Equation 2 of [34] repeats it. Both identify it as the statistic of the final version of the work.
Algorithm
The branch of comovement_ratio that this tag selects runs these steps.
- Add the concordant, discordant and neutral counts into the denominator
den. - Return
zero(T)whendenis zero. An asset that never crosses its own threshold gives a zerodenfor every pair that contains it. - Otherwise return
(p - n) / den.
Constructors
Gerber1() -> Gerber1Examples
julia> Gerber1()Gerber1()Related
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.2, Equation 11.
- [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2, Equation 2.
PortfolioOptimisers.Gerber2 — Type
struct Gerber2 <: GerberCovarianceAlgorithmDivides each net co-movement count by the geometric mean of the crossing counts of the two assets.
Mathematical definition
\[\begin{align} h_{i,\,j} &= n_{c} - n_{d}\,, \\ \rho_{i,\,j} &= \frac{h_{i,\,j}}{\sqrt{h_{i,\,i} \, h_{j,\,j}}}\,. \end{align}\]
Where:
- $\rho_{i,\,j}$: Pairwise correlation coefficient between assets $i$ and $j$.
- $h_{i,\,j}$: Net co-movement count of the pair, before the division.
- $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
- $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.
The denominator comes from the diagonal of the whole matrix, not from a count of the pair alone. Gerber0 and Gerber1 divide by a count of the pair. An asset moves in the same direction as itself at each of its crossings, so $h_{i,\,i}$ is the number of crossings of asset $i$. The diagonal of the result is therefore one by construction.
The denominator is the geometric mean of the number of observations on which asset $i$ crosses and the number on which asset $j$ crosses. Footnote 3 of [33] states this denominator as the alternative of the 2019 preprint, and states that the covariance is positive semidefinite. The footnote misprints one count of its second factor. Equation 1 of [34] states the correct count and gives the same attribution. [33] is the final version of that preprint, as its footnote 1 states. On a random sample of four assets, the footnote with the correct count gives the entries of the library to within 2.3e-16.
Algorithm
The branch of comovement_ratio and of standardise_comovement! that this tag selects runs these steps.
- Return the difference
p - nfor every pair, with no denominator. - Divide the assembled matrix by the outer product of the square roots of its own diagonal.
standardise_comovement!clamps each root from below atsqrt(eps(eltype(rho))), so an asset that never crosses gives a zero row and no division by zero.
Constructors
Gerber2() -> Gerber2Examples
julia> Gerber2()Gerber2()Related
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.2, footnote 3, and footnote 1.
- [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Section 2, Equation 1.
PortfolioOptimisers.GerberCovariance — Type
struct GerberCovariance{__T_ve, __T_me, __T_pdm, __T_t, __T_alg, __T_cache} <: BaseGerberCovarianceEstimates a covariance matrix from how often the two assets of each pair cross their thresholds together.
Each entry of a Gerber matrix is a separate statistic of one pair, so the matrix is not positive definite in general. Equation 4 of [33] can give a matrix that is not positive semidefinite. The pdm field is the repair of the library, not of the source. It projects the result onto the nearest positive definite matrix, and pdm = nothing returns the statistic with no repair.
The threshold t is the fraction $c$ of Equation 2 of [33]. An asset crosses when its centred return is at least t standard deviations from zero. Section 2.1 of [33] calls 1/2 typical and names 7/10 and 9/10 as alternatives, and the default t = 0.5 follows it. The covariance is the correlation scaled by the standard deviations, as in Equation 9.
The library compares the centred return with the threshold. Equation 1 of [33] compares the return itself with the threshold. Footnote 4 of [34] states that the threshold is centred about zero, not about the mean of each asset. This estimator subtracts the mean that me computes before it compares. On a sample of 200 observations of five assets with unit standard deviation and a mean of 0.1, the entries differ from the uncentred statistic by up to 0.06 under Gerber0, 0.03 under Gerber1 and 0.04 under Gerber2. me = CustomValueExpectedReturns(; val = 0.0) gives the statistic of the source.
Fields
ve: Variance estimator.
me: Expected returns estimator. The estimator subtracts its mean from the returns before the threshold test.
pdm: Positive definite matrix estimator.
t: Threshold value.
alg: Gerber covariance algorithm.
cache: Optional partial-fit state. It isnothinguntilpartial_fit!writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it.factorykeeps it unchanged, because a factory call resolves the configuration, not the sample.port_opt_viewcopies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone.obs_weights_viewremoves it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.
Constructors
GerberCovariance(; ve::StatsBase.CovarianceEstimator = SimpleVariance(), me::AbstractExpectedReturnsEstimator = SimpleExpectedReturns(), pdm::Option{<:AbstractPosdefEstimator} = Posdef(), t::Number = 0.5, alg::GerberCovarianceAlgorithm = Gerber1(), cache::Option{<:AbstractPartialFitState} = nothing) -> GerberCovarianceKeywords correspond to the struct's fields.
Validation
0 <= t.
Propagated parameters
factory passes its arguments on to these @fprop fields:
ve: Recursively updated throughfactory.me: Recursively updated throughfactory.alg: Recursively updated throughfactory.cache: Carried unchanged throughfactory.
View parameters
port_opt_view takes the view of these @vprop fields at the selected indices:
ve: Recursively viewed throughport_opt_view.me: Recursively viewed throughport_opt_view.cache: Sliced to the selected assets throughport_opt_view.
Examples
julia> GerberCovariance()GerberCovariance ve ┼ SimpleVariance │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ w ┼ nothing │ corrected ┴ Bool: true me ┼ SimpleExpectedReturns │ w ┴ nothing pdm ┼ Posdef │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ kwargs ┴ @NamedTuple{}: NamedTuple() t ┼ Float64: 0.5 alg ┴ Gerber1()Related
BaseGerberCovarianceGerberCovarianceAlgorithmStatsBase.CovarianceEstimatorSimpleVariancePosdefGerber0Gerber1Gerber2factoryport_opt_view
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2, Equations 1, 2 and 9.
- [34] E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023). Footnote 4.
Statistics.cov — Method
Statistics.cov(
ce::GerberCovariance,
X::MatNum;
dims::Int = 1,
kwargs...
) -> MatNumCompute the Gerber covariance matrix with the algorithm in ce.alg.
Mathematical definition
\[\begin{align} \hat{\mathbf{\Sigma}} &= \mathrm{Diag}(\boldsymbol{\sigma}) \, \hat{\boldsymbol{\rho}} \, \mathrm{Diag}(\boldsymbol{\sigma})\,. \end{align}\]
Where:
- $\hat{\mathbf{\Sigma}}$: Gerber covariance matrix.
- $\hat{\boldsymbol{\rho}}$: Gerber correlation matrix of the same estimator.
- $\boldsymbol{\sigma}$: Standard deviation vector of the assets.
The Gerber statistic gives only the correlations. The variances come from ce.ve, so the diagonal of $\hat{\mathbf{\Sigma}}$ is $\boldsymbol{\sigma}^2$. This is Equation 9 of [33], with the estimator ce.ve in place of the sample standard deviation.
Algorithm
- Orient
Xtoobservations × assetswithdims_oriented. - Compute the standard deviation vector
sdwithce.ve, and raise each entry to at leasteps(eltype(sd)). A constant column has a standard deviation and a demeaned residual of the same round-off order. Without the floor, the threshold marks every observation of that column as a crossing. With the floor, it marks none of them. - Centre the returns with
ce.methroughdemean_returns. - Compute the Gerber correlation matrix
sigmawithgerber, through the branch thatce.algselects. - Rescale
sigmain place to a covariance matrix withStatsBase.cor2cov!andsd.
Arguments
ce: Gerber covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments for the standard deviation estimator and fordemean_returns.
Validation
dims in (1, 2).
Returns
sigma::MatNum: Covariance matrixassets x assets.
Related
GerberCovariancegerber(ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber0}, X::MatNum, sd::ArrNum)gerber(ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber1}, X::MatNum, sd::ArrNum)gerber(ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber2}, X::MatNum, sd::ArrNum)demean_returnscor(ce::GerberCovariance, X::MatNum; dims::Int = 1, kwargs...)
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.2, Equation 9.
Statistics.cor — Method
Statistics.cor(
ce::GerberCovariance,
X::MatNum;
dims::Int = 1,
kwargs...
) -> MatNumCompute the Gerber correlation matrix with the algorithm in ce.alg.
Algorithm
- Orient
Xtoobservations × assetswithdims_oriented. - Compute the standard deviation vector
sdwithce.ve, and raise each entry to at leasteps(eltype(sd)). A constant column has a standard deviation and a demeaned residual of the same round-off order. Without the floor, the threshold marks every observation of that column as a crossing. With the floor, it marks none of them. - Centre the returns with
ce.methroughdemean_returns. - Return the Gerber correlation matrix from
gerber, through the branch thatce.algselects.
Arguments
ce: Gerber covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.dims: Dimension along which the computation runs.kwargs...: Keyword arguments for the standard deviation estimator and fordemean_returns.
Validation
dims in (1, 2).
Returns
rho::MatNum: Correlation matrixassets x assets.
Related
GerberCovariancegerber(ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber0}, X::MatNum, sd::ArrNum)gerber(ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber1}, X::MatNum, sd::ArrNum)gerber(ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber2}, X::MatNum, sd::ArrNum)demean_returnscov(ce::GerberCovariance, X::MatNum; dims::Int = 1, kwargs...)
References
- [33] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021). Section 2.
References
- [33]
- S. Gerber, H. Markowitz, P. Ernst, Y. Miao, B. Javid and P. Sargen. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021).
- [34]
- E. Flint and D. Polakow. Deconstructing the Gerber statistic. Finance Research Letters 56, 104144 (2023).