Gerber covariance: private API

gerber_updown(
    ce::GerberCovariance,
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    sd::AbstractArray{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}
) -> Tuple{Matrix{Bool}, Matrix{Bool}}

Build the up and down indicator matrices shared by every Gerber correlation variant.

Mathematical definition

\[\begin{align} U_{t,\,i} &= \mathbf{1}[x_{t,\,i} \geq t \, \sigma_i \land x_{t,\,i} > 0]\,, \\ D_{t,\,i} &= \mathbf{1}[x_{t,\,i} \leq -t \, \sigma_i \land x_{t,\,i} < 0]\,. \end{align}\]

Where:

• $\mathbf{U} \in \{0,1\}^{T \times N}$: Up indicator matrix, $U_{t,\,i} = \mathbf{1}[x_{t,\,i} \geq t \, \sigma_i \land x_{t,\,i} > 0]$.
• $\mathbf{D} \in \{0,1\}^{T \times N}$: Down indicator matrix, $D_{t,\,i} = \mathbf{1}[x_{t,\,i} \leq -t \, \sigma_i \land x_{t,\,i} < 0]$.
• $x_{t,\,i}$: Return of asset $i$ at observation $t$.
• $t$: Threshold parameter, read as a standalone symbol; a subscript $t$ is the observation index. An asset crosses at an observation when its return is at least $t$ of its own standard deviations away from zero, and a return of exactly zero never crosses.
• $\sigma_i$: Standard deviation of asset $i$.
• $T$: Number of observations.
• $N$: Number of assets.

The two bands never overlap, so no observation is marked in both matrices and their sum is the crossing indicator. The sign test is what keeps them apart at a zero band edge. ce.t = 0 produces one, and so does a zero entry of sd, which the Statistics.cor method rules out by raising sd to at least eps. Without the sign test the two closed comparisons both hold on an exactly zero return, and the sum is two rather than one.

For a positive band edge the sign test is redundant, because $x_{t,\,i} \geq t \, \sigma_i > 0$ already implies $x_{t,\,i} > 0$. A zero band edge therefore makes the pair of matrices the sign of the return, and the Gerber statistic the sign concordance.

Algorithm

1. Scale the standard deviation vector by the threshold, giving the per-asset band edge ts = sd * ce.t.
2. Mark U[t, i] when X[t, i] >= ts[i] and X[t, i] is positive.
3. Mark D[t, i] when X[t, i] <= -ts[i] and X[t, i] is negative.

Arguments

• ce: Gerber covariance estimator.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• sd: Standard deviation vector of X, shaped to be consistent with X.

Returns

• (U, D)::Tuple{Matrix{Bool}, Matrix{Bool}}: The up and the down indicator matrices.

Related

• GerberCovariance
• concordance_counts

concordance_counts(
    pmn::AbstractMatrix,
    ppn::AbstractMatrix
) -> Tuple{Any, Any}

Split the concordant and discordant co-movement counts out of their difference and their sum.

A matrix product delivers the difference and the sum directly, so the two counts are recovered from those instead of by two more matrix products. The split is exact, and the reduction in comovement_ratio sees the same numerator and denominator as the matrix formula.

Mathematical definition

\[\begin{align} n_{c} &= \frac{(n_{c} + n_{d}) + (n_{c} - n_{d})}{2}\,, \\ n_{d} &= \frac{(n_{c} + n_{d}) - (n_{c} - n_{d})}{2}\,. \end{align}\]

Where:

• $n_{c}$: Concordant count of a pair, the observations on which both assets crossed their thresholds in the same direction.
• $n_{d}$: Discordant count of a pair, the observations on which both assets crossed their thresholds in opposite directions.

Arguments

• pmn::AbstractMatrix: The difference nconc - ndisc.
• ppn::AbstractMatrix: The sum nconc + ndisc.

Returns

• (nconc, ndisc)::Tuple{AbstractMatrix, AbstractMatrix}: The concordant and the discordant counts.

Related

• gerber_updown
• comovement_ratio

gerber(
    ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber0},
    X::MatNum,
    sd::ArrNum
) -> MatNum

Implements the original Gerber correlation algorithm.

Mathematical definition

\[\begin{align} \hat{\boldsymbol{\rho}} &= \left(\mathbf{H}^\intercal \mathbf{H}\right) \oslash \left(\mathbf{V}^\intercal \mathbf{V}\right)\,. \end{align}\]

Where:

• $\mathbf{U} \in \{0,1\}^{T \times N}$: Up indicator matrix, $U_{t,\,i} = \mathbf{1}[x_{t,\,i} \geq t \, \sigma_i \land x_{t,\,i} > 0]$.
• $\mathbf{D} \in \{0,1\}^{T \times N}$: Down indicator matrix, $D_{t,\,i} = \mathbf{1}[x_{t,\,i} \leq -t \, \sigma_i \land x_{t,\,i} < 0]$.
• $\mathbf{H} = \mathbf{U} - \mathbf{D}$: Signed crossing matrix. Its entry is $1$ when the asset crossed upwards, $-1$ when it crossed downwards, and $0$ when it did not cross.
• $\mathbf{V} = \mathbf{U} + \mathbf{D}$: Crossing matrix. Its entry is $1$ when the asset crossed its threshold in either direction, and $0$ when it did not.
• $x_{t,\,i}$: Return of asset $i$ at observation $t$.
• $t$: Threshold parameter, read as a standalone symbol; a subscript $t$ is the observation index. An asset crosses at an observation when its return is at least $t$ of its own standard deviations away from zero, and a return of exactly zero never crosses.
• $\sigma_i$: Standard deviation of asset $i$.
• $T$: Number of observations.
• $N$: Number of assets.
• $\oslash$: Element-wise division.

The entry of $\mathbf{H}^\intercal \mathbf{H}$ is $n_{c} - n_{d}$ and the entry of $\mathbf{V}^\intercal \mathbf{V}$ is $n_{c} + n_{d}$, so this is the pairwise statistic of Gerber0 written over the whole matrix.

Algorithm

1. Build the indicator matrices U and D with gerber_updown.
2. Form the signed crossing matrix UmD = U - D and the crossing matrix UpD = U + D.
3. Recover the concordant count nconc and the discordant count ndisc from transpose(UmD) * UmD and transpose(UpD) * UpD with concordance_counts.
4. Reduce every pair with comovement_ratio, giving rho.
5. Write one onto a zero diagonal entry of rho with comovement_unit_diagonal!. An asset that crosses no threshold reduces to a zero diagonal entry, and that entry is one by definition.
6. Repair rho with posdef!, which is a no-op when ce.pdm is nothing.

Arguments

• ce: Gerber covariance estimator.. Configured with the Gerber0 algorithm.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• sd: Standard deviation vector of X, shaped to be consistent with X.

Returns

• rho::MatNum: Correlation matrix assets x assets.

Related

• GerberCovariance
• Gerber0
• gerber_updown
• concordance_counts
• comovement_ratio
• comovement_unit_diagonal!
• posdef!

References

• [12] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, P. Sargen and others. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021).

gerber(
    ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber1},
    X::MatNum,
    sd::ArrNum
) -> MatNum

Implements the first variant of the Gerber correlation algorithm.

Mathematical definition

\[\begin{align} \hat{\boldsymbol{\rho}} &= \left(\mathbf{H}^\intercal \mathbf{H}\right) \oslash \left(T \boldsymbol{1}\boldsymbol{1}^\intercal - \mathbf{N}^\intercal \mathbf{N}\right)\,. \end{align}\]

Where:

• $\mathbf{U} \in \{0,1\}^{T \times N}$: Up indicator matrix, $U_{t,\,i} = \mathbf{1}[x_{t,\,i} \geq t \, \sigma_i \land x_{t,\,i} > 0]$.
• $\mathbf{D} \in \{0,1\}^{T \times N}$: Down indicator matrix, $D_{t,\,i} = \mathbf{1}[x_{t,\,i} \leq -t \, \sigma_i \land x_{t,\,i} < 0]$.
• $\mathbf{N} \in \{0,1\}^{T \times N}$: Neutral indicator matrix, $N_{t,\,i} = \mathbf{1}[\lvert x_{t,\,i} \rvert < t \, \sigma_i \lor x_{t,\,i} = 0]$. It is the complement of $\mathbf{U} + \mathbf{D}$.
• $\mathbf{H} = \mathbf{U} - \mathbf{D}$: Signed crossing matrix. Its entry is $1$ when the asset crossed upwards, $-1$ when it crossed downwards, and $0$ when it did not cross.
• $x_{t,\,i}$: Return of asset $i$ at observation $t$.
• $t$: Threshold parameter, read as a standalone symbol; a subscript $t$ is the observation index. An asset crosses at an observation when its return is at least $t$ of its own standard deviations away from zero, and a return of exactly zero never crosses.
• $\sigma_i$: Standard deviation of asset $i$.
• $T$: Number of observations.
• $N$: Number of assets.
• $\oslash$: Element-wise division.
• $\boldsymbol{1}$: Vector of ones.

The entry of $\mathbf{N}^\intercal \mathbf{N}$ counts the observations on which neither asset crossed, so the denominator counts the observations on which at least one of them did. That is $n_{c} + n_{d} + n_{n}$, the pairwise denominator of Gerber1.

Algorithm

1. Build the indicator matrices U and D with gerber_updown.
2. Form the neutral matrix Nt, which marks the observations on which the asset crossed in neither direction.
3. Form NtN = transpose(Nt) * Nt, the count of observations on which both assets of a pair are neutral, and nneutral, the count of neutral observations of each asset on its own.
4. Form the signed crossing matrix UmD = U - D.
5. Recover the concordant count nconc and the discordant count ndisc from transpose(UmD) * UmD and the both-crossed count T .- nneutral .- transpose(nneutral) .+ NtN with concordance_counts.
6. Form nneut = nneutral .+ transpose(nneutral) .- 2 .* NtN, the count of observations on which exactly one asset of the pair crossed.
7. Reduce every pair with comovement_ratio, giving rho.
8. Write one onto a zero diagonal entry of rho with comovement_unit_diagonal!. An asset that crosses no threshold reduces to a zero diagonal entry, and that entry is one by definition.
9. Repair rho with posdef!, which is a no-op when ce.pdm is nothing.

Arguments

• ce: Gerber covariance estimator.. Configured with the Gerber1 algorithm.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• sd: Standard deviation vector of X, shaped to be consistent with X.

Returns

• rho::MatNum: Correlation matrix assets x assets.

Related

• GerberCovariance
• Gerber1
• gerber_updown
• concordance_counts
• comovement_ratio
• comovement_unit_diagonal!
• posdef!

References

• [12] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, P. Sargen and others. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021).

gerber(
    ce::GerberCovariance{<:Any, <:Any, <:Any, <:Any, <:Gerber2},
    X::MatNum,
    sd::ArrNum
) -> MatNum

Implements the second variant of the Gerber correlation algorithm.

Mathematical definition

\[\begin{align} \mathbf{G} &= \mathbf{H}^\intercal \mathbf{H}\,, \\ \boldsymbol{g} &= \sqrt{\mathrm{diag}(\mathbf{G})}\,, \\ \hat{\boldsymbol{\rho}} &= \mathbf{G} \oslash (\boldsymbol{g} \boldsymbol{g}^\intercal)\,. \end{align}\]

Where:

• $\mathbf{U} \in \{0,1\}^{T \times N}$: Up indicator matrix, $U_{t,\,i} = \mathbf{1}[x_{t,\,i} \geq t \, \sigma_i \land x_{t,\,i} > 0]$.
• $\mathbf{D} \in \{0,1\}^{T \times N}$: Down indicator matrix, $D_{t,\,i} = \mathbf{1}[x_{t,\,i} \leq -t \, \sigma_i \land x_{t,\,i} < 0]$.
• $\mathbf{H} = \mathbf{U} - \mathbf{D}$: Signed crossing matrix. Its entry is $1$ when the asset crossed upwards, $-1$ when it crossed downwards, and $0$ when it did not cross.
• $\mathbf{G}$: Raw net co-movement matrix, whose entry is $n_{c} - n_{d}$.
• $\boldsymbol{g}$: Square roots of the diagonal of $\mathbf{G}$.
• $x_{t,\,i}$: Return of asset $i$ at observation $t$.
• $t$: Threshold parameter, read as a standalone symbol; a subscript $t$ is the observation index. An asset crosses at an observation when its return is at least $t$ of its own standard deviations away from zero, and a return of exactly zero never crosses.
• $\sigma_i$: Standard deviation of asset $i$.
• $\oslash$: Element-wise division.

The diagonal of $\mathbf{G}$ counts the crossings of each asset, so the normalisation is a property of the whole matrix and the diagonal of $\hat{\boldsymbol{\rho}}$ is unit by construction.

Algorithm

1. Build the indicator matrices U and D with gerber_updown.
2. Form the signed crossing matrix UmD = U - D.
3. Form the raw net co-movement matrix rho = transpose(UmD) * UmD.
4. Normalise rho in place with standardise_comovement!.
5. Write one onto a zero diagonal entry of rho with comovement_unit_diagonal!. An asset that crosses no threshold gets a zero diagonal entry of $\mathbf{G}$, which the clamp of step 4 leaves at zero, and that entry is one by definition.
6. Repair rho with posdef!, which is a no-op when ce.pdm is nothing.

Arguments

• ce: Gerber covariance estimator.. Configured with the Gerber2 algorithm.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• sd: Standard deviation vector of X, shaped to be consistent with X.

Returns

• rho::MatNum: Correlation matrix assets x assets.

Related

• GerberCovariance
• Gerber2
• gerber_updown
• standardise_comovement!
• comovement_unit_diagonal!
• posdef!

References

• [12] S. Gerber, H. Markowitz, P. Ernst, Y. Miao, P. Sargen and others. The Gerber statistic: A robust co-movement measure for portfolio optimization. Available at SSRN 3880054 (2021).