Gerber IQ covariance (b): private API
PortfolioOptimisers.show_fields — Method
show_fields(_::GerberIQCovariance) -> NTuple{10, Symbol}
Renders every field of a GerberIQCovariance except cache.
A cache holds the state of an incremental fit, not a part of the configuration. Without this method, that state prints under the estimator at every site that renders one. Set set_show_nothing_fields!(:GerberIQCovariance, true) to render it.
Arguments
::GerberIQCovariance: Gerber IQ covariance estimator, read for its type alone.
Returns
fields::Tuple: The field names to render, which is(:ve, :me, :pdm, :c, :decay, :oc, :sc, :kind, :alg, :ex).
Related
PortfolioOptimisers.gerber_IQ_delta — Function
gerber_IQ_delta(
xi::Number,
xj::Number,
axi::Number,
axj::Number,
decay::GerberIQDecayEstimator,
T::Integer,
k::Number,
sci::Number,
scj::Number,
kind::GerberIQCovarianceAlgorithm
) -> Any
Computes the contribution of one co-movement to the Gerber IQ statistic.
The contribution is the product of the two factors of the squeezing statistic of the source. The first factor is the weight that the squeezing template gives the co-movement. The second factor is the discount for the age of the observation. comovement_step adds the contribution to pos, neg or nn, the three sums of the pair.
Mathematical definition
\[\begin{align} \Delta_{t,\,i,\,j} &= \eta_{t,\,i,\,j} \, v_{t}\,. \end{align}\]
Where:
- $\Delta_{t,\,i,\,j}$: Contribution of the co-movement of assets $i$ and $j$ at observation $t$.
- $\eta_{t,\,i,\,j}$: Squeezing weight of that co-movement, from the squeezing template.
- $v_{t}$: Temporal discount of observation $t$, from the decay estimator.
Algorithm
- Compute the squeezing weight
wwithgerber_iq_weightfrom the two returns, the two scaling factors of the pair and the squeezing template. - Compute the temporal discount
pwith thedecayfunctor atTandk. - Return
w * p.
Arguments
xi: Return of asseti.xj: Return of assetj.axi: Absolute return of asseti.axj: Absolute return of assetj.decay: The decay estimator, afterregenerate_decayresolved it.T: The number of observations.k: The index of the observation.sci: Scaling factor of asseti.scj: Scaling factor of assetj.kind: The squeezing template.
Returns
rho::Number: The weighted and discounted contribution of one co-movement. It is not a correlation.
Related
gerber_iq_weightcomovement_stepgerber_IQGerberIQDecayEstimatorGerberIQCovarianceAlgorithmGerberIQCovariance
References
- [36] S. Gerber, W. Smyth, H. Markowitz, Y. Miao, P. Ernst and P. Sargen. Squeezing financial noise: A novel approach to covariance matrix estimation. Available at SSRN 4986939 (2025). Section 3.5, Equations 4 and 5.
PortfolioOptimisers.GerberIQKernel — Type
struct GerberIQKernel{__T_alg, __T_kind, __T_decay, __T_sc, __T_c, __T_sd}Co-movement policy of the Gerber IQ family for gerber_comovement!.
The policy compares each return with the noise threshold of its asset, scaled by gerber_iq_scaling. It sorts a co-movement by the sign of the product of the two returns. gerber_IQ_delta gives the co-movement its squeezing weight and its temporal discount. The alg marker selects the denominator. comovement_ratio computes it for Gerber0 and Gerber1, and comovement_finalise computes it for Gerber2.
Fields
alg: Gerber marker that selects the denominator.kind: Squeezing template.decay: Temporal decay estimator, afterregenerate_decayresolved it.sc: Threshold scaler.c: Noise threshold, before the policy scales it.sd: Vector of the standard deviations of the assets.
Related
PortfolioOptimisers.GerberIQPairTerms — Type
struct GerberIQPairTerms{__T_sci, __T_scj, __T_ci, __T_cj}Holds the quantities of one pair of assets that the Gerber IQ step of comovement_step reads, and that do not depend on the observation.
This type is an implementation detail and is not intended for direct use. comovement_pair_state makes it once for each pair, from the scale factors of gerber_iq_scaling.
Fields
sci:sci: The scale factor of assetiin this pair.
scj:scj: The scale factor of assetjin this pair.
ci:ci: The scaled noise threshold of asseti,sci * c.
cj:cj: The scaled noise threshold of assetj,scj * c.
Related
PortfolioOptimisers.iq_add_neutral — Function
iq_add_neutral(
pol::GerberIQKernel{<:Gerber1},
acc::ComovementSums,
st::GerberIQPairTerms,
xi::Number,
xj::Number,
axi::Number,
axj::Number,
T::Integer,
k::Integer
) -> ComovementSums
Adds a neutral observation to the Gerber IQ pair accumulator.
A neutral observation is one on which exactly one of the two assets leaves the noise zone. iq_crossed decides which asset leaves it. Only the denominator of Gerber1 contains the neutral sum. So the Gerber1 method adds the gerber_IQ_delta contribution to nn, and the fall-through method returns the accumulator unchanged.
Arguments
pol: TheGerberIQKernelpolicy.acc: Pair accumulator, aComovementSums. This method changesnnalone.st: Pair state fromcomovement_pair_state.xi,xj: Returns of assetsiandjat observationk.axi,axj: Absolute returns of assetsiandj.T: Number of observations.k: Index of the observation.
Returns
- The accumulator. Under
Gerber1,nnincreases by thegerber_IQ_deltacontribution. Under the other two markers, the accumulator does not change.
Related
PortfolioOptimisers.iq_add_diagonal — Function
iq_add_diagonal(
pol::GerberIQKernel{<:Gerber2},
acc::ComovementSums,
st::GerberIQPairTerms,
xi::Number,
xj::Number,
axi::Number,
axj::Number,
crossi::Bool,
crossj::Bool,
T::Integer,
k::Integer
) -> ComovementSums
Adds the two diagonal projections of one observation to the Gerber IQ pair accumulator.
Only the denominator of Gerber2 contains the projections, so only Gerber2 reaches the method that adds them. The fall-through method returns the accumulator unchanged. The other two markers thus compute no weight that they do not use.
The projection of an observation in the i direction is the co-movement (x_i, x_i), in the units of this pair. Both coordinates fall in the same magnitude class, and the co-movement is concordant. So the projection gives the diagonal weight of asset i at that observation. The j direction is the mirror. An asset contributes only when it leaves the noise zone, which is the test that the numerator applies too.
The projection keeps the statistic inside [-1, 1]. When sc is not pair-separable, the magnitude class of an asset changes with its partner. A denominator from the assembled diagonal uses the class of the pair (i, i) instead, and the Cauchy-Schwarz bound then does not hold. A projection in the units of the pair makes the bound hold for every scaler.
Arguments
pol: TheGerberIQKernelpolicy.acc: Pair accumulator, aComovementSums. This method changesdianddjalone.st: Pair state fromcomovement_pair_state.xi,xj: Returns of assetsiandjat observationk.axi,axj: Absolute returns of assetsiandj.crossi,crossj:truefor an asset that leaves the noise zone, fromiq_crossed.T: Number of observations.k: Index of the observation.
Returns
- The accumulator. Under
Gerber2,dianddjincrease by the projectedgerber_IQ_deltacontributions. Under the other two markers, the accumulator does not change.
Related
PortfolioOptimisers.iq_crossed — Function
iq_crossed(x::Number, ax::Number, c::Number) -> Any
Decides whether one asset leaves the noise zone at one observation.
An asset leaves the noise zone when its absolute return is at least the scaled threshold of the pair, and the return is not zero. For a positive threshold, the second test changes nothing, because ax >= c > 0 makes x nonzero. The second test matters only at c = 0, where ax >= 0 holds for every return, a zero return too. So a return of exactly zero never crosses. The Gerber statistic has the same rule.
The rule keeps the diagonal of the statistic at one. The pair (i, i) crosses on both axes or on neither, so it never adds to the neutral sum that Gerber1 divides by. Without the second test, a zero return crosses on both axes but has no sign. It then goes to the neutral sum, and the diagonal entry becomes less than one.
Arguments
x: Return of the asset at the observation.ax: Absolute return of the asset.c: The scaled noise threshold of the asset, fromcomovement_pair_state.
Returns
crossed::Bool:truewhen the asset leaves the noise zone.
Related
PortfolioOptimisers.gerber_IQ — Function
gerber_IQ(
ce::GerberIQCovariance,
X::MatNum,
sd::ArrNum
) -> MatNumComputes the Gerber IQ correlation matrix with the squeezing template in ce.kind and the denominator that ce.alg selects.
Mathematical definition
For each pair of assets $(i,j)$, the statistic sums the weighted and discounted co-movements of each sign.
\[\begin{align} H_{ij}^{+} &= \sum_{k=1}^{T} w_{ij,k} \cdot d_k \cdot \mathbf{1}[\text{concordant}]\,, \\ H_{ij}^{-} &= \sum_{k=1}^{T} w_{ij,k} \cdot d_k \cdot \mathbf{1}[\text{discordant}]\,. \end{align}\]
Where:
- $H_{ij}^{+}$, $H_{ij}^{-}$: Weighted concordant and discordant sums.
- $T$: Number of observations.
- $w_{ij,k}$: Squeezing weight of the co-movement at observation $k$, from the squeezing template.
- $d_k = \exp[-y \max(0, T-k-e)]$: Temporal decay at observation $k$.
The Gerber IQ correlation is:
\[\begin{align} \rho_{ij} &= \begin{cases} (H_{ij}^{+} - H_{ij}^{-}) / (H_{ij}^{+} + H_{ij}^{-}) & \text{Gerber0} \\ (H_{ij}^{+} - H_{ij}^{-}) / (H_{ij}^{+} + H_{ij}^{-} + H_{ij}^{0}) & \text{Gerber1} \\ (H_{ij}^{+} - H_{ij}^{-}) / \sqrt{D_{ij}\,D_{ji}} & \text{Gerber2} \end{cases}\,. \end{align}\]
Where:
- $\rho_{ij}$: Gerber IQ correlation of assets $i$ and $j$.
- $H_{ij}^{+}$, $H_{ij}^{-}$: Weighted concordant and discordant sums.
- $H_{ij}^{0}$: Weighted neutral sum, over the observations on which exactly one asset leaves the noise zone. Only the
Gerber1denominator contains it. - $D_{ij} = \sum_{k} w_{ii,k}\, d_{k}$: Projection of asset $i$ onto the lead diagonal, in the units of the pair $(i, j)$. The sum runs over the observations on which asset $i$ leaves the noise zone. The projected co-movement $(x_{ki}, x_{ki})$ falls in one magnitude class on both axes, so $w_{ii,k}$ is the diagonal weight of asset $i$ at that observation. $D_{ij}$ and $D_{ji}$ differ. When
scis not pair-separable, both change with the pair.
The Gerber1 statistic is the statistic of the source. Its numerator sums over the observations on which both assets leave the noise zone. Its denominator sums over the observations on which at least one asset leaves it. The Gerber0 and Gerber2 statistics apply the denominators of the classic Gerber family to the weighted and discounted sums. The main text of the source states neither of them.
The Internet Appendix of the source states a third form. Its denominator is the geometric mean of the two diagonal projections, summed over the observations on which both assets cross. The library does not ship that form. Gerber2 makes the same projections, but it keeps the observation set of the classic denominator. With that set, the statistic reduces to GerberCovariance.
The Gerber statistic is the special case of the Gerber IQ statistic with no squeezing and no decay. Set every weight to one, $\gamma = 0$, the scaler AssetVolatilityGerberIQScaler and $c$ equal to a Gerber threshold. All three markers then give the result of GerberCovariance to the last bit. The reduction holds at $c = 0$ too, because iq_crossed gives this family the rule of the Gerber statistic. Under that rule, a return of exactly zero never leaves the noise zone.
All three markers keep $|\rho_{ij}| \leq 1$. Gerber0 and Gerber1 keep it by construction, because each divides by a sum of the weights that it subtracts. Gerber2 keeps it under a condition of the source on the squeezing template. Each weight that joins two distinct magnitude classes must be at most the geometric mean of the two diagonal weights of those classes. clamp_gerber_iq_n enforces that condition on every such weight.
The Internet Appendix of [36] proves that the condition is necessary and sufficient for its own diagonal projection, in Equations A1 to A9. For the larger observation set of the library, the condition is sufficient. The Cauchy-Schwarz inequality then bounds the ratio, because $D_{ij}$ measures the class of asset $i$ in the units of the numerator, for every sc. The diagonal is exactly one, because for the pair $(i, i)$ the numerator and both projections are the same sum.
Algorithm
- Allocate the
N × Noutput matrixrho. - Resolve the decay estimator against
Xwithregenerate_decay, so that its delay and its rate are numbers before the loop starts. - Make the
GerberIQKernelpolicy from the resolved decay, from thealg,kind,scandcof the estimator, and from the standard deviationssd. - Fill
rhowithgerber_comovement!. It loops over every pair and every observation, and it reduces the sums of each pair. The Smyth-Broby family uses the same loop. - Write one on each zero diagonal entry with
comovement_unit_diagonal!. An asset that never leaves its noise zone gets a zero diagonal entry, but the diagonal of a correlation matrix is one by definition. - Repair the matrix with
posdef!, because the statistic is not always positive semidefinite. The source states the same, and it repairs the matrix with the nearest correlation matrix.
The three GerberCovarianceAlgorithm markers differ at step 4 alone. comovement_ratio computes the Gerber0 and Gerber1 denominators, and comovement_finalise computes the Gerber2 denominator. This family does not call standardise_comovement!. That function normalises the assembled matrix, so it cannot use the units of each pair.
Arguments
ce: Covariance estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.sd: Standard deviation vector ofX, shaped to be consistent withX.
Returns
rho::MatNum: Correlation matrixassets x assets.
Related
GerberIQCovarianceGerberIQKernelgerber_comovement!comovement_unit_diagonal!Gerber0Gerber1Gerber2gerber_IQ_deltaregenerate_decaycor(ce::GerberIQCovariance, X::MatNum; dims::Int = 1, kwargs...)cov(ce::GerberIQCovariance, X::MatNum; dims::Int = 1, kwargs...)
References
- [36] S. Gerber, W. Smyth, H. Markowitz, Y. Miao, P. Ernst and P. Sargen. Squeezing financial noise: A novel approach to covariance matrix estimation. Available at SSRN 4986939 (2025). Section 3.5, Equations 1 to 5, and the Internet Appendix, Equation A1.
References
- [36]
- S. Gerber, W. Smyth, H. Markowitz, Y. Miao, P. Ernst and P. Sargen. Squeezing financial noise: A novel approach to covariance matrix estimation. Available at SSRN 4986939 (2025).