Gerber IQ covariance (b): private API

PortfolioOptimisers.show_fields — Method
show_fields(_::GerberIQCovariance) -> NTuple{10, Symbol}

Renders every field of a GerberIQCovariance except cache.

A cache holds the state of an incremental fit, not a part of the configuration. Without this method, that state prints under the estimator at every site that renders one. Set set_show_nothing_fields!(:GerberIQCovariance, true) to render it.

Arguments

  • ::GerberIQCovariance: Gerber IQ covariance estimator, read for its type alone.

Returns

  • fields::Tuple: The field names to render, which is (:ve, :me, :pdm, :c, :decay, :oc, :sc, :kind, :alg, :ex).

Related

source
PortfolioOptimisers.gerber_IQ_delta — Function
gerber_IQ_delta(
    xi::Number,
    xj::Number,
    axi::Number,
    axj::Number,
    decay::GerberIQDecayEstimator,
    T::Integer,
    k::Number,
    sci::Number,
    scj::Number,
    kind::GerberIQCovarianceAlgorithm
) -> Any

Computes the contribution of one co-movement to the Gerber IQ statistic.

The contribution is the product of the two factors of the squeezing statistic of the source. The first factor is the weight that the squeezing template gives the co-movement. The second factor is the discount for the age of the observation. comovement_step adds the contribution to pos, neg or nn, the three sums of the pair.

Mathematical definition

\[\begin{align} \Delta_{t,\,i,\,j} &= \eta_{t,\,i,\,j} \, v_{t}\,. \end{align}\]

Where:

  • $\Delta_{t,\,i,\,j}$: Contribution of the co-movement of assets $i$ and $j$ at observation $t$.
  • $\eta_{t,\,i,\,j}$: Squeezing weight of that co-movement, from the squeezing template.
  • $v_{t}$: Temporal discount of observation $t$, from the decay estimator.

Algorithm

  1. Compute the squeezing weight w with gerber_iq_weight from the two returns, the two scaling factors of the pair and the squeezing template.
  2. Compute the temporal discount p with the decay functor at T and k.
  3. Return w * p.

Arguments

  • xi: Return of asset i.
  • xj: Return of asset j.
  • axi: Absolute return of asset i.
  • axj: Absolute return of asset j.
  • decay: The decay estimator, after regenerate_decay resolved it.
  • T: The number of observations.
  • k: The index of the observation.
  • sci: Scaling factor of asset i.
  • scj: Scaling factor of asset j.
  • kind: The squeezing template.

Returns

  • rho::Number: The weighted and discounted contribution of one co-movement. It is not a correlation.

Related

References

  • [36] S. Gerber, W. Smyth, H. Markowitz, Y. Miao, P. Ernst and P. Sargen. Squeezing financial noise: A novel approach to covariance matrix estimation. Available at SSRN 4986939 (2025). Section 3.5, Equations 4 and 5.
source
PortfolioOptimisers.GerberIQKernel — Type
struct GerberIQKernel{__T_alg, __T_kind, __T_decay, __T_sc, __T_c, __T_sd}

Co-movement policy of the Gerber IQ family for gerber_comovement!.

The policy compares each return with the noise threshold of its asset, scaled by gerber_iq_scaling. It sorts a co-movement by the sign of the product of the two returns. gerber_IQ_delta gives the co-movement its squeezing weight and its temporal discount. The alg marker selects the denominator. comovement_ratio computes it for Gerber0 and Gerber1, and comovement_finalise computes it for Gerber2.

Fields

  • alg: Gerber marker that selects the denominator.
  • kind: Squeezing template.
  • decay: Temporal decay estimator, after regenerate_decay resolved it.
  • sc: Threshold scaler.
  • c: Noise threshold, before the policy scales it.
  • sd: Vector of the standard deviations of the assets.

Related

source
PortfolioOptimisers.GerberIQPairTerms — Type
struct GerberIQPairTerms{__T_sci, __T_scj, __T_ci, __T_cj}

Holds the quantities of one pair of assets that the Gerber IQ step of comovement_step reads, and that do not depend on the observation.

This type is an implementation detail and is not intended for direct use. comovement_pair_state makes it once for each pair, from the scale factors of gerber_iq_scaling.

Fields

  • sci: sci: The scale factor of asset i in this pair.
  • scj: scj: The scale factor of asset j in this pair.
  • ci: ci: The scaled noise threshold of asset i, sci * c.
  • cj: cj: The scaled noise threshold of asset j, scj * c.

Related

source
PortfolioOptimisers.iq_add_neutral — Function
iq_add_neutral(
    pol::GerberIQKernel{<:Gerber1},
    acc::ComovementSums,
    st::GerberIQPairTerms,
    xi::Number,
    xj::Number,
    axi::Number,
    axj::Number,
    T::Integer,
    k::Integer
) -> ComovementSums

Adds a neutral observation to the Gerber IQ pair accumulator.

A neutral observation is one on which exactly one of the two assets leaves the noise zone. iq_crossed decides which asset leaves it. Only the denominator of Gerber1 contains the neutral sum. So the Gerber1 method adds the gerber_IQ_delta contribution to nn, and the fall-through method returns the accumulator unchanged.

Arguments

  • pol: The GerberIQKernel policy.
  • acc: Pair accumulator, a ComovementSums. This method changes nn alone.
  • st: Pair state from comovement_pair_state.
  • xi, xj: Returns of assets i and j at observation k.
  • axi, axj: Absolute returns of assets i and j.
  • T: Number of observations.
  • k: Index of the observation.

Returns

  • The accumulator. Under Gerber1, nn increases by the gerber_IQ_delta contribution. Under the other two markers, the accumulator does not change.

Related

source
PortfolioOptimisers.iq_add_diagonal — Function
iq_add_diagonal(
    pol::GerberIQKernel{<:Gerber2},
    acc::ComovementSums,
    st::GerberIQPairTerms,
    xi::Number,
    xj::Number,
    axi::Number,
    axj::Number,
    crossi::Bool,
    crossj::Bool,
    T::Integer,
    k::Integer
) -> ComovementSums

Adds the two diagonal projections of one observation to the Gerber IQ pair accumulator.

Only the denominator of Gerber2 contains the projections, so only Gerber2 reaches the method that adds them. The fall-through method returns the accumulator unchanged. The other two markers thus compute no weight that they do not use.

The projection of an observation in the i direction is the co-movement (x_i, x_i), in the units of this pair. Both coordinates fall in the same magnitude class, and the co-movement is concordant. So the projection gives the diagonal weight of asset i at that observation. The j direction is the mirror. An asset contributes only when it leaves the noise zone, which is the test that the numerator applies too.

The projection keeps the statistic inside [-1, 1]. When sc is not pair-separable, the magnitude class of an asset changes with its partner. A denominator from the assembled diagonal uses the class of the pair (i, i) instead, and the Cauchy-Schwarz bound then does not hold. A projection in the units of the pair makes the bound hold for every scaler.

Arguments

  • pol: The GerberIQKernel policy.
  • acc: Pair accumulator, a ComovementSums. This method changes di and dj alone.
  • st: Pair state from comovement_pair_state.
  • xi, xj: Returns of assets i and j at observation k.
  • axi, axj: Absolute returns of assets i and j.
  • crossi, crossj: true for an asset that leaves the noise zone, from iq_crossed.
  • T: Number of observations.
  • k: Index of the observation.

Returns

  • The accumulator. Under Gerber2, di and dj increase by the projected gerber_IQ_delta contributions. Under the other two markers, the accumulator does not change.

Related

source
PortfolioOptimisers.iq_crossed — Function
iq_crossed(x::Number, ax::Number, c::Number) -> Any

Decides whether one asset leaves the noise zone at one observation.

An asset leaves the noise zone when its absolute return is at least the scaled threshold of the pair, and the return is not zero. For a positive threshold, the second test changes nothing, because ax >= c > 0 makes x nonzero. The second test matters only at c = 0, where ax >= 0 holds for every return, a zero return too. So a return of exactly zero never crosses. The Gerber statistic has the same rule.

The rule keeps the diagonal of the statistic at one. The pair (i, i) crosses on both axes or on neither, so it never adds to the neutral sum that Gerber1 divides by. Without the second test, a zero return crosses on both axes but has no sign. It then goes to the neutral sum, and the diagonal entry becomes less than one.

Arguments

  • x: Return of the asset at the observation.
  • ax: Absolute return of the asset.
  • c: The scaled noise threshold of the asset, from comovement_pair_state.

Returns

  • crossed::Bool: true when the asset leaves the noise zone.

Related

source
PortfolioOptimisers.gerber_IQ — Function
gerber_IQ(
    ce::GerberIQCovariance,
    X::MatNum,
    sd::ArrNum
) -> MatNum

Computes the Gerber IQ correlation matrix with the squeezing template in ce.kind and the denominator that ce.alg selects.

Mathematical definition

For each pair of assets $(i,j)$, the statistic sums the weighted and discounted co-movements of each sign.

\[\begin{align} H_{ij}^{+} &= \sum_{k=1}^{T} w_{ij,k} \cdot d_k \cdot \mathbf{1}[\text{concordant}]\,, \\ H_{ij}^{-} &= \sum_{k=1}^{T} w_{ij,k} \cdot d_k \cdot \mathbf{1}[\text{discordant}]\,. \end{align}\]

Where:

  • $H_{ij}^{+}$, $H_{ij}^{-}$: Weighted concordant and discordant sums.
  • $T$: Number of observations.
  • $w_{ij,k}$: Squeezing weight of the co-movement at observation $k$, from the squeezing template.
  • $d_k = \exp[-y \max(0, T-k-e)]$: Temporal decay at observation $k$.

The Gerber IQ correlation is:

\[\begin{align} \rho_{ij} &= \begin{cases} (H_{ij}^{+} - H_{ij}^{-}) / (H_{ij}^{+} + H_{ij}^{-}) & \text{Gerber0} \\ (H_{ij}^{+} - H_{ij}^{-}) / (H_{ij}^{+} + H_{ij}^{-} + H_{ij}^{0}) & \text{Gerber1} \\ (H_{ij}^{+} - H_{ij}^{-}) / \sqrt{D_{ij}\,D_{ji}} & \text{Gerber2} \end{cases}\,. \end{align}\]

Where:

  • $\rho_{ij}$: Gerber IQ correlation of assets $i$ and $j$.
  • $H_{ij}^{+}$, $H_{ij}^{-}$: Weighted concordant and discordant sums.
  • $H_{ij}^{0}$: Weighted neutral sum, over the observations on which exactly one asset leaves the noise zone. Only the Gerber1 denominator contains it.
  • $D_{ij} = \sum_{k} w_{ii,k}\, d_{k}$: Projection of asset $i$ onto the lead diagonal, in the units of the pair $(i, j)$. The sum runs over the observations on which asset $i$ leaves the noise zone. The projected co-movement $(x_{ki}, x_{ki})$ falls in one magnitude class on both axes, so $w_{ii,k}$ is the diagonal weight of asset $i$ at that observation. $D_{ij}$ and $D_{ji}$ differ. When sc is not pair-separable, both change with the pair.

The Gerber1 statistic is the statistic of the source. Its numerator sums over the observations on which both assets leave the noise zone. Its denominator sums over the observations on which at least one asset leaves it. The Gerber0 and Gerber2 statistics apply the denominators of the classic Gerber family to the weighted and discounted sums. The main text of the source states neither of them.

The Internet Appendix of the source states a third form. Its denominator is the geometric mean of the two diagonal projections, summed over the observations on which both assets cross. The library does not ship that form. Gerber2 makes the same projections, but it keeps the observation set of the classic denominator. With that set, the statistic reduces to GerberCovariance.

The Gerber statistic is the special case of the Gerber IQ statistic with no squeezing and no decay. Set every weight to one, $\gamma = 0$, the scaler AssetVolatilityGerberIQScaler and $c$ equal to a Gerber threshold. All three markers then give the result of GerberCovariance to the last bit. The reduction holds at $c = 0$ too, because iq_crossed gives this family the rule of the Gerber statistic. Under that rule, a return of exactly zero never leaves the noise zone.

All three markers keep $|\rho_{ij}| \leq 1$. Gerber0 and Gerber1 keep it by construction, because each divides by a sum of the weights that it subtracts. Gerber2 keeps it under a condition of the source on the squeezing template. Each weight that joins two distinct magnitude classes must be at most the geometric mean of the two diagonal weights of those classes. clamp_gerber_iq_n enforces that condition on every such weight.

The Internet Appendix of [36] proves that the condition is necessary and sufficient for its own diagonal projection, in Equations A1 to A9. For the larger observation set of the library, the condition is sufficient. The Cauchy-Schwarz inequality then bounds the ratio, because $D_{ij}$ measures the class of asset $i$ in the units of the numerator, for every sc. The diagonal is exactly one, because for the pair $(i, i)$ the numerator and both projections are the same sum.

Algorithm

  1. Allocate the N × N output matrix rho.
  2. Resolve the decay estimator against X with regenerate_decay, so that its delay and its rate are numbers before the loop starts.
  3. Make the GerberIQKernel policy from the resolved decay, from the alg, kind, sc and c of the estimator, and from the standard deviations sd.
  4. Fill rho with gerber_comovement!. It loops over every pair and every observation, and it reduces the sums of each pair. The Smyth-Broby family uses the same loop.
  5. Write one on each zero diagonal entry with comovement_unit_diagonal!. An asset that never leaves its noise zone gets a zero diagonal entry, but the diagonal of a correlation matrix is one by definition.
  6. Repair the matrix with posdef!, because the statistic is not always positive semidefinite. The source states the same, and it repairs the matrix with the nearest correlation matrix.

The three GerberCovarianceAlgorithm markers differ at step 4 alone. comovement_ratio computes the Gerber0 and Gerber1 denominators, and comovement_finalise computes the Gerber2 denominator. This family does not call standardise_comovement!. That function normalises the assembled matrix, so it cannot use the units of each pair.

Arguments

  • ce: Covariance estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • sd: Standard deviation vector of X, shaped to be consistent with X.

Returns

  • rho::MatNum: Correlation matrix assets x assets.

Related

References

  • [36] S. Gerber, W. Smyth, H. Markowitz, Y. Miao, P. Ernst and P. Sargen. Squeezing financial noise: A novel approach to covariance matrix estimation. Available at SSRN 4986939 (2025). Section 3.5, Equations 1 to 5, and the Internet Appendix, Equation A1.
source

References

[36]
S. Gerber, W. Smyth, H. Markowitz, Y. Miao, P. Ernst and P. Sargen. Squeezing financial noise: A novel approach to covariance matrix estimation. Available at SSRN 4986939 (2025).