GitHub

Cross-Sectional Regression Diagnostics

Functions

PortfolioOptimisers.cs_gram — Function
cs_gram(B::Arr3Num, w::Option{<:MatNum} = nothing) -> Array{<:Real, 3}

Return the weighted Gram history of a cross-sectional regression, one slice per observation.

The slice of observation $t$ is the normal matrix of the weighted design, and every diagnostic that reads the geometry of the cross-section is a function of it: the variance inflation factors, the condition number and the standard errors of the factor returns all read this one history. The design is masked first, so an asset whose exposures are not all finite contributes nothing, and so does an asset whose weight is zero.

Mathematical definition

\[\mathbf{G}_{t} = \mathbf{B}_{t}^{\intercal} \mathbf{W}_{t} \mathbf{B}_{t}\]

Where:

  • $\mathbf{B}_{t}$: Exposure matrix of observation $t$, $N \times K$, whose row of an ineligible asset is zero.
  • $\mathbf{W}_{t}$: Diagonal weight matrix of observation $t$, whose entry of an ineligible asset is zero.
  • $N$: Number of assets.
  • $K$: Number of factors.

Algorithm

  1. Take the mask and the weights with cs_estimation_mask_weights.
  2. For each observation, scale the exposure rows by the square root of their weights, which is the weighted design.
  3. Multiply the weighted design by its own transpose, giving the slice of that observation.

Arguments

  • B: Exposure history observations × assets × factors, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights. A weight of zero excludes the pair.

Validation

  • !isempty(B).
  • size(w) == (size(B, 1), size(B, 2)), when w is present.

Returns

  • G::Array{<:Real, 3}: Gram history observations × factors × factors.

Examples

julia> B = reshape([1.0, 1.0, 0.0, 1.0], 1, 2, 2);julia> cs_gram(B)1×2×2 Array{Float64, 3}:[:, :, 1] = 2.0  1.0[:, :, 2] = 1.0  1.0

Related

source
PortfolioOptimisers.exposure_vif — Function
exposure_vif(G::Arr3Num) -> Matrix{<:Real}
exposure_vif(B::Arr3Num, w::Option{<:MatNum}) -> Matrix{<:Real}
exposure_vif(csfm::CrossSectionalFactorModel) -> Matrix{<:Real}

Return the variance inflation factor of every factor, one row per observation.

The factor measures how much the collinearity of the cross-sectional design inflates the variance of a factor return. A value of one says that the factor is orthogonal to the others of that observation, and a large value says that the factor is nearly a combination of them, so its estimated return is unstable rather than wrong.

Mathematical definition

\[\mathrm{VIF}_{t,k} = (\mathbf{G}_{t})_{kk} \, (\mathbf{G}_{t}^{-1})_{kk}\]

Where:

  • $\mathbf{G}_{t}$: Gram matrix of observation $t$, which cs_gram defines.
  • $\mathrm{VIF}_{t,k}$: Variance inflation factor of factor $k$ at observation $t$.

Algorithm

  1. Take the diagonal of the inverse of every slice with cs_gram_inverse_diagonal.
  2. Multiply it entry by entry by the diagonal of the slice itself.
  3. On the method that reads a design, answer NaN at an observation whose eligible asset count does not exceed the factor count, because such a design determines no variance.

Arguments

  • G: Gram history observations × factors × factors, which cs_gram returns.
  • B: Exposure history observations × assets × factors, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • csfm: A cross-sectional factor model block. The answer is on the reduced factor axis when the block carries a family re-basis.

Validation

  • !isempty(B), and size(w) == (size(B, 1), size(B, 2)) when w is present.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • vif::Matrix{<:Real}: observations × factors.

Examples

julia> B = reshape([1.0, 0.0, 0.0, 1.0], 1, 2, 2);julia> exposure_vif(cs_gram(B))1×2 Matrix{Float64}: 1.0  1.0

Related

source
PortfolioOptimisers.exposure_condition_number — Function
exposure_condition_number(G::Arr3Num) -> Vector{<:Real}
exposure_condition_number(B::Arr3Num, w::Option{<:MatNum}) -> Vector{<:Real}
exposure_condition_number(csfm::CrossSectionalFactorModel) -> Vector{<:Real}

Return the two-norm condition number of the cross-sectional design, one entry per observation.

The condition number reads the whole design at once, where a variance inflation factor reads one factor of it. A value of one says that the weighted design is orthonormal, and a large value says that one direction of the factor space carries almost no weighted exposure, so the fit of that observation is sensitive to a small change in the returns. An exactly collinear design answers Inf or a value near the reciprocal of the machine epsilon, and which of the two it answers is a property of the singular value decomposition rather than of the design: the smallest singular value of a rank-deficient matrix rounds either to zero or to a value near the machine epsilon. Read both answers as one statement, that the design of that observation is singular.

Mathematical definition

\[\kappa_{t} = \frac{\sigma_{\max}(\mathbf{G}_{t})}{\sigma_{\min}(\mathbf{G}_{t})}\]

Where:

  • $\mathbf{G}_{t}$: Gram matrix of observation $t$, which cs_gram defines.
  • $\sigma_{\max}$, $\sigma_{\min}$: Largest and smallest singular values.

Arguments

  • G: Gram history observations × factors × factors, which cs_gram returns.
  • B: Exposure history observations × assets × factors, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • csfm: A cross-sectional factor model block. The answer is on the reduced factor axis when the block carries a family re-basis.

Validation

  • !isempty(B), and size(w) == (size(B, 1), size(B, 2)) when w is present.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • kappa::Vector{<:Real}: One entry per observation. The methods that read a design answer NaN at an observation whose eligible asset count does not exceed the factor count.

Examples

julia> B = reshape([1.0, 0.0, 0.0, 1.0], 1, 2, 2);julia> exposure_condition_number(cs_gram(B))1-element Vector{Float64}: 1.0

Related

source
PortfolioOptimisers.cs_regression_t_stats — Function
cs_regression_t_stats(
    B::Arr3Num,
    f::MatNum,
    eps::MatNum,
    w::Option{<:MatNum} = nothing;
    G::Option{<:Arr3Num} = nothing
) -> Matrix{<:Real}
cs_regression_t_stats(csfm::CrossSectionalFactorModel) -> Matrix{<:Real}

Return the t-statistic of every factor return, one row per observation.

The statistic says how many standard errors a factor return of one observation sits from zero, so a rule of thumb reads an absolute value above two as significant at about the five per cent level. The fit stores no standard error, so this verb recomputes it from the Gram matrix and the residuals, and the fit result gains no field.

Mathematical definition

\[\begin{align} t_{t,k} &= \frac{f_{t,k}}{\mathrm{SE}_{t,k}}\,,\\ \mathrm{SE}_{t,k} &= \sqrt{\hat{\sigma}^{2}_{t} \, (\mathbf{G}_{t}^{-1})_{kk}}\,,\\ \hat{\sigma}^{2}_{t} &= \frac{\mathrm{RSS}_{t}}{n_{t} - K}\,,\\ \mathrm{RSS}_{t} &= \sum_{i} u_{t,i} \, \varepsilon_{t,i}^{2}\,. \end{align}\]

Where:

  • $f_{t,k}$: Factor return of factor $k$ at observation $t$, which is the coefficient of the cross-sectional fit.
  • $\mathbf{G}_{t}$: Gram matrix of observation $t$, which cs_gram defines.
  • $u_{t,i}$: Resolved regression weight of asset $i$ at observation $t$, zero outside the mask.
  • $\varepsilon_{t,i}$: Residual of asset $i$ at observation $t$.
  • $n_{t}$: Number of eligible assets at observation $t$.
  • $K$: Number of factors.

Algorithm

  1. Resolve the mask and the weights with cs_diagnostic_mask_weights, which also excludes a pair whose residual is not finite.
  2. Build the Gram history with cs_gram_from_weights, or take the one the caller supplied through G.
  3. Take the residual sum of squares of every observation, and divide it by the degrees of freedom.
  4. Scale the diagonal of the inverse Gram by that variance, and take the square root, which is the standard error.
  5. Divide the factor returns by the standard errors. Answer NaN where the degrees of freedom are not positive, where a factor return of that observation is not finite, or where the standard error is zero.

Arguments

  • B: Exposure history observations × assets × factors, already lagged.
  • f: Factor return matrix observations × factors, already lagged.
  • eps: Residual matrix observations × assets, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • G: Gram history observations × factors × factors the caller already holds, or nothing to build it. A caller that supplies one is stating that it was built from the same mask, which this verb does not check.
  • csfm: A cross-sectional factor model block. The answer is on the reduced factor axis when the block carries a family re-basis.

Validation

  • !isempty(B), and every history agrees with B on the axes it shares.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • t::Matrix{<:Real}: observations × factors.

Examples

julia> B = reshape([1.0, 1.0, 1.0, -1.0, 1.0, 0.0], 1, 3, 2);julia> round.(cs_regression_t_stats(B, [2.0 1.0], [0.1 -0.1 0.05]); digits = 4)1×2 Matrix{Float64}: 23.094  9.4281

Related

source
PortfolioOptimisers.cs_regression_t_stat_exceedance_rate — Function
cs_regression_t_stat_exceedance_rate(t::MatNum; threshold::Number = 2) -> Vector{<:Real}
cs_regression_t_stat_exceedance_rate(
    csfm::CrossSectionalFactorModel;
    threshold::Number = 2
) -> Vector{<:Real}

Return the fraction of observations at which a factor's t-statistic exceeds a threshold.

A factor whose true cross-sectional coefficient is zero, and whose t-statistics are about Gaussian, exceeds a threshold of two at about five per cent of the observations. A rate above that reference level says that the factor is repeatedly significant rather than significant once. An observation whose t-statistic is NaN counts in neither the numerator nor the denominator.

Mathematical definition

\[\mathrm{rate}_{k} = \frac{\#\{t : |t_{t,k}| > \tau\}}{\#\{t : t_{t,k} \text{ is finite}\}}\]

Where:

  • $t_{t,k}$: T-statistic of factor $k$ at observation $t$, which cs_regression_t_stats returns.
  • $\tau$: Absolute threshold.

Arguments

  • t: T-statistic matrix observations × factors.
  • csfm: A cross-sectional factor model block. The answer is on the reduced factor axis when the block carries a family re-basis.
  • threshold: Absolute t-statistic above which an observation counts as significant.

Validation

  • !isempty(t).

Returns

  • rate::Vector{<:Real}: One entry per factor. A factor with no finite t-statistic answers zero.

Examples

julia> cs_regression_t_stat_exceedance_rate([3.0 1.0; 1.0 1.0; NaN 1.0])2-element Vector{Float64}: 0.5 0.0

Related

source
PortfolioOptimisers.cs_regression_r2 — Function
cs_regression_r2(
    B::Arr3Num,
    f::MatNum,
    eps::MatNum,
    w::Option{<:MatNum} = nothing
) -> Vector{<:Real}
cs_regression_r2(csfm::CrossSectionalFactorModel) -> Vector{<:Real}

Return the weighted cross-sectional coefficient of determination, one entry per observation.

The score says what share of the weighted cross-sectional variance of the returns the factors of that observation explain. The weights are normalised to sum to one, so an observation with many assets and an observation with few are on one scale.

Mathematical definition

\[R^{2}_{t} = 1 - \frac{\sum_{i} q_{t,i} \, \varepsilon_{t,i}^{2}} {\sum_{i} q_{t,i} \, (r_{t,i} - \bar{r}_{t})^{2}}\]

Where:

  • $q_{t,i}$: Regression weight of asset $i$ at observation $t$, normalised over the eligible assets to sum to one.
  • $\varepsilon_{t,i}$: Residual of asset $i$ at observation $t$.
  • $r_{t,i}$: Return of asset $i$ at observation $t$, reconstructed as the systematic part plus the residual.
  • $\bar{r}_{t}$: Weighted mean return of observation $t$.

Arguments

  • B: Exposure history observations × assets × factors, already lagged.
  • f: Factor return matrix observations × factors, already lagged.
  • eps: Residual matrix observations × assets, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • csfm: A cross-sectional factor model block.

Validation

  • !isempty(B), and every history agrees with B on the axes it shares.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • r2::Vector{<:Real}: One entry per observation, NaN where the weighted total sum of squares is zero.

Examples

julia> B = reshape([1.0, 1.0, -1.0, 1.0], 1, 2, 2);julia> cs_regression_r2(B, [1.0 1.0], [0.0 0.0])1-element Vector{Float64}: 1.0

Related

source
PortfolioOptimisers.cs_regression_adjusted_r2 — Function
cs_regression_adjusted_r2(
    B::Arr3Num,
    f::MatNum,
    eps::MatNum,
    w::Option{<:MatNum} = nothing;
    k::Integer = size(B, 3)
) -> Vector{<:Real}
cs_regression_adjusted_r2(csfm::CrossSectionalFactorModel) -> Vector{<:Real}

Return the cross-sectional coefficient of determination adjusted for the regressor count, one entry per observation.

The adjustment charges the score for every regressor, so a factor that explains nothing lowers it rather than leaving it flat. The adjusted score therefore never exceeds cs_regression_r2, and it is the score to read when two designs of different sizes are compared.

Mathematical definition

\[\bar{R}^{2}_{t} = 1 - (1 - R^{2}_{t}) \, \frac{n_{t} - 1}{n_{t} - k - 1}\]

Where:

  • $R^{2}_{t}$: Coefficient of determination of observation $t$, which cs_regression_r2 defines.
  • $n_{t}$: Number of eligible assets at observation $t$.
  • $k$: Effective number of regressors.

Arguments

  • B: Exposure history observations × assets × factors, already lagged.
  • f: Factor return matrix observations × factors, already lagged.
  • eps: Residual matrix observations × assets, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • k: Effective number of regressors. The block method takes the factor count of the reduced axis, which is what the fit spent.
  • csfm: A cross-sectional factor model block.

Validation

  • !isempty(B), and every history agrees with B on the axes it shares.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • adj::Vector{<:Real}: One entry per observation, NaN where the eligible asset count does not exceed k + 1.

Examples

julia> B = reshape([1.0, 1.0, 1.0, -1.0, 1.0, 0.0], 1, 3, 2);julia> cs_regression_adjusted_r2(B, [1.0 0.0], [0.0 0.0 0.0])1-element Vector{Float64}: NaN

Related

source
PortfolioOptimisers.cs_regression_aic — Function
cs_regression_aic(
    B::Arr3Num,
    f::MatNum,
    eps::MatNum,
    w::Option{<:MatNum} = nothing;
    k::Integer = size(B, 3)
) -> Vector{<:Real}
cs_regression_aic(csfm::CrossSectionalFactorModel) -> Vector{<:Real}

Return the Akaike information criterion of every cross-sectional fit, one entry per observation.

The criterion trades the fit of an observation against the size of its design, and a lower value is the better trade. It shares its residual term with cs_regression_bic and differs only in the penalty, which is flat in the asset count here and grows with it there.

Mathematical definition

\[\mathrm{AIC}_{t} = n_{t} \ln (\mathrm{RSS}_{t}) + 2 k\]

Where:

  • $\mathrm{RSS}_{t}$: Weight-normalised residual sum of squares of observation $t$.
  • $n_{t}$: Number of eligible assets at observation $t$.
  • $k$: Effective number of regressors.

Arguments

  • B: Exposure history observations × assets × factors, already lagged.
  • f: Factor return matrix observations × factors, already lagged.
  • eps: Residual matrix observations × assets, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • k: Effective number of regressors. The block method takes the factor count of the reduced axis, which is what the fit spent.
  • csfm: A cross-sectional factor model block.

Validation

  • !isempty(B), and every history agrees with B on the axes it shares.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • aic::Vector{<:Real}: One entry per observation, NaN where the eligible asset count does not exceed k.

Examples

julia> B = reshape([1.0, 1.0, 1.0, -1.0, 1.0, 0.0], 1, 3, 2);julia> cs_regression_aic(B, [1.0 0.0], [0.1 0.1 0.1]; k = 1)1-element Vector{Float64}: -11.815510557964274

Related

source
PortfolioOptimisers.cs_regression_bic — Function
cs_regression_bic(
    B::Arr3Num,
    f::MatNum,
    eps::MatNum,
    w::Option{<:MatNum} = nothing;
    k::Integer = size(B, 3)
) -> Vector{<:Real}
cs_regression_bic(csfm::CrossSectionalFactorModel) -> Vector{<:Real}

Return the Bayesian information criterion of every cross-sectional fit, one entry per observation.

The criterion trades the fit of an observation against the size of its design, and a lower value is the better trade. Its penalty grows with the logarithm of the eligible asset count, so it charges a large cross-section more for a regressor than cs_regression_aic does.

Mathematical definition

\[\mathrm{BIC}_{t} = n_{t} \ln (\mathrm{RSS}_{t}) + k \ln (n_{t})\]

Where:

  • $\mathrm{RSS}_{t}$: Weight-normalised residual sum of squares of observation $t$.
  • $n_{t}$: Number of eligible assets at observation $t$.
  • $k$: Effective number of regressors.

Arguments

  • B: Exposure history observations × assets × factors, already lagged.
  • f: Factor return matrix observations × factors, already lagged.
  • eps: Residual matrix observations × assets, already lagged.
  • w: Regression weight history observations × assets, or nothing for equal weights.
  • k: Effective number of regressors. The block method takes the factor count of the reduced axis, which is what the fit spent.
  • csfm: A cross-sectional factor model block.

Validation

  • !isempty(B), and every history agrees with B on the axes it shares.
  • On the block method, csfm.Ms and csfm.csr are not nothing.

Returns

  • bic::Vector{<:Real}: One entry per observation, NaN where the eligible asset count does not exceed k.

Examples

julia> B = reshape([1.0, 1.0, 1.0, -1.0, 1.0, 0.0], 1, 3, 2);julia> cs_regression_bic(B, [1.0 0.0], [0.1 0.1 0.1]; k = 1)1-element Vector{Float64}: -12.716898269296165

Related

source