Cross-Sectional Regression Diagnostics: private API

cs_estimation_mask_weights(B::Arr3Num, w::Nothing)
cs_estimation_mask_weights(B::Arr3Num, w::MatNum)

Return the eligibility mask of a cross-sectional regression history, and the weights that go with it.

An asset enters the fit of an observation when every one of its exposures is finite, and when the weight it carries is positive. The two answers travel together because every diagnostic of this file needs both: the mask counts the assets, and the weights scale the sums. An absent weight matrix is the dispatch rather than a branch, so no caller writes an isnothing test.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• w: Regression weight history observations × assets, or nothing for equal weights.

Validation

• !isempty(B).
• size(w) == (size(B, 1), size(B, 2)), when w is present.

Returns

• mask::Matrix{Bool}: observations × assets. Entry (t, i) is true when asset i enters the fit of observation t.
• u::Matrix{<:Real}: observations × assets. The weight of an eligible pair, and zero outside the mask.

Related

• cs_gram
• exposure_vif
• cs_regression_t_stats

cs_gram_from_weights(B::Arr3Num, u::MatNum)

Return the weighted Gram history from a mask that a caller has already resolved.

cs_gram resolves the mask itself and calls this method. A diagnostic that has already built its own weights — one that also excludes a pair whose residual is not finite — calls this method instead, so the mask is resolved once per diagnostic rather than twice.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• u: Resolved weight history observations × assets, zero outside the mask.

Returns

• G::Array{<:Real, 3}: Gram history observations × factors × factors.

Related

• cs_gram
• cs_estimation_mask_weights

cs_gram_inverse_diagonal(G::Arr3Num) -> Matrix{<:Real}

Return the diagonal of the inverse of every slice of a Gram history.

The inverse is taken slice by slice, through the singular value decomposition of the slice. That is one formulation for both cases a design presents: a full-rank slice gets its inverse, and a collinear slice gets its pseudo-inverse, because a direction whose singular value falls under the tolerance contributes nothing. A branch that inverts and falls back on failure answers the same, and it costs the analysis of this file the whole body of the pseudo-inverse.

The diagonal is the only part any diagnostic of this file reads: the variance inflation factor multiplies it by the diagonal of the slice itself, and the standard error of a factor return scales it by the residual variance.

Mathematical definition

\[(\mathbf{G}_{t}^{+})_{kk} = \sum_{i \,:\, \sigma_{i} > \tau} \frac{u_{ki} \, v_{ki}}{\sigma_{i}}\]

Where:

• $\sigma_{i}$, $\boldsymbol{u}_{i}$, $\boldsymbol{v}_{i}$: $i$-th singular value and the two singular vectors of the slice.
• $\tau = K \, \varepsilon \, \max_{i} \sigma_{i}$: Tolerance below which a direction is dropped. It is the one the pseudo-inverse of the standard library applies.
• $K$: Number of factors.

Arguments

• G: Gram history observations × factors × factors.

Returns

• D::Matrix{<:Real}: observations × factors. Row t is the diagonal of the inverse of slice t.

Related

• cs_gram
• cs_gram_slice!
• cs_inverse_diagonal!
• exposure_vif
• cs_regression_t_stats

cs_gram_slice!(Gt::AbstractMatrix, G::Arr3Num, t::Integer)

Copy one slice of a Gram history into a working matrix.

The two verbs that read a slice — cs_gram_inverse_diagonal and exposure_condition_number — each need it as a matrix of one concrete element type, and each reuses one buffer across the observations rather than allocating per slice.

Arguments

• Gt: Working matrix factors × factors, written in place.
• G: Gram history observations × factors × factors.
• t: Observation to copy.

Returns

• nothing.

Related

• cs_gram
• cs_gram_inverse_diagonal
• exposure_condition_number

cs_inverse_diagonal!(D::AbstractMatrix, Gt::AbstractMatrix, t::Integer)

Write the diagonal of the inverse of one Gram slice into the answer.

The inverse is the sum over the singular directions of the outer product of the two singular vectors, scaled by the reciprocal singular value. A direction whose singular value falls under the tolerance is dropped, which makes the answer the pseudo-inverse of a collinear slice and the inverse of a full-rank one.

Arguments

• D: Answer observations × factors, written in place.
• Gt: One Gram slice factors × factors.
• t: Observation to write.

Returns

• nothing.

Related

• cs_gram_inverse_diagonal
• cs_gram_slice!

cs_regression_data(csfm::CrossSectionalFactorModel)

Return the lag-aligned regression history a cross-sectional diagnostic reads off a factor model block.

Every level-2 diagnostic of this file starts here. The exposures are trimmed at the tail and the return-like histories at the head, so the exposures of observation $t - \ell$ line up with the returns of observation $t$. When the block carries a family re-basis, the exposures and the factor returns are mapped onto the reduced axis, and every answer that carries a factor axis is then on the reduced axis too.

Algorithm

1. Refuse a block that carries no exposure history, or no cross-sectional fit.
2. Trim the exposure history at the tail by csfm.lag, and the factor returns, the residuals and the regression weights at the head by the same count.
3. When csfm.fcb is set, slice the basis to the trimmed observation axis, map the exposures through reduce_exposures, and map the factor returns through reduce_factor_returns.

Arguments

• csfm: A cross-sectional factor model block.

Validation

• csfm.Ms is not nothing, else an IsNothingError naming Ms is raised.
• csfm.csr is not nothing, else an IsNothingError naming csr is raised.

Returns

• data::NamedTuple: (; B, f, eps, w), the lag-aligned exposures, factor returns, residuals and regression weights. w is nothing when the block carries no regression weight history.

Related

• CrossSectionalFactorModel
• reduce_exposures
• reduce_factor_returns
• cs_regression_t_stats

cs_regression_lag(lag::Nothing)
cs_regression_lag(lag::Integer)

Return the exposure lag of a factor model block as a count.

A block that states no lag lags by nothing, so the two cases answer 0 and the stated count. The shape is the dispatch, which keeps the isnothing test out of every caller.

Arguments

• lag: The lag field of a CrossSectionalFactorModel, or nothing.

Returns

• lag::Int: The lag as a count.

Related

• CrossSectionalFactorModel
• cs_regression_data

cs_lagged_rows(A::Nothing, rows)
cs_lagged_rows(A::MatNum, rows)

Trim a per-asset history at the head, keeping the rows a lag alignment leaves.

Arguments

• A: A per-asset history observations × assets, or nothing.
• rows: Indices of the observations to keep.

Returns

• A::Option{<:MatNum}: The trimmed history, or nothing when the block carries none.

Related

• cs_regression_data

cs_reduce_regression(fcb::Nothing, B::Arr3Num, f::MatNum, lag::Integer)
cs_reduce_regression(fcb::FactorFamilyBasis, B::Arr3Num, f::MatNum, lag::Integer)

Map a lag-aligned regression history onto the reduced factor axis of a family re-basis.

A block that carries no re-basis is already on its own axis, so that case returns the pair unchanged. A block that carries one has a rank-deficient design on the raw axis, because every constrained family sums to zero, so the diagnostics are answered on the reduced axis instead. The basis is sliced to the trimmed observation axis before it maps the exposures, because the exposures were trimmed at the tail.

Arguments

• fcb: The fcb field of a CrossSectionalFactorModel, or nothing.
• B: Lag-aligned exposure history observations × assets × factors.
• f: Lag-aligned factor return matrix observations × factors.
• lag: Number of observations by which the exposures lag the returns.

Returns

• B::Arr3Num: The exposure history, on the reduced axis when a re-basis is set.
• f::MatNum: The factor returns, on the reduced axis when a re-basis is set.

Related

• FactorFamilyBasis
• reduce_exposures
• reduce_factor_returns
• cs_regression_data

cs_masked_vif(B::Arr3Num, mask::AbstractMatrix{Bool}, u::MatNum)

Return the variance inflation factors of a design whose mask a caller has already resolved.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• mask: Eligibility mask observations × assets.
• u: Resolved weight history observations × assets, zero outside the mask.

Returns

• vif::Matrix{<:Real}: observations × factors, NaN at an observation with no degrees of freedom.

Related

• exposure_vif

cs_masked_condition_number(B::Arr3Num, mask::AbstractMatrix{Bool}, u::MatNum)

Return the condition numbers of a design whose mask a caller has already resolved.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• mask: Eligibility mask observations × assets.
• u: Resolved weight history observations × assets, zero outside the mask.

Returns

• kappa::Vector{<:Real}: One entry per observation, NaN at an observation with no degrees of freedom.

Related

• exposure_condition_number

cs_diagnostic_mask_weights(B::Arr3Num, eps::MatNum, w::Option{<:MatNum})

Return the eligibility mask of a cross-sectional regression diagnostic, and the weights that go with it.

A diagnostic of the fit reads the residual of every pair it counts, so a pair whose residual is not finite leaves the mask on top of the rule cs_estimation_mask_weights states. This is the mask every level-2 verb of this file resolves before it calls its level-1 method, which is why a level-2 answer reproduces the fit rather than the exposure history alone.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• eps: Residual matrix observations × assets, already lagged.
• w: Regression weight history observations × assets, or nothing for equal weights.

Validation

• size(eps) == (size(B, 1), size(B, 2)).

Returns

• mask::Matrix{Bool}: observations × assets.
• u::Matrix{<:Real}: observations × assets, zero outside the mask.

Related

• cs_estimation_mask_weights
• cs_regression_t_stats

cs_row_is_finite(A::MatNum, t::Integer)

Return whether every entry of one row of a history is finite.

A cross-sectional fit whose factor returns are not all finite has no t-statistic at that observation, and this is the test that says so.

Arguments

• A: A history observations × columns.
• t: Observation to read.

Returns

• val::Bool: true when every entry of the row is finite.

Related

• cs_regression_t_stats

cs_weighted_rss(eps::MatNum, u::MatNum, mask::AbstractMatrix{Bool}, t::Integer, ::Type{T})

Return the weighted residual sum of squares of one observation.

The weights are the resolved ones rather than the normalised ones, which is what the standard error of a factor return divides by its degrees of freedom. cs_regression_score_parts normalises instead, because a score compares observations of different sizes.

Arguments

• eps: Residual matrix observations × assets.
• u: Resolved weight history observations × assets, zero outside the mask.
• mask: Eligibility mask observations × assets.
• t: Observation to read.
• T: Element type of the answer.

Returns

• rss::Real: The weighted residual sum of squares of observation t.

Related

• cs_regression_t_stats
• cs_regression_score_parts

cs_t_stat_row!(t::MatNum, f::MatNum, D::MatNum, s2, tt::Integer)

Write the t-statistics of one observation into the answer.

A factor whose standard error is zero keeps the absent answer the caller filled the row with, so this writes only the entries that have one.

Arguments

• t: T-statistic matrix observations × factors, written in place.
• f: Factor return matrix observations × factors.
• D: Diagonal of the inverse Gram, observations × factors.
• s2: Residual variance of observation tt.
• tt: Observation to write.

Returns

• nothing.

Related

• cs_regression_t_stats
• cs_gram_inverse_diagonal

cs_resolved_gram(G::Nothing, B::Arr3Num, u::MatNum)
cs_resolved_gram(G::Arr3Num, B::Arr3Num, u::MatNum)

Return the Gram history a diagnostic reads, building it only when the caller supplied none.

Arguments

• G: Gram history the caller holds, or nothing.
• B: Exposure history observations × assets × factors, already lagged.
• u: Resolved weight history observations × assets, zero outside the mask.

Returns

• G::Arr3Num: The supplied history, or the one cs_gram_from_weights builds.

Related

• cs_gram
• cs_regression_t_stats

cs_regression_score_parts(B::Arr3Num, f::MatNum, eps::MatNum, w::Option{<:MatNum})

Return the pieces every cross-sectional regression score is built from.

The four scores read the same three quantities: the eligible asset count, the weight-normalised residual sum of squares, and the coefficient of determination. They are computed here once per score rather than held on a Result, because #709 refused a grouped Result: the scores are independent statistics with no identity between them.

The mask of a score is the finiteness of the asset return, which the exposures, the factor returns and the residuals reconstruct. That differs from the mask of a Gram diagnostic, which reads the exposures and the residuals separately.

Algorithm

1. Reconstruct the asset returns as the systematic part plus the residual.
2. Mask a pair whose reconstructed return is not finite, and a pair whose weight is not positive.
3. Normalise the weights of every observation to sum to one.
4. Take the weighted residual sum of squares, the weighted mean return and the weighted total sum of squares.
5. Answer the coefficient of determination as one less the ratio of the two sums, and NaN where the total sum of squares is zero.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• f: Factor return matrix observations × factors, already lagged.
• eps: Residual matrix observations × assets, already lagged.
• w: Regression weight history observations × assets, or nothing for equal weights.

Validation

• !isempty(B), and every history agrees with B on the axes it shares.

Returns

• n::Vector{Int}: Eligible asset count of every observation.
• rss::Vector{<:Real}: Weight-normalised residual sum of squares of every observation.
• r2::Vector{<:Real}: Coefficient of determination of every observation.

Related

• cs_regression_r2
• cs_regression_adjusted_r2
• cs_regression_aic
• cs_regression_bic

cs_score_observation!(
    r::AbstractVector,
    q::AbstractVector,
    B::Arr3Num,
    f::MatNum,
    eps::MatNum,
    u0::MatNum,
    t::Integer
)

Reconstruct the asset returns of one observation, and normalise its weights to sum to one.

The mask of a cross-sectional regression score is the finiteness of the reconstructed asset return, which differs from the mask of a Gram diagnostic. This is where that mask is applied: an ineligible pair leaves with a zero return and a zero weight, so every sum the score takes over the cross-section ignores it.

Arguments

• r: Reconstructed asset returns of the observation, written in place.
• q: Normalised weights of the observation, written in place.
• B: Exposure history observations × assets × factors, already lagged.
• f: Factor return matrix observations × factors, already lagged.
• eps: Residual matrix observations × assets, already lagged.
• u0: Regression weights observations × assets, zero where the weight is not positive.
• t: Observation to read.

Returns

• n::Int: Number of eligible assets at observation t.

Related

• cs_regression_score_parts
• cs_weighted_score_sums

cs_systematic_return(B::Arr3Num, f::MatNum, t::Integer, i::Integer, ::Type{T})

Return the part of one asset's return the factors of one observation span.

Arguments

• B: Exposure history observations × assets × factors, already lagged.
• f: Factor return matrix observations × factors, already lagged.
• t: Observation to read.
• i: Asset to read.
• T: Element type of the answer.

Returns

• sys::Real: The exposures of asset i at observation t, weighted by the factor returns of that observation.

Related

• cs_score_observation!
• cs_regression_score_parts

cs_weighted_score_sums(r::AbstractVector, q::AbstractVector, eps::MatNum, t::Integer)

Return the weighted residual and total sums of squares of one observation.

The two sums are the numerator and the denominator of the coefficient of determination, and both read the weights cs_score_observation! normalised.

Arguments

• r: Reconstructed asset returns of the observation, zero outside the mask.
• q: Normalised weights of the observation, zero outside the mask.
• eps: Residual matrix observations × assets, already lagged.
• t: Observation to read.

Returns

• rss::Real: Weighted residual sum of squares.
• tss::Real: Weighted total sum of squares about the weighted mean return.

Related

• cs_regression_score_parts
• cs_score_observation!

cs_estimation_weights_only(w::Nothing, T::Integer, N::Integer, ::Type{Tf})
cs_estimation_weights_only(w::MatNum, T::Integer, N::Integer, ::Type{Tf})

Return the regression weights of a score, without reading the exposures.

A cross-sectional regression score masks on the reconstructed asset return rather than on the exposures, so it resolves its weights here rather than through cs_estimation_mask_weights. An absent weight matrix gives every pair a unit weight.

Arguments

• w: Regression weight history observations × assets, or nothing for equal weights.
• T: Number of observations.
• N: Number of assets.
• Tf: Element type of the answer.

Validation

• size(w) == (T, N), when w is present.

Returns

• u::Matrix{<:Real}: observations × assets. A non-positive weight reads back as zero.

Related

• cs_regression_score_parts
• cs_estimation_mask_weights

cs_diagnostic_factor_names(csfm::CrossSectionalFactorModel)

Return the factor names of the axis a cross-sectional regression diagnostic answers on.

A diagnostic that carries a factor axis answers on the reduced axis when the block carries a family re-basis, so the names of the raw axis do not label it. This verb maps them, and it is what a plot reads to label its axis. A block that names no factor answers nothing, and the caller then labels the axis by position.

Arguments

• csfm: A cross-sectional factor model block.

Returns

• nf::Option{<:Vector{String}}: The names of the answer's factor axis, or nothing when the block names no factor.

Related

• CrossSectionalFactorModel
• reduce_factor_names
• cs_regression_t_stats
• exposure_vif