Factor Model Summary: private API

Types

PortfolioOptimisers.FactorReturnHistory — Type
struct FactorReturnHistory{__T_f, __T_pos}

Holds the factor return history that a summary reads off a factor model block, and the position of each raw factor on the axis of that history.

This type is an implementation detail and is not intended for direct use. factor_summary_returns makes it, and factor_model_summary and the factor diagnostics plots read it.

Fields

  • f: f: Factor return history observations × factors.
  • pos: pos: One entry per raw factor. Entry k is the column of f that holds raw factor k, or 0 when f holds none.

Related

source

Functions

PortfolioOptimisers.factor_summary_finite_column — Function
factor_summary_finite_column(A::MatNum, k::Integer)

Return the present entries of one column of a diagnostic series.

Every column of the summary aggregates a series over the observations. An observation whose answer is absent holds NaN, and it takes no part in the aggregate. This function drops the absent entries once, so each aggregate reads a dense vector and carries no test of its own. An infinite entry is a value and not an absence, so it stays, and the aggregate of its column is infinite too.

Mathematical definition

\[\begin{align} \boldsymbol{v} &= \left(a_{tk}\right)_{t \in \mathcal{T}_{k}}\,. \end{align}\]

Where:

  • $\boldsymbol{v}$: Present entries of column $k$, in the order of the observations.
  • $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$.
  • $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not NaN. An infinite entry is present.

Arguments

  • A: A series observations × factors.
  • k: Position of the factor.

Returns

  • v::Vector{<:Real}: The entries of column k that are not NaN, in the order of the observations.

Related

source
PortfolioOptimisers.factor_summary_column_mean — Function
factor_summary_column_mean(A::MatNum, k::Integer)

Return the mean of the present entries of one column of a diagnostic series.

Mathematical definition

\[\begin{align} \bar{a}_{k} &= \frac{1}{|\mathcal{T}_{k}|} \sum_{t \in \mathcal{T}_{k}} a_{tk}\,. \end{align}\]

Where:

  • $\bar{a}_{k}$: Mean of the present entries of column $k$.
  • $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$.
  • $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not NaN. An infinite entry is present.

The mean is not defined for an empty $\mathcal{T}_{k}$.

Arguments

  • A: A series observations × factors.
  • k: Position of the factor.

Returns

  • m::Real: The mean, and NaN when the column has no present entry.

Related

source
PortfolioOptimisers.factor_summary_column_median — Function
factor_summary_column_median(A::MatNum, k::Integer)

Return the median of the present entries of one column of a diagnostic series.

Mathematical definition

\[\begin{align} \tilde{a}_{k} &= \operatorname{median} \left\{ a_{tk} : t \in \mathcal{T}_{k} \right\}\,. \end{align}\]

Where:

  • $\tilde{a}_{k}$: Median of the present entries of column $k$. For an even count it is the mean of the two middle entries.
  • $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$.
  • $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not NaN. An infinite entry is present.

The median is not defined for an empty $\mathcal{T}_{k}$.

Arguments

  • A: A series observations × factors.
  • k: Position of the factor.

Returns

  • m::Real: The median, and NaN when the column has no present entry.

Related

source
PortfolioOptimisers.factor_summary_ratio — Function
factor_summary_ratio(m::Real, v::Real)

Return a ratio of two summary statistics, and NaN where the ratio has no value.

The summary takes one ratio, the Sharpe ratio, whose denominator is a volatility. A zero volatility, an absent volatility and a division that overflows each leave the ratio without a value, and this function answers NaN for all three.

Mathematical definition

\[\begin{align} r &= \begin{cases} m / v & \text{when } m / v \text{ is finite}\,,\\ \mathrm{NaN} & \text{otherwise}\,. \end{cases} \end{align}\]

Where:

  • $r$: The ratio.
  • $m$: Numerator.
  • $v$: Denominator. A zero or NaN value of $v$ leaves $m / v$ not finite.

Arguments

  • m: Numerator.
  • v: Denominator.

Returns

  • r::Real: The ratio, and NaN when the denominator is zero or absent, or when the ratio is not finite.

Related

source
PortfolioOptimisers.factor_summary_return_stats — Function
factor_summary_return_stats(f::MatNum, ppy::Number)

Return the annualised mean, the annualised volatility and the Sharpe ratio of every factor return series.

An absent factor return takes no part in the mean or the volatility, so a series with a gap still has a summary. The volatility is the corrected sample standard deviation, which needs two observations, and a series with fewer reads NaN. A constant series has a zero volatility, so its Sharpe ratio reads NaN.

Mathematical definition

\[\begin{align} \hat{\mu}_{k} &= \frac{1}{T_{k}} \sum_{t \in \mathcal{T}_{k}} f_{tk}\,,\\ \mathrm{ann\_return}_{k} &= p \, \hat{\mu}_{k}\,,\\ \mathrm{ann\_volatility}_{k} &= \sqrt{\frac{p}{T_{k} - 1} \sum_{t \in \mathcal{T}_{k}} \left(f_{tk} - \hat{\mu}_{k}\right)^{2}}\,,\\ \mathrm{sharpe}_{k} &= \frac{\mathrm{ann\_return}_{k}}{\mathrm{ann\_volatility}_{k}}\,. \end{align}\]

Where:

  • $\hat{\mu}_{k}$: Per-period mean of the present returns of factor $k$.
  • $\mathrm{ann\_return}_{k}$, $\mathrm{ann\_volatility}_{k}$, $\mathrm{sharpe}_{k}$: The three returned entries of factor $k$.
  • $f_{tk}$: Factor return of factor $k$ at observation $t$, an entry of the factor return history.
  • $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not NaN. An infinite entry is present.
  • $T_{k} = |\mathcal{T}_{k}|$: Count of the present returns of factor $k$.
  • $p$: Periods per year, ppy.

The mean is not defined for $T_{k} = 0$, the volatility is not defined for $T_{k} < 2$, and the Sharpe ratio is not defined where the volatility is zero.

Algorithm

For each factor k:

  1. Drop the absent returns of column k with factor_summary_finite_column, giving v.
  2. Take the mean of v and clamp it to the least and the greatest entry of v, giving m. The clamp changes nothing in exact arithmetic. The rounded mean of a constant v can differ from its common value, and the clamp makes m equal to that value, so every deviation in step 4 is exactly zero.
  3. Multiply m by ppy, giving ann_return[k]. An empty v gives NaN.
  4. Take the corrected sample standard deviation of v about m and multiply it by the square root of ppy, giving ann_volatility[k]. A v with fewer than two entries gives NaN.
  5. Divide the two with factor_summary_ratio, giving sharpe[k].

Arguments

  • f: Factor return history observations × factors.
  • ppy: Periods per year.

Returns

  • ann_return::Vector{<:Real}: One entry per factor.
  • ann_volatility::Vector{<:Real}: One entry per factor.
  • sharpe::Vector{<:Real}: One entry per factor.

Related

source
PortfolioOptimisers.factor_summary_autocorrelation — Function
factor_summary_autocorrelation(f::MatNum)

Return the lag-one autocorrelation of every factor return series.

The answer is the Pearson correlation of the pair (f[1:end - 1, k], f[2:end, k]), with each half centred on its own mean. StatsBase.autocor is a different estimator. It centres both halves on the mean of the whole series and divides by the sum of squares of the whole series. The two agree in the limit and differ on a short series, so a caller who wants the other definition calls StatsBase.autocor on the factor return history itself.

Mathematical definition

\[\begin{align} \bar{a}_{k} &= \frac{1}{T - 1} \sum_{t = 1}^{T - 1} f_{tk}\,,\\ \bar{b}_{k} &= \frac{1}{T - 1} \sum_{t = 2}^{T} f_{tk}\,,\\ \rho_{k} &= \frac{\sum_{t = 1}^{T - 1} \left(f_{tk} - \bar{a}_{k}\right)\left(f_{(t + 1)k} - \bar{b}_{k}\right)}{\sqrt{\sum_{t = 1}^{T - 1} \left(f_{tk} - \bar{a}_{k}\right)^{2} \sum_{t = 1}^{T - 1} \left(f_{(t + 1)k} - \bar{b}_{k}\right)^{2}}}\,. \end{align}\]

Where:

  • $\rho_{k}$: Lag-one autocorrelation of factor $k$.
  • $\bar{a}_{k}$: Mean of the leading half, the first $T - 1$ returns of factor $k$.
  • $\bar{b}_{k}$: Mean of the trailing half, the last $T - 1$ returns of factor $k$.
  • $f_{tk}$: Factor return of factor $k$ at observation $t$, an entry of the factor return history.
  • $T$: Number of observations.

The coefficient is not defined for $T < 2$, nor for a series with a constant half, whose denominator is zero. An absent return makes the mean of its half absent, so the coefficient of that series is absent too.

Algorithm

  1. Return NaN for every factor when T < 2.
  2. For each factor k, sum each half and track the least and the greatest entry of each half.
  3. Divide each sum by T - 1 and clamp it to the least and the greatest entry of its half, giving ma and mb. The clamp changes nothing in exact arithmetic. It makes the mean of a constant half equal to its common value, so the deviations of that half are exactly zero and the coefficient is NaN.
  4. Accumulate the cross product cab and the two sums of squares caa and cbb of the deviations.
  5. Divide cab by the square root of caa * cbb, giving ac[k].

Arguments

  • f: Factor return history observations × factors.

Returns

  • autocorr::Vector{<:Real}: One entry per factor, and NaN for a series of one observation, a series with a constant half and a series with an absent return.

Related

source
PortfolioOptimisers.factor_summary_returns — Function
factor_summary_returns(csfm::CrossSectionalFactorModel) -> FactorReturnHistory
factor_summary_returns(csfm::CrossSectionalFactorModel, csr::Nothing, fr)
factor_summary_returns(csfm::CrossSectionalFactorModel, csr::CrossSectionalRegression,
                       fr::Nothing) -> FactorReturnHistory
factor_summary_returns(csfm::CrossSectionalFactorModel, csr::CrossSectionalRegression,
                       fr::MatNum) -> FactorReturnHistory

Return the factor return history that a summary reads off a factor model block, and the position of each raw factor on the axis of that history.

A block that carries fr states its factor returns on the raw axis, so the history is fr and the position of raw factor k is k. That covers a factor a family re-basis dropped, whose return the constraint of its family fixes. A block with no fr gives the reduced axis, which is the axis the fit estimated in, with the observed factors after the estimated ones, as cross_sectional_factor_returns states it. The positions are then those of factor_summary_positions, so a dropped factor takes 0. factor_model_summary writes each statistic of the history onto the raw axis with factor_summary_mapped. A separate method handles a block with no fit, and its message names the field the caller must populate.

Arguments

  • csfm: A cross-sectional factor model block.
  • csr: The csr field of the block, or nothing.
  • fr: The fr field of the block, or nothing.

Validation

  • csfm.csr is not nothing, else the function raises an IsNothingError that names csr.
  • The rules of factor_summary_positions when the block carries no fr.

Returns

  • h::FactorReturnHistory: The history f::MatNum, observations × factors, and the positions pos::Vector{Int}, as FactorReturnHistory states them.

Related

source
PortfolioOptimisers.factor_summary_positions — Function
factor_summary_positions(csfm::CrossSectionalFactorModel)
factor_summary_positions(fcb::Nothing, nf::Nothing, K::Integer)
factor_summary_positions(fcb::Nothing, nf::VecStr, K::Integer)
factor_summary_positions(fcb::AbstractFactorFamilyBasis, nf::Nothing, K::Integer)
factor_summary_positions(fcb::FactorFamilyBasis, nf::VecStr, K::Integer)

Return the position of each raw factor on the reduced factor axis.

The regression group answers on the reduced axis and the summary answers on the raw one, so the summary joins the two by name. A raw factor that the re-basis dropped takes the position 0, and its Gram columns read NaN for that reason. A block with no re-basis needs no join, because its two axes are the same axis.

Mathematical definition

\[\begin{align} \pi_{k} &= \begin{cases} j & \text{when } n_{k} = r_{j}\,,\\ 0 & \text{when } n_{k} \notin \left\{r_{1}, \ldots, r_{K_{r}}\right\}\,. \end{cases} \end{align}\]

Where:

  • $\pi_{k}$: Reduced position of raw factor $k$, its position on the reduced factor axis, and $0$ when the family re-basis dropped it.
  • $n_{k}$: Name of raw factor $k$.
  • $r_{j}$: Name of reduced factor $j$.
  • $K_{r}$: Number of reduced factors.

A block with no re-basis has $\pi_{k} = k$.

Arguments

  • csfm: A cross-sectional factor model block.
  • fcb: The fcb field of the block, or nothing.
  • nf: The nf field of the block, or nothing.
  • K: Number of raw factors.

Validation

  • csfm.nf is not nothing when csfm.fcb is present, else the function raises an IsNothingError that names nf. The summary cannot join a re-based block to the raw axis without the names.

Returns

  • pos::Vector{Int}: One entry per raw factor. Entry k is the position of raw factor k on the reduced axis, or 0 when the re-basis dropped it.

Related

source
PortfolioOptimisers.factor_summary_mapped — Function
factor_summary_mapped(v::VecNum, pos::AbstractVector{Int})

Return a statistic of the axis of a factor return history, written onto the raw factor axis.

The axis is the reduced one, or the raw one when the block carries fr, and pos names it.

Mathematical definition

\[\begin{align} m_{k} &= \begin{cases} v_{\pi_{k}} & \text{when } 0 < \pi_{k} \leq n\,,\\ \mathrm{NaN} & \text{otherwise}\,. \end{cases} \end{align}\]

Where:

  • $m_{k}$: The statistic of raw factor $k$.
  • $v_{j}$: The statistic of factor $j$ of the axis.
  • $\pi_{k}$: Position of raw factor $k$ on the axis, and $0$ when the axis holds none.
  • $n$: Length of $\boldsymbol{v}$. A statistic of the regression covers the estimated factors alone, which come before the observed ones, so the position of an observed factor lies past $n$.

Arguments

  • v: A statistic, one entry per factor of the axis, or per estimated reduced factor.
  • pos: Position of each raw factor on the axis of v, 0 where that axis holds none, as factor_summary_returns gives it.

Returns

  • m::Vector{<:Real}: One entry per raw factor, and NaN at a factor whose position is 0 or that v does not cover.

Related

source
PortfolioOptimisers.factor_summary_gram — Function
factor_summary_gram(csfm::CrossSectionalFactorModel, threshold::Number)
factor_summary_gram(Ms::Nothing, csfm::CrossSectionalFactorModel, threshold::Number)
factor_summary_gram(Ms::Arr3Num, csfm::CrossSectionalFactorModel, threshold::Number)

Return the three regression columns of a factor model summary, on the raw factor axis.

The columns are the mean absolute t-statistic, the rate at which the absolute t-statistic passes threshold, and the mean variance inflation factor. Each one is the time average of a series of the regression group, joined back onto the raw factor axis by name.

A block with no exposure history has no regression design either, so all three columns are absent, and a separate method handles that case.

Algorithm

  1. Take the position of each raw factor on the reduced axis with factor_summary_positions, giving pos.
  2. Take the t-statistic history with cs_regression_t_stats, giving t, and the variance inflation factor history with exposure_vif, giving vif.
  3. Take the exceedance rate at threshold with cs_regression_t_stat_exceedance_rate, giving rate.
  4. For each reduced factor, take the mean of the present entries of abs.(t) and of vif with factor_summary_column_mean, giving abs_t and mvif.
  5. Write abs_t, rate and mvif onto the raw axis with factor_summary_mapped.

Arguments

  • csfm: A cross-sectional factor model block.
  • Ms: The Ms field of the block, or nothing.
  • threshold: Absolute t-statistic that the exceedance rate counts against.

Returns

  • mean_abs_t::Option{<:Vector{<:Real}}: One entry per raw factor, or nothing.
  • t_rate::Option{<:Vector{<:Real}}: One entry per raw factor, or nothing.
  • mean_vif::Option{<:Vector{<:Real}}: One entry per raw factor, or nothing.

Related

source
PortfolioOptimisers.factor_summary_exposure_variance — Function
factor_summary_exposure_variance(Ms::Arr3Num, t::Integer, k::Integer)

Return the cross-sectional variance of one factor exposure at one observation.

The variance is uncorrected, and it reads the exposures that are not NaN.

Mathematical definition

\[\begin{align} \bar{b}_{tk} &= \frac{1}{|\mathcal{N}_{tk}|} \sum_{i \in \mathcal{N}_{tk}} b_{tik}\,,\\ s^{2}_{tk} &= \frac{1}{|\mathcal{N}_{tk}|} \sum_{i \in \mathcal{N}_{tk}} \left(b_{tik} - \bar{b}_{tk}\right)^{2}\,. \end{align}\]

Where:

  • $s^{2}_{tk}$: Uncorrected cross-sectional variance of the exposure to factor $k$ at observation $t$, over the assets at which that exposure is not NaN.
  • $\bar{b}_{tk}$: Cross-sectional mean of the exposure to factor $k$ at observation $t$.
  • $b_{tik}$: Exposure of asset $i$ to factor $k$ at observation $t$.
  • $\mathcal{N}_{tk}$: Assets at which $b_{tik}$ is not NaN.

The variance is not defined for an empty $\mathcal{N}_{tk}$. An infinite exposure makes the deviations undefined, so the variance reads NaN.

Algorithm

  1. Sum the exposures of the cross-section that are not NaN, count them, and track their least and greatest value.
  2. Return NaN when the count is zero.
  3. Divide the sum by the count and clamp it to the least and the greatest value, giving m. The clamp changes nothing in exact arithmetic. It makes the mean of a constant cross-section equal to its common value, so the variance of that cross-section is exactly zero.
  4. Average the squared deviations from m, giving the variance.

Arguments

  • Ms: Exposure history assets × factors × observations, unlagged.
  • t: Position of the observation.
  • k: Position of the factor.

Returns

  • v::Real: The variance, and NaN when the cross-section has no exposure that is not NaN.

Related

source
PortfolioOptimisers.factor_summary_constant_exposures — Function
factor_summary_constant_exposures(Ms::Arr3Num)

Return which factor exposures never vary across the cross-section.

The global intercept and the constant column of a one-hot family are constant exposures. The cross-section of a constant exposure has no spread, so its stability coefficient is not defined, and the summary writes 1 in its place. An exposure that never moves is perfectly stable. The summary applies that patch, and exposure_stability keeps the NaN that the correlation gives.

Mathematical definition

\[\begin{align} c_{k} &= \begin{cases} \max_{t \in \mathcal{V}_{k}} s^{2}_{tk} < 10^{-12} & \text{when } \mathcal{V}_{k} \neq \emptyset\,,\\ \mathrm{false} & \text{when } \mathcal{V}_{k} = \emptyset\,. \end{cases} \end{align}\]

Where:

  • $c_{k}$: Constant-exposure flag of factor $k$, true when the largest $s^{2}_{tk}$ over the observations is under $10^{-12}$.
  • $s^{2}_{tk}$: Uncorrected cross-sectional variance of the exposure to factor $k$ at observation $t$, over the assets at which that exposure is not NaN.
  • $\mathcal{V}_{k}$: Observations at which $s^{2}_{tk}$ is not NaN.

A factor with no exposure anywhere is not constant.

Arguments

  • Ms: Exposure history assets × factors × observations, unlagged.

Returns

  • c::BitVector: One entry per factor, true when the largest cross-sectional variance of the factor is under 1e-12.

Related

source
PortfolioOptimisers.factor_summary_stability — Function
factor_summary_stability(Ms::Arr3Num, csfm::CrossSectionalFactorModel; step::Integer,
                         weighting)

Return the stability column of a factor model summary, on the raw factor axis.

The column is the median over the observations of exposure_stability, and a constant exposure reads 1. A history with no more observations than step has no stability series, so every factor reads NaN except the constant ones, which still read 1.

Mathematical definition

\[\begin{align} \mathrm{stability}_{k} &= \begin{cases} 1 & \text{when } c_{k}\,,\\ \operatorname{median} \left\{ a_{tk} : t \in \mathcal{T}_{k} \right\} & \text{otherwise}\,. \end{cases} \end{align}\]

Where:

  • $\mathrm{stability}_{k}$: The returned entry of factor $k$.
  • $c_{k}$: Constant-exposure flag of factor $k$, true when the largest $s^{2}_{tk}$ over the observations is under $10^{-12}$.
  • $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$. Here the series is the stability coefficient that exposure_stability returns.
  • $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not NaN. An infinite entry is present.

The median is not defined for an empty $\mathcal{T}_{k}$, and that is the case for every factor when the history has no more observations than step.

Arguments

  • Ms: Exposure history assets × factors × observations, unlagged.
  • csfm: A cross-sectional factor model block.
  • step: Number of observations between the two cross-sections that the coefficient reads.
  • weighting: The AbstractOrthogonalityMetric whose weight history the coefficient reads.

Returns

  • stability::Vector{<:Real}: One entry per raw factor.

Related

source
PortfolioOptimisers.factor_summary_exposure — Function
factor_summary_exposure(Ms::Nothing, csfm::CrossSectionalFactorModel, step::Integer,
                        weighting, coverage_weighting)
factor_summary_exposure(Ms::Arr3Num, csfm::CrossSectionalFactorModel, step::Integer,
                        weighting, coverage_weighting)

Return the two exposure columns of a factor model summary, on the raw factor axis.

The columns are the median exposure stability and the average coverage. Both read the unlagged exposure history, as the whole exposure group does. A block with no exposure history has neither column, and a separate method handles that case.

The two columns read different weight histories. The stability reads the history that weighting names, and the coverage reads the history that coverage_weighting names.

Arguments

  • csfm: A cross-sectional factor model block.
  • Ms: The Ms field of the block, or nothing.
  • step: Number of observations between the two cross-sections that the stability reads.
  • weighting: The AbstractOrthogonalityMetric that the stability reads.
  • coverage_weighting: The AbstractOrthogonalityMetric whose positive weights are the universe of the coverage.

Returns

  • stability::Option{<:Vector{<:Real}}: One entry per raw factor, or nothing.
  • coverage::Option{<:Vector{<:Real}}: One entry per raw factor, or nothing.

Related

source
PortfolioOptimisers.assert_factor_summary_columns — Function
assert_factor_summary_columns(fs::FactorSummaryResult)

Check the columns of a FactorSummaryResult that its constructor built.

The constructor builds the Result first and checks it here, so the check reads each column by its name and not by its position among twelve arguments.

Arguments

  • fs: The summary to check.

Validation

  • Each field but ppy is nothing or holds as many entries as ann_return. Raises a DimensionMismatch.
  • ppy > 0. Raises a DomainError.
  • The rules of assert_nan_or_nonneg on ann_volatility and mean_abs_t, and of assert_nan_or_closed_unit_interval on t_rate and coverage. An absent column is nothing, and those rules check nothing on it.

Returns

  • nothing.

Related

source