Factor Model Summary: private API
Types
PortfolioOptimisers.FactorReturnHistory — Type
struct FactorReturnHistory{__T_f, __T_pos}Holds the factor return history that a summary reads off a factor model block, and the position of each raw factor on the axis of that history.
This type is an implementation detail and is not intended for direct use. factor_summary_returns makes it, and factor_model_summary and the factor diagnostics plots read it.
Fields
f:f: Factor return historyobservations × factors.
pos:pos: One entry per raw factor. Entrykis the column offthat holds raw factork, or0whenfholds none.
Related
Functions
PortfolioOptimisers.factor_summary_finite_column — Function
factor_summary_finite_column(A::MatNum, k::Integer)Return the present entries of one column of a diagnostic series.
Every column of the summary aggregates a series over the observations. An observation whose answer is absent holds NaN, and it takes no part in the aggregate. This function drops the absent entries once, so each aggregate reads a dense vector and carries no test of its own. An infinite entry is a value and not an absence, so it stays, and the aggregate of its column is infinite too.
Mathematical definition
\[\begin{align} \boldsymbol{v} &= \left(a_{tk}\right)_{t \in \mathcal{T}_{k}}\,. \end{align}\]
Where:
- $\boldsymbol{v}$: Present entries of column $k$, in the order of the observations.
- $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$.
- $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not
NaN. An infinite entry is present.
Arguments
A: A seriesobservations × factors.k: Position of the factor.
Returns
v::Vector{<:Real}: The entries of columnkthat are notNaN, in the order of the observations.
Related
PortfolioOptimisers.factor_summary_column_mean — Function
factor_summary_column_mean(A::MatNum, k::Integer)Return the mean of the present entries of one column of a diagnostic series.
Mathematical definition
\[\begin{align} \bar{a}_{k} &= \frac{1}{|\mathcal{T}_{k}|} \sum_{t \in \mathcal{T}_{k}} a_{tk}\,. \end{align}\]
Where:
- $\bar{a}_{k}$: Mean of the present entries of column $k$.
- $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$.
- $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not
NaN. An infinite entry is present.
The mean is not defined for an empty $\mathcal{T}_{k}$.
Arguments
A: A seriesobservations × factors.k: Position of the factor.
Returns
m::Real: The mean, andNaNwhen the column has no present entry.
Related
PortfolioOptimisers.factor_summary_column_median — Function
factor_summary_column_median(A::MatNum, k::Integer)Return the median of the present entries of one column of a diagnostic series.
Mathematical definition
\[\begin{align} \tilde{a}_{k} &= \operatorname{median} \left\{ a_{tk} : t \in \mathcal{T}_{k} \right\}\,. \end{align}\]
Where:
- $\tilde{a}_{k}$: Median of the present entries of column $k$. For an even count it is the mean of the two middle entries.
- $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$.
- $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not
NaN. An infinite entry is present.
The median is not defined for an empty $\mathcal{T}_{k}$.
Arguments
A: A seriesobservations × factors.k: Position of the factor.
Returns
m::Real: The median, andNaNwhen the column has no present entry.
Related
PortfolioOptimisers.factor_summary_ratio — Function
factor_summary_ratio(m::Real, v::Real)Return a ratio of two summary statistics, and NaN where the ratio has no value.
The summary takes one ratio, the Sharpe ratio, whose denominator is a volatility. A zero volatility, an absent volatility and a division that overflows each leave the ratio without a value, and this function answers NaN for all three.
Mathematical definition
\[\begin{align} r &= \begin{cases} m / v & \text{when } m / v \text{ is finite}\,,\\ \mathrm{NaN} & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $r$: The ratio.
- $m$: Numerator.
- $v$: Denominator. A zero or
NaNvalue of $v$ leaves $m / v$ not finite.
Arguments
m: Numerator.v: Denominator.
Returns
r::Real: The ratio, andNaNwhen the denominator is zero or absent, or when the ratio is not finite.
Related
PortfolioOptimisers.factor_summary_return_stats — Function
factor_summary_return_stats(f::MatNum, ppy::Number)Return the annualised mean, the annualised volatility and the Sharpe ratio of every factor return series.
An absent factor return takes no part in the mean or the volatility, so a series with a gap still has a summary. The volatility is the corrected sample standard deviation, which needs two observations, and a series with fewer reads NaN. A constant series has a zero volatility, so its Sharpe ratio reads NaN.
Mathematical definition
\[\begin{align} \hat{\mu}_{k} &= \frac{1}{T_{k}} \sum_{t \in \mathcal{T}_{k}} f_{tk}\,,\\ \mathrm{ann\_return}_{k} &= p \, \hat{\mu}_{k}\,,\\ \mathrm{ann\_volatility}_{k} &= \sqrt{\frac{p}{T_{k} - 1} \sum_{t \in \mathcal{T}_{k}} \left(f_{tk} - \hat{\mu}_{k}\right)^{2}}\,,\\ \mathrm{sharpe}_{k} &= \frac{\mathrm{ann\_return}_{k}}{\mathrm{ann\_volatility}_{k}}\,. \end{align}\]
Where:
- $\hat{\mu}_{k}$: Per-period mean of the present returns of factor $k$.
- $\mathrm{ann\_return}_{k}$, $\mathrm{ann\_volatility}_{k}$, $\mathrm{sharpe}_{k}$: The three returned entries of factor $k$.
- $f_{tk}$: Factor return of factor $k$ at observation $t$, an entry of the factor return history.
- $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not
NaN. An infinite entry is present. - $T_{k} = |\mathcal{T}_{k}|$: Count of the present returns of factor $k$.
- $p$: Periods per year,
ppy.
The mean is not defined for $T_{k} = 0$, the volatility is not defined for $T_{k} < 2$, and the Sharpe ratio is not defined where the volatility is zero.
Algorithm
For each factor k:
- Drop the absent returns of column
kwithfactor_summary_finite_column, givingv. - Take the mean of
vand clamp it to the least and the greatest entry ofv, givingm. The clamp changes nothing in exact arithmetic. The rounded mean of a constantvcan differ from its common value, and the clamp makesmequal to that value, so every deviation in step 4 is exactly zero. - Multiply
mbyppy, givingann_return[k]. An emptyvgivesNaN. - Take the corrected sample standard deviation of
vaboutmand multiply it by the square root ofppy, givingann_volatility[k]. Avwith fewer than two entries givesNaN. - Divide the two with
factor_summary_ratio, givingsharpe[k].
Arguments
f: Factor return historyobservations × factors.ppy: Periods per year.
Returns
ann_return::Vector{<:Real}: One entry per factor.ann_volatility::Vector{<:Real}: One entry per factor.sharpe::Vector{<:Real}: One entry per factor.
Related
PortfolioOptimisers.factor_summary_autocorrelation — Function
factor_summary_autocorrelation(f::MatNum)Return the lag-one autocorrelation of every factor return series.
The answer is the Pearson correlation of the pair (f[1:end - 1, k], f[2:end, k]), with each half centred on its own mean. StatsBase.autocor is a different estimator. It centres both halves on the mean of the whole series and divides by the sum of squares of the whole series. The two agree in the limit and differ on a short series, so a caller who wants the other definition calls StatsBase.autocor on the factor return history itself.
Mathematical definition
\[\begin{align} \bar{a}_{k} &= \frac{1}{T - 1} \sum_{t = 1}^{T - 1} f_{tk}\,,\\ \bar{b}_{k} &= \frac{1}{T - 1} \sum_{t = 2}^{T} f_{tk}\,,\\ \rho_{k} &= \frac{\sum_{t = 1}^{T - 1} \left(f_{tk} - \bar{a}_{k}\right)\left(f_{(t + 1)k} - \bar{b}_{k}\right)}{\sqrt{\sum_{t = 1}^{T - 1} \left(f_{tk} - \bar{a}_{k}\right)^{2} \sum_{t = 1}^{T - 1} \left(f_{(t + 1)k} - \bar{b}_{k}\right)^{2}}}\,. \end{align}\]
Where:
- $\rho_{k}$: Lag-one autocorrelation of factor $k$.
- $\bar{a}_{k}$: Mean of the leading half, the first $T - 1$ returns of factor $k$.
- $\bar{b}_{k}$: Mean of the trailing half, the last $T - 1$ returns of factor $k$.
- $f_{tk}$: Factor return of factor $k$ at observation $t$, an entry of the factor return history.
- $T$: Number of observations.
The coefficient is not defined for $T < 2$, nor for a series with a constant half, whose denominator is zero. An absent return makes the mean of its half absent, so the coefficient of that series is absent too.
Algorithm
- Return
NaNfor every factor whenT < 2. - For each factor
k, sum each half and track the least and the greatest entry of each half. - Divide each sum by
T - 1and clamp it to the least and the greatest entry of its half, givingmaandmb. The clamp changes nothing in exact arithmetic. It makes the mean of a constant half equal to its common value, so the deviations of that half are exactly zero and the coefficient isNaN. - Accumulate the cross product
caband the two sums of squarescaaandcbbof the deviations. - Divide
cabby the square root ofcaa * cbb, givingac[k].
Arguments
f: Factor return historyobservations × factors.
Returns
autocorr::Vector{<:Real}: One entry per factor, andNaNfor a series of one observation, a series with a constant half and a series with an absent return.
Related
PortfolioOptimisers.factor_summary_returns — Function
factor_summary_returns(csfm::CrossSectionalFactorModel) -> FactorReturnHistory
factor_summary_returns(csfm::CrossSectionalFactorModel, csr::Nothing, fr)
factor_summary_returns(csfm::CrossSectionalFactorModel, csr::CrossSectionalRegression,
fr::Nothing) -> FactorReturnHistory
factor_summary_returns(csfm::CrossSectionalFactorModel, csr::CrossSectionalRegression,
fr::MatNum) -> FactorReturnHistoryReturn the factor return history that a summary reads off a factor model block, and the position of each raw factor on the axis of that history.
A block that carries fr states its factor returns on the raw axis, so the history is fr and the position of raw factor k is k. That covers a factor a family re-basis dropped, whose return the constraint of its family fixes. A block with no fr gives the reduced axis, which is the axis the fit estimated in, with the observed factors after the estimated ones, as cross_sectional_factor_returns states it. The positions are then those of factor_summary_positions, so a dropped factor takes 0. factor_model_summary writes each statistic of the history onto the raw axis with factor_summary_mapped. A separate method handles a block with no fit, and its message names the field the caller must populate.
Arguments
csfm: A cross-sectional factor model block.csr: Thecsrfield of the block, ornothing.fr: Thefrfield of the block, ornothing.
Validation
csfm.csris notnothing, else the function raises anIsNothingErrorthat namescsr.- The rules of
factor_summary_positionswhen the block carries nofr.
Returns
h::FactorReturnHistory: The historyf::MatNum,observations × factors, and the positionspos::Vector{Int}, asFactorReturnHistorystates them.
Related
PortfolioOptimisers.factor_summary_positions — Function
factor_summary_positions(csfm::CrossSectionalFactorModel)
factor_summary_positions(fcb::Nothing, nf::Nothing, K::Integer)
factor_summary_positions(fcb::Nothing, nf::VecStr, K::Integer)
factor_summary_positions(fcb::AbstractFactorFamilyBasis, nf::Nothing, K::Integer)
factor_summary_positions(fcb::FactorFamilyBasis, nf::VecStr, K::Integer)Return the position of each raw factor on the reduced factor axis.
The regression group answers on the reduced axis and the summary answers on the raw one, so the summary joins the two by name. A raw factor that the re-basis dropped takes the position 0, and its Gram columns read NaN for that reason. A block with no re-basis needs no join, because its two axes are the same axis.
Mathematical definition
\[\begin{align} \pi_{k} &= \begin{cases} j & \text{when } n_{k} = r_{j}\,,\\ 0 & \text{when } n_{k} \notin \left\{r_{1}, \ldots, r_{K_{r}}\right\}\,. \end{cases} \end{align}\]
Where:
- $\pi_{k}$: Reduced position of raw factor $k$, its position on the reduced factor axis, and $0$ when the family re-basis dropped it.
- $n_{k}$: Name of raw factor $k$.
- $r_{j}$: Name of reduced factor $j$.
- $K_{r}$: Number of reduced factors.
A block with no re-basis has $\pi_{k} = k$.
Arguments
csfm: A cross-sectional factor model block.fcb: Thefcbfield of the block, ornothing.nf: Thenffield of the block, ornothing.K: Number of raw factors.
Validation
csfm.nfis notnothingwhencsfm.fcbis present, else the function raises anIsNothingErrorthat namesnf. The summary cannot join a re-based block to the raw axis without the names.
Returns
pos::Vector{Int}: One entry per raw factor. Entrykis the position of raw factorkon the reduced axis, or0when the re-basis dropped it.
Related
PortfolioOptimisers.factor_summary_mapped — Function
factor_summary_mapped(v::VecNum, pos::AbstractVector{Int})Return a statistic of the axis of a factor return history, written onto the raw factor axis.
The axis is the reduced one, or the raw one when the block carries fr, and pos names it.
Mathematical definition
\[\begin{align} m_{k} &= \begin{cases} v_{\pi_{k}} & \text{when } 0 < \pi_{k} \leq n\,,\\ \mathrm{NaN} & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $m_{k}$: The statistic of raw factor $k$.
- $v_{j}$: The statistic of factor $j$ of the axis.
- $\pi_{k}$: Position of raw factor $k$ on the axis, and $0$ when the axis holds none.
- $n$: Length of $\boldsymbol{v}$. A statistic of the regression covers the estimated factors alone, which come before the observed ones, so the position of an observed factor lies past $n$.
Arguments
v: A statistic, one entry per factor of the axis, or per estimated reduced factor.pos: Position of each raw factor on the axis ofv,0where that axis holds none, asfactor_summary_returnsgives it.
Returns
m::Vector{<:Real}: One entry per raw factor, andNaNat a factor whose position is0or thatvdoes not cover.
Related
PortfolioOptimisers.factor_summary_gram — Function
factor_summary_gram(csfm::CrossSectionalFactorModel, threshold::Number)
factor_summary_gram(Ms::Nothing, csfm::CrossSectionalFactorModel, threshold::Number)
factor_summary_gram(Ms::Arr3Num, csfm::CrossSectionalFactorModel, threshold::Number)Return the three regression columns of a factor model summary, on the raw factor axis.
The columns are the mean absolute t-statistic, the rate at which the absolute t-statistic passes threshold, and the mean variance inflation factor. Each one is the time average of a series of the regression group, joined back onto the raw factor axis by name.
A block with no exposure history has no regression design either, so all three columns are absent, and a separate method handles that case.
Algorithm
- Take the position of each raw factor on the reduced axis with
factor_summary_positions, givingpos. - Take the t-statistic history with
cs_regression_t_stats, givingt, and the variance inflation factor history withexposure_vif, givingvif. - Take the exceedance rate at
thresholdwithcs_regression_t_stat_exceedance_rate, givingrate. - For each reduced factor, take the mean of the present entries of
abs.(t)and ofvifwithfactor_summary_column_mean, givingabs_tandmvif. - Write
abs_t,rateandmvifonto the raw axis withfactor_summary_mapped.
Arguments
csfm: A cross-sectional factor model block.Ms: TheMsfield of the block, ornothing.threshold: Absolute t-statistic that the exceedance rate counts against.
Returns
mean_abs_t::Option{<:Vector{<:Real}}: One entry per raw factor, ornothing.t_rate::Option{<:Vector{<:Real}}: One entry per raw factor, ornothing.mean_vif::Option{<:Vector{<:Real}}: One entry per raw factor, ornothing.
Related
PortfolioOptimisers.factor_summary_exposure_variance — Function
factor_summary_exposure_variance(Ms::Arr3Num, t::Integer, k::Integer)Return the cross-sectional variance of one factor exposure at one observation.
The variance is uncorrected, and it reads the exposures that are not NaN.
Mathematical definition
\[\begin{align} \bar{b}_{tk} &= \frac{1}{|\mathcal{N}_{tk}|} \sum_{i \in \mathcal{N}_{tk}} b_{tik}\,,\\ s^{2}_{tk} &= \frac{1}{|\mathcal{N}_{tk}|} \sum_{i \in \mathcal{N}_{tk}} \left(b_{tik} - \bar{b}_{tk}\right)^{2}\,. \end{align}\]
Where:
- $s^{2}_{tk}$: Uncorrected cross-sectional variance of the exposure to factor $k$ at observation $t$, over the assets at which that exposure is not
NaN. - $\bar{b}_{tk}$: Cross-sectional mean of the exposure to factor $k$ at observation $t$.
- $b_{tik}$: Exposure of asset $i$ to factor $k$ at observation $t$.
- $\mathcal{N}_{tk}$: Assets at which $b_{tik}$ is not
NaN.
The variance is not defined for an empty $\mathcal{N}_{tk}$. An infinite exposure makes the deviations undefined, so the variance reads NaN.
Algorithm
- Sum the exposures of the cross-section that are not
NaN, count them, and track their least and greatest value. - Return
NaNwhen the count is zero. - Divide the sum by the count and clamp it to the least and the greatest value, giving
m. The clamp changes nothing in exact arithmetic. It makes the mean of a constant cross-section equal to its common value, so the variance of that cross-section is exactly zero. - Average the squared deviations from
m, giving the variance.
Arguments
Ms: Exposure historyassets × factors × observations, unlagged.t: Position of the observation.k: Position of the factor.
Returns
v::Real: The variance, andNaNwhen the cross-section has no exposure that is notNaN.
Related
PortfolioOptimisers.factor_summary_constant_exposures — Function
factor_summary_constant_exposures(Ms::Arr3Num)Return which factor exposures never vary across the cross-section.
The global intercept and the constant column of a one-hot family are constant exposures. The cross-section of a constant exposure has no spread, so its stability coefficient is not defined, and the summary writes 1 in its place. An exposure that never moves is perfectly stable. The summary applies that patch, and exposure_stability keeps the NaN that the correlation gives.
Mathematical definition
\[\begin{align} c_{k} &= \begin{cases} \max_{t \in \mathcal{V}_{k}} s^{2}_{tk} < 10^{-12} & \text{when } \mathcal{V}_{k} \neq \emptyset\,,\\ \mathrm{false} & \text{when } \mathcal{V}_{k} = \emptyset\,. \end{cases} \end{align}\]
Where:
- $c_{k}$: Constant-exposure flag of factor $k$, true when the largest $s^{2}_{tk}$ over the observations is under $10^{-12}$.
- $s^{2}_{tk}$: Uncorrected cross-sectional variance of the exposure to factor $k$ at observation $t$, over the assets at which that exposure is not
NaN. - $\mathcal{V}_{k}$: Observations at which $s^{2}_{tk}$ is not
NaN.
A factor with no exposure anywhere is not constant.
Arguments
Ms: Exposure historyassets × factors × observations, unlagged.
Returns
c::BitVector: One entry per factor,truewhen the largest cross-sectional variance of the factor is under1e-12.
Related
PortfolioOptimisers.factor_summary_stability — Function
factor_summary_stability(Ms::Arr3Num, csfm::CrossSectionalFactorModel; step::Integer,
weighting)Return the stability column of a factor model summary, on the raw factor axis.
The column is the median over the observations of exposure_stability, and a constant exposure reads 1. A history with no more observations than step has no stability series, so every factor reads NaN except the constant ones, which still read 1.
Mathematical definition
\[\begin{align} \mathrm{stability}_{k} &= \begin{cases} 1 & \text{when } c_{k}\,,\\ \operatorname{median} \left\{ a_{tk} : t \in \mathcal{T}_{k} \right\} & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $\mathrm{stability}_{k}$: The returned entry of factor $k$.
- $c_{k}$: Constant-exposure flag of factor $k$, true when the largest $s^{2}_{tk}$ over the observations is under $10^{-12}$.
- $a_{tk}$: Diagnostic series entry at observation $t$ for factor $k$. Here the series is the stability coefficient that
exposure_stabilityreturns. - $\mathcal{T}_{k}$: Present observations of column $k$ of a series, those at which its entry is not
NaN. An infinite entry is present.
The median is not defined for an empty $\mathcal{T}_{k}$, and that is the case for every factor when the history has no more observations than step.
Arguments
Ms: Exposure historyassets × factors × observations, unlagged.csfm: A cross-sectional factor model block.step: Number of observations between the two cross-sections that the coefficient reads.weighting: TheAbstractOrthogonalityMetricwhose weight history the coefficient reads.
Returns
stability::Vector{<:Real}: One entry per raw factor.
Related
PortfolioOptimisers.factor_summary_exposure — Function
factor_summary_exposure(Ms::Nothing, csfm::CrossSectionalFactorModel, step::Integer,
weighting, coverage_weighting)
factor_summary_exposure(Ms::Arr3Num, csfm::CrossSectionalFactorModel, step::Integer,
weighting, coverage_weighting)Return the two exposure columns of a factor model summary, on the raw factor axis.
The columns are the median exposure stability and the average coverage. Both read the unlagged exposure history, as the whole exposure group does. A block with no exposure history has neither column, and a separate method handles that case.
The two columns read different weight histories. The stability reads the history that weighting names, and the coverage reads the history that coverage_weighting names.
Arguments
csfm: A cross-sectional factor model block.Ms: TheMsfield of the block, ornothing.step: Number of observations between the two cross-sections that the stability reads.weighting: TheAbstractOrthogonalityMetricthat the stability reads.coverage_weighting: TheAbstractOrthogonalityMetricwhose positive weights are the universe of the coverage.
Returns
stability::Option{<:Vector{<:Real}}: One entry per raw factor, ornothing.coverage::Option{<:Vector{<:Real}}: One entry per raw factor, ornothing.
Related
PortfolioOptimisers.assert_factor_summary_columns — Function
assert_factor_summary_columns(fs::FactorSummaryResult)Check the columns of a FactorSummaryResult that its constructor built.
The constructor builds the Result first and checks it here, so the check reads each column by its name and not by its position among twelve arguments.
Arguments
fs: The summary to check.
Validation
- Each field but
ppyisnothingor holds as many entries asann_return. Raises aDimensionMismatch. ppy > 0. Raises aDomainError.- The rules of
assert_nan_or_nonnegonann_volatilityandmean_abs_t, and ofassert_nan_or_closed_unit_intervalont_rateandcoverage. An absent column isnothing, and those rules check nothing on it.
Returns
nothing.
Related