Cross-Sectional Calibration Rules: private API

PortfolioOptimisers.cross_sectional_reduced_history — Function

cross_sectional_reduced_history(fcb::Nothing, Ms::Arr3Num) -> Arr3Num
cross_sectional_reduced_history(fcb::FactorFamilyBasis, Ms::Arr3Num) -> Arr3Num

Return the exposure history of the block of a CrossSectionalFactorPrior on the reduced factor axis.

The block carries the raw exposure history Ms and the Factor Family Basis fcb of the same rows. The latest loadings L are the reduced exposures of the last row, so a split of a past row reads the reduced exposures of that row, and a split of the last row gives the coefficients that the prior blends. A prior that constrains no Factor Family carries no basis, and its raw history is the reduced one.

Arguments

fcb: The Factor Family Basis of the block, or nothing.
Ms: The exposure history of the block on the raw axis, observations × assets × factors.

Validation

The rules of reduce_exposures, when fcb is a FactorFamilyBasis.

Returns

Ms::Arr3Num: The exposure history on the reduced axis, observations × assets × reduced factors.

Related

source

PortfolioOptimisers.cross_sectional_split_history — Function

cross_sectional_split_history(cs::NamedTuple) -> NamedTuple

Split each row of the Return Forecast history of a CrossSectionalFactorPrior against the Factor Exposures of that row.

The prior splits the latest forecast against the latest exposures, under the regression weights of the latest fit. This function makes the same split at each row of the history, with the exposures and the weights of that row. So the last row gives the g and the ap that the prior holds, and each earlier row gives the parts that the forecast stated at that row.

Algorithm

Read the reduced exposure history with cross_sectional_reduced_history, and keep the columns of the estimated factors, the columns of cs.csfm.csr.f.
Fill g, estimated factors × rows, and ap, rows × assets, with NaN.
At each row of cs.hist that holds a finite forecast, split the row with cross_sectional_alpha_split under cs.cre, against the exposures of that row and the regression weights cs.csfm.rw of that row, and write the two parts.

Arguments

cs: The fit of a CrossSectionalFactorPrior as (; g, ap, csfm, hist, cre), or nothing when the site names none. g holds the spanned coefficients of the Return Forecast, one per estimated factor, and ap its unscaled orthogonal part, one per asset; both are nothing when the prior states no Return Forecast Estimator. csfm is the CrossSectionalFactorModel block, with a zero b and no Return Forecast, which carries the idiosyncratic variance history vs and the regression weights rw. hist is the Return Forecast history from forecast_history, or nothing unless a slot answers true to reads_forecast_history. cre is the Cross-Sectional Regression Estimator of the prior, which splits a row of the history as the prior splits the latest forecast.

Validation

cs.hist is given. Raises an IsNothingError.
cs.hist, the exposure history and the weight history of the block have one row per observation of the block. Raises a DimensionMismatch.
The rules of cross_sectional_alpha_split.

Returns

g::MatNum: The spanned coefficients of each row, estimated factors × rows. A row with no finite forecast carries NaN.
ap::MatNum: The orthogonal part of each row, rows × assets. A row with no finite forecast carries NaN.

Related

source

PortfolioOptimisers.cross_sectional_context — Function

cross_sectional_context(ctx) -> NamedTuple

Return the fit of a Cross-Sectional Factor Prior that a rule of its Calibration Slots reads.

A rule of AbstractSpannedShrinkageCalibrationAlgorithm or of AbstractOrthogonalForecastScaleCalibrationAlgorithm reads ctx.cs, which only the prior states. A caller who runs such a rule outside the prior builds that context, and this function refuses a context that has none.

Arguments

ctx: The site's CalibrationContext.

Validation

ctx.cs is given. Raises an IsNothingError.

Returns

cs::NamedTuple: The fit of the prior, as CalibrationContext states it.

Related

source

PortfolioOptimisers.spanned_shrinkage_moments — Function

spanned_shrinkage_moments(pr::AbstractPriorResult, cs::NamedTuple, G::AbstractMatrix)
    -> NamedTuple

Return the factor mean, the spanned coefficients and the factor covariance in the form that a Spanned Shrinkage rule reads.

The blend acts on the estimated factors alone, so the function keeps the leading entries of the factor moments, one per column of cs.csfm.csr.f. The factor covariance of those factors can be singular. A market factor beside a full one-hot industry block is the common case: the market exposure is the sum of the industry exposures, so the fitted factor returns, their mean and the spanned coefficients all lie in a subspace, and the covariance is zero across it. An Empty Factor carries a zero variance and no return, which is the same case in one direction. So the function reads the covariance through its pseudo-inverse over the eigenvalues it keeps, and K is the number of those eigenvalues, the dimension in which the factor mean has a sampling error. A factor whose mean or variance the factor prior does not state has no sampling error to measure, so the function keeps the estimated factors with a finite mean and a finite variance alone.

Mathematical definition

\[\begin{align} \hat{\mathbf{\Sigma}}_{f} &= \sum_{i} \ell_{i} \boldsymbol{u}_{i} \boldsymbol{u}_{i}^{\intercal}\,, \\ \mathbf{W} &= \left[ \boldsymbol{u}_{i} / \sqrt{\ell_{i}} \right]_{\ell_{i} > \tau}\,, \\ \tau &= \sqrt{\epsilon} \max_{i} \ell_{i}\,. \end{align}\]

Where:

$\hat{\mathbf{\Sigma}}_{f}$: Factor covariance of the nested factor prior, over the estimated factors with a finite mean and a finite variance.
$\ell_{i}$, $\boldsymbol{u}_{i}$: Eigenvalue $i$ of the factor covariance and its eigenvector.
$\mathbf{W}$: The kept eigenvectors, each divided by the square root of its eigenvalue, so $\mathbf{W} \mathbf{W}^{\intercal}$ is the pseudo-inverse of the factor covariance.
$\tau$: Threshold below which an eigenvalue is taken as zero. The zero eigenvalue of a singular factor covariance is round-off, and its size depends on the machine, so the threshold sits far above round-off: an eigenvalue that small carries no variance a sample can measure.
$\epsilon$: Machine epsilon of the element type of the eigenvalues.

Arguments

pr: The factor moments of the nested factor prior, on the reduced factor axis.
cs: The fit of a CrossSectionalFactorPrior as (; g, ap, csfm, hist, cre), or nothing when the site names none. g holds the spanned coefficients of the Return Forecast, one per estimated factor, and ap its unscaled orthogonal part, one per asset; both are nothing when the prior states no Return Forecast Estimator. csfm is the CrossSectionalFactorModel block, with a zero b and no Return Forecast, which carries the idiosyncratic variance history vs and the regression weights rw. hist is the Return Forecast history from forecast_history, or nothing unless a slot answers true to reads_forecast_history. cre is the Cross-Sectional Regression Estimator of the prior, which splits a row of the history as the prior splits the latest forecast.
G: The spanned coefficients, estimated factors × rows.

Returns

mu::VecNum: The factor mean of the kept estimated factors.
G::AbstractMatrix: The rows of the spanned coefficients at the kept estimated factors.
W::MatNum: The whitening matrix, kept estimated factors × K.
ev::VecNum: The kept eigenvalues.
K::Int: Number of kept eigenvalues.

Related

source

PortfolioOptimisers.stein_spanned_shrinkage — Function

stein_spanned_shrinkage(alg::BayesStein, mu::VecNum, g::VecNum, W::MatNum, ev::VecNum,
                        T::Number)
stein_spanned_shrinkage(alg::JamesStein, mu::VecNum, g::VecNum, W::MatNum, ev::VecNum,
                        T::Number)
stein_spanned_shrinkage(alg::BodnarOkhrinParolya, mu::VecNum, g::VecNum, W::MatNum,
                        ev::VecNum, T::Number)

Return the weight of the factor mean that the shrinkage intensity of alg gives, with g as the target.

SteinShrinkage states the three formulas. This function takes one method per algorithm.

Arguments

alg: The shrunk expected-returns algorithm.
mu: The factor mean of the estimated factors.
g: The spanned coefficients.
W: The whitening matrix of spanned_shrinkage_moments, so W W' is the pseudo-inverse of the factor covariance and K is its number of columns.
ev: The kept eigenvalues of the factor covariance.
T: The effective sample size.

Validation

JamesStein: when mu == g, every weight gives the same blend, and the function returns one.
BodnarOkhrinParolya: T > K, with K the rank of the factor covariance. Raises a DomainError.
BodnarOkhrinParolya: u v - s^2 is not zero, which fails when g is a multiple of mu, and so when g is zero. Raises a DomainError.

Returns

lambda::Number: The Spanned Shrinkage, in [0, 1].

Related

source

PortfolioOptimisers.forecast_scale_distance — Function

forecast_scale_distance(c::Number, se::Number, target::Number) -> Option{<:Number}

Return the distance of the clipped calibration slope from the target, or nothing when a shrinkage form takes the target.

The two shrinkage forms of AbstractForecastScaleWarmUpAlgorithm divide by the squared distance, and they read the standard error. So a slope that is not finite, a standard error that is not finite, and a distance of zero each give nothing, and the form returns the target.

Arguments

c: The calibration slope.
se: The standard error of the slope.
target: The number that the rule falls back to.

Returns

d::Option{<:Number}: max(c, 0) - target, or nothing.

Related

source

PortfolioOptimisers.orthogonal_forecast_pairs — Function

orthogonal_forecast_pairs(cs::NamedTuple) -> NamedTuple

Pool the pairs of the orthogonal part of each forecast row and the idiosyncratic return of the next row.

Algorithm

Split the Return Forecast history with cross_sectional_split_history, giving the orthogonal part ap of each row.
Take the idiosyncratic return of the next row as the target of each row, with forward_mean_returns at a horizon and a lag of one, over the residuals that forecast_idiosyncratic_returns reads. A target at a Leverage-One Pair is NaN, because its residual is zero by construction and is no observation of the return, so the pair drops out.
Pool the pairs of the rows before the latest with forecast_calibration_pairs, under the regression weights cs.csfm.rw.
Count the rows that give at least one pair, giving n.

Arguments

cs: The fit of a CrossSectionalFactorPrior as (; g, ap, csfm, hist, cre), or nothing when the site names none. g holds the spanned coefficients of the Return Forecast, one per estimated factor, and ap its unscaled orthogonal part, one per asset; both are nothing when the prior states no Return Forecast Estimator. csfm is the CrossSectionalFactorModel block, with a zero b and no Return Forecast, which carries the idiosyncratic variance history vs and the regression weights rw. hist is the Return Forecast history from forecast_history, or nothing unless a slot answers true to reads_forecast_history. cre is the Cross-Sectional Regression Estimator of the prior, which splits a row of the history as the prior splits the latest forecast.

Validation

The rules of cross_sectional_split_history.

Returns

a::VecNum: The orthogonal forecast of each pair.
b::VecNum: The idiosyncratic return of each pair, at the next row.
q::VecNum: The weight of each pair.
n::Int: Number of rows that give at least one pair.

Related

source

PortfolioOptimisers.forecast_calibration_slope_se — Function

forecast_calibration_slope_se(a::AbstractVector{<:Real}, b::AbstractVector{<:Real},
                              q::AbstractVector{<:Real}, c::Real) -> Real

Return the heteroskedasticity-consistent standard error of a calibration slope through the origin.

ForecastCalibrationSlope states the formula, the HC0 sandwich of the weighted regression of b on a.

Arguments

a: The forecast of each pair.
b: The target of each pair.
q: The weight of each pair.
c: The slope, from forecast_calibration_slope.

Returns

se::Real: The standard error. It is NaN when no pair has a positive weighted square.

Related

source

PortfolioOptimisers.carry_growing_parts — Function

carry_growing_parts(
    cfg::Union{NamedTuple, CrossSectionalFactorPrior}
) -> Vector{String}

Lists the parts of a CrossSectionalFactorPrior whose step cost on the carry fold grows with the number of folded observations.

Each entry names the part and the method that it lacks. The test reads the configuration alone, so the constructor of the prior can run it.

Algorithm

For each factor name => xe, list the factor when carry_lookback of xe is nothing. The carry then keeps every panel row.
List rfe when cross_sectional_forecast_reads_panel answers true, because the forecast fits again over every row. Otherwise list it when its carry_lookback is nothing, because the carry keeps every row under it.
List ve when supports_partial_fit answers false.
List pe when carry_folds answers false.
List th when it is positive and supports_partial_fit answers false for ce, because the idiosyncratic correlation then has no fold.
List choice with carry_choice_parts: a BatchChoice with an automatic dropped member, because a step that moves the member folds the factor prior again over every carried factor return.

Arguments

cfg: The prior, or a NamedTuple of its fields factors, pe, ve, rfe, th, ce, choice and families.

Returns

parts::Vector{String}: One description for each part that grows, empty when no part grows.

Related

source

PortfolioOptimisers.assert_systematic_repair — Function

assert_systematic_repair(srep::AbstractSystematicRepair,
                         mp::AbstractMatrixProcessingEstimator) -> nothing
assert_systematic_repair(srep::NoSystematicRepair,
                         mp::AbstractMatrixProcessingEstimator) -> nothing

Checks that the Systematic Repair rule of a CrossSectionalFactorPrior can process the systematic block under its matrix processing estimator.

Algorithm

The method that Julia selects is the algorithm.

Every rule but NoSystematicRepair accepts every mp. A rule of a caller states its own needs in its method of systematic_processing!.
NoSystematicRepair reads the steps of mp one by one, so it accepts a MatrixProcessing alone. Without this check, another estimator fails at the fit with a MethodError of systematic_processing!.

Arguments

srep: Systematic Repair rule of the lift, a member of AbstractSystematicRepair. NoSystematicRepair runs every step of mp on the systematic block except the :pdm step, and SystematicRepair runs every step. Under each rule the lift then adds the idiosyncratic block and repairs the sum under the estimator that matrix_processing_posdef gives for mp.
mp: Matrix processing estimator of the prior.

Validation

Under NoSystematicRepair, mp is a MatrixProcessing.

Returns

nothing.

Related

source

PortfolioOptimisers.carry_growing_part — Function

carry_growing_part(
    grows::Bool,
    msg::String
) -> Vector{String}

Returns the description of one part of a CrossSectionalFactorPrior in a vector when the part grows, and an empty vector otherwise.

Arguments

grows: Whether the step cost of the part grows with the stream.
msg: The description of the part.

Returns

parts::Vector{String}: [msg] when grows is true, and an empty vector otherwise.

Related

carry_growing_parts

source

PortfolioOptimisers.carry_forecast_parts — Function

carry_forecast_parts(rfe::Nothing) -> Vector{String}
carry_forecast_parts(rfe::AbstractReturnForecastEstimator) -> Vector{String}

Describes the Return Forecast of a CrossSectionalFactorPrior when its step cost on the carry fold grows with the stream.

Algorithm

The method that Julia selects is the algorithm.

An absent forecast does not grow.
A forecast for which cross_sectional_forecast_reads_panel answers true fits again over every row at each call with no data.
Any other forecast whose carry_lookback is nothing makes the carry keep every panel row. A Descriptor of its scores that carries a state counts one row there.

Arguments

rfe: The Return Forecast Estimator, or nothing.

Returns

parts::Vector{String}: One description when the forecast grows, and an empty vector otherwise.

Related

source

PortfolioOptimisers.carry_choice_parts — Function

carry_choice_parts(choice::AbstractChoiceRule, families) -> Vector{String}
carry_choice_parts(choice::BatchChoice, families::AbstractVector{<:Pair})
    -> Vector{String}

Describes the Choice Rule of a CrossSectionalFactorPrior when its step cost on the carry fold grows with the stream.

A BatchChoice chooses each automatic dropped member again at each step. A step that moves it folds the factor prior again over every carried factor return, as cross_sectional_fold_move states, so the cost of that step grows with the stream. A named member, or a PinnedChoice, never moves.

Algorithm

The method that Julia selects is the algorithm.

A rule other than BatchChoice, or a prior with no constrained Factor Family, does not grow.
A BatchChoice grows when a family names no dropped member.

Arguments

choice: The Choice Rule of the prior.
families: The constrained Factor Families of the prior, or nothing.

Returns

parts::Vector{String}: One description when the choice grows, and an empty vector otherwise.

Related

source