Factor Prior: private API

Types

PortfolioOptimisers.LiftedFactorMoments — Type
struct LiftedFactorMoments{__T_mu, __T_sigma, __T_chol, __T_esigma, __T_edof, __T_ediv}

The asset moments that a factor lift projects from the factor moments, with the residual variances that it measured.

This type is an implementation detail and is not intended for direct use. factor_lift makes it. FactorPrior and FactorBlackLittermanPrior read it.

Fields

  • mu: mu: The asset expected returns.
  • sigma: sigma: The asset covariance matrix.
  • chol: chol: The square-root factor, whose trailing block is the residual standard deviations when the lift adds a residual block.
  • esigma: esigma: The residual variances, or nothing when the lift adds no residual block.
  • edof: edof: The degrees of freedom of the residual variances, or nothing when none is known.
  • ediv: ediv: The divisor of the residual variances, or nothing when the variance estimator states none.

Related

source
PortfolioOptimisers.ResidualVarianceCounts — Type
struct ResidualVarianceCounts{__T_edof, __T_ediv}

The degrees of freedom and the divisor of each residual variance that a lift measures.

This type is an implementation detail and is not intended for direct use. residual_variance_counts and cross_sectional_variance_counts make it. factor_lift and cross_sectional_assemble read it.

Fields

  • edof: edof: The degrees of freedom of each variance, or nothing when they are not known.
  • ediv: ediv: The divisor of each variance, or nothing when the variance estimator states none.

Related

source
PortfolioOptimisers.ResidualBlockConfig — Type
struct ResidualBlockConfig{__T_ve, __T_pdm, __T_rsd}

How a prior estimator adds a residual block to the covariance it lifts.

This type is an implementation detail and is not intended for direct use. factor_residual_config makes it. assert_factor_residual_config checks it, and HighOrderFactorPriorEstimator reads it.

Fields

  • ve: ve: The variance estimator that sizes the residual block.
  • pdm: pdm: The positive definite matrix estimator that re-conditions the covariance after a consumer removes the block, or nothing when the estimator adds no block.
  • rsd: rsd: Whether the estimator adds the residual block.

Related

source

Functions

PortfolioOptimisers.factor_reconstruction — Function
factor_reconstruction(re::AbstractTimeSeriesRegressionEstimator, X::MatNum,
                      F::MatNum) -> Tuple{AbstractLoadingsRegressionResult, MatNum}

Fit the loadings and rebuild the asset returns from the factor returns.

This is the first half of the factor lift, and the only half that every factor-axis estimator uses. It fits re on (X, F), and returns the regression result and the posterior returns matrix F * transpose(M) .+ transpose(b). FactorPrior, FactorBlackLittermanPrior and AugmentedBlackLittermanPrior each write this matrix into LowOrderPrior.X.

factor_lift is the second half, which projects the factor moments through the loadings. The two halves are separate functions because FactorBlackLittermanPrior needs the reconstruction before it has the moments to project. Its views change the factor distribution, so its factor moments exist only after the Black-Litterman update.

Algorithm

  1. Fit re on (X, F) with regression to get rr. rr.M holds the $N \times K$ loadings, and rr.b holds the $N \times 1$ intercepts.
  2. Rebuild the asset returns as F * transpose(rr.M) .+ transpose(rr.b) to get posterior_X. It has the rows of F and the columns of X.

Arguments

  • re: Regression estimator.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Returns

  • rr::AbstractLoadingsRegressionResult: Regression result carrying the loadings M and intercepts b.
  • posterior_X::MatNum: Reconstructed asset returns, observations × assets.

Related

source
PortfolioOptimisers.factor_lift — Function
factor_lift(mp::AbstractMatrixProcessingEstimator, ve::AbstractVarianceEstimator,
            rsd::Bool, rr::AbstractLoadingsRegressionResult, f_mu::VecNum, f_sigma::MatNum,
            X::MatNum, posterior_X::MatNum, T::Real = size(posterior_X, 1);
            mtx_sqrt::Option{<:AbstractMatrixSquareRootAlgorithm} = EigenFallbackSquareRoot(),
            kwargs...) -> LiftedFactorMoments

Project factor moments onto the asset axis through the regression loadings.

This is the second half of the factor lift, and FactorPrior and FactorBlackLittermanPrior both call it. It maps f_mu and f_sigma through the loadings and processes the covariance. When rsd is true, it adds the diagonal residual block to the covariance and to the square-root factor.

The two callers differ only in the factor moments that they pass. FactorPrior passes the moments of the wrapped prior, and FactorBlackLittermanPrior passes the Black-Litterman posterior moments.

Mathematical definition

\[\begin{align} \hat{\boldsymbol{\mu}} &= \mathbf{B} \hat{\boldsymbol{f}} + \boldsymbol{\alpha}\,, \\ \hat{\mathbf{\Sigma}} &= \mathbf{B} \mathbf{\Sigma}_f \mathbf{B}^\intercal + \mathbf{\Sigma}_\varepsilon\,. \end{align}\]

Where:

  • $\hat{\boldsymbol{\mu}}$: $N \times 1$ asset expected returns vector.
  • $\hat{\mathbf{\Sigma}}$: $N \times N$ asset covariance matrix.
  • $\mathbf{B}$: $N \times K$ factor loadings matrix, rr.M.
  • $\boldsymbol{\alpha}$: $N \times 1$ vector of regression intercepts, rr.b.
  • $\hat{\boldsymbol{f}}$: $K \times 1$ factor expected returns vector, f_mu.
  • $\mathbf{\Sigma}_f$: $K \times K$ factor covariance matrix, f_sigma.
  • $\mathbf{\Sigma}_\varepsilon$: $N \times N$ diagonal matrix of residual variances, present only when rsd is true.

The returned chol is the transpose of $[\mathbf{B} \mathbf{L}_f \quad \mathbf{\Sigma}_\varepsilon^{1/2}]$, where $\mathbf{L}_f$ is the square root of $\mathbf{\Sigma}_f$ that matrix_square_root takes under mtx_sqrt. By default it is the lower Cholesky factor of a positive definite matrix, and the eigen square root of a singular one. chol is $(K + N) \times N$ when rsd is true, and $K \times N$ when rsd is false. With rsd = false, neither chol nor $\hat{\mathbf{\Sigma}}$ holds the residual block.

The identity $\mathtt{chol}^\intercal \mathtt{chol} = \hat{\mathbf{\Sigma}}$ holds before matrix processing only. The lift builds chol from the f_sigma that the caller passed, so step 4 of the algorithm changes sigma and does not change chol. The default MatrixProcessing leaves the projected covariance as it is, because its pdm changes nothing on a positive semi-definite matrix. Under that mp the identity holds on the returned pair. An mp that denoises or detones moves sigma and leaves chol as it was, so the identity then holds against the unprocessed covariance only. A consumer that needs a factor of the returned sigma must factorise it again.

Algorithm

  1. Read the loadings M and the intercepts b off rr.
  2. Project the factor mean through the loadings to get posterior_mu.
  3. Project the factor covariance through the loadings to get posterior_sigma, the systematic block.
  4. Process posterior_sigma in place with sample_matrix_processing!, under mp, posterior_X and T. The matrix is assembled from the factor moments, so the denoising step reads the count of the factor covariance, and not the number of rows of posterior_X.
  5. Take the square root of f_sigma under mtx_sqrt with matrix_square_root, and multiply it by the loadings to get posterior_csigma. This reads the f_sigma that the caller passed, which step 4 does not change.
  6. When rsd is true, take the reconstruction error err = X - posterior_X. Measure esigma, the column variances of err, with ve. Read the counts of those variances with variance_count, and combine them with the edof of rr through residual_variance_counts to get edof and ediv. Let err_sigma be the diagonal matrix of the variances. When rsd is false, esigma and ediv are nothing, and edof is the edof of rr.
  7. Under rsd, add err_sigma to posterior_sigma, and re-condition the sum with posdef! under the estimator that matrix_processing_posdef gives for mp. This is the only explicit posdef! call of the function. For a MatrixProcessing that estimator is mp.pdm, and it also acts on posterior_sigma inside step 4 when :pdm is a member of mp.order.
  8. Under rsd, append the columns of sqrt.(err_sigma) to posterior_csigma, so the factor also holds the block that step 7 added to the covariance.
  9. Reshape posterior_csigma to length(posterior_mu) columns and transpose it into chol. Return mu, sigma, chol, esigma, edof and ediv.

Arguments

  • mp: Matrix processing estimator.
  • ve: Variance estimator.
  • rsd: Whether to include residual variance in the posterior covariance.
  • rr: Regression result carrying the loadings M and intercepts b.
  • f_mu: Factor expected returns, factors × 1.
  • f_sigma: Factor covariance matrix, factors × factors.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • posterior_X: Reconstructed asset returns from factor_reconstruction.
  • T: The effective number of observations of f_sigma, which the denoising step of mp reads. A caller passes the count that the factor prior states, from moment_observation_count. It is size(posterior_X, 1) by default.
  • mtx_sqrt: Square-root algorithm of the factor covariance, and of the idiosyncratic covariance when that block is a full matrix, whose square roots make up chol. The default EigenFallbackSquareRoot takes the Cholesky factor where it exists, and the eigen square root of a singular positive semidefinite matrix otherwise. RidgeCholeskySquareRoot takes the Cholesky factor of the matrix with a small ridge added, and nothing takes the plain Cholesky factor, which raises a LinearAlgebra.PosDefException on a matrix that is not positive definite. matrix_square_root states each algorithm.
  • kwargs...: Additional keyword arguments passed to matrix processing.

Returns

  • lift::LiftedFactorMoments: Asset expected returns, asset covariance, the square-root factor whose trailing block is the residual standard deviations when rsd is true, the residual variances, and their degrees of freedom and divisors. esigma and ediv are nothing when rsd is false, because the lift added no residual block. A caller writes the last three onto the loadings result that it returns. A consumer that needs the idiosyncratic variances, or their sampling law, then reads them off the block, and does not compute them again from the reconstruction error.

Related

source
PortfolioOptimisers.residual_variance_counts — Function
residual_variance_counts(cnt::Nothing, edof, T::Integer) -> ResidualVarianceCounts
residual_variance_counts(cnt::VarianceCount, edof::Nothing, T::Integer) -> ResidualVarianceCounts
residual_variance_counts(cnt::VarianceCount, edof::VecNum, T::Integer) -> ResidualVarianceCounts

Degrees of freedom and divisor of each residual variance that a lift measures.

The sampling law of a residual variance needs one fact from the regression and one from the variance estimator. The regression records in edof the count of equally weighted observations that remain after the parameters it spent on each asset. variance_count states in cnt the effective count of the observations that the variance estimator reads, and its divisor. This function combines the two. Each case is a method, so no caller writes an isnothing test.

Mathematical definition

\[\begin{align} \nu_{i} &= n_{i} - \left(T - \nu_{i}^{\mathrm{fit}}\right)\,. \end{align}\]

Where:

  • $\nu_{i}$: Degrees of freedom of the residual variance of asset $i$.
  • $n_{i}$: Effective count of the observations that the variance estimator reads, cnt.n.
  • $T$: Number of observations.
  • $\nu_{i}^{\mathrm{fit}}$: Degrees of freedom that the regression left, edof. $T - \nu_{i}^{\mathrm{fit}}$ is the number of parameters it spent on the asset.

With equal weights $n_{i} = T$, and the result is the edof of the regression. Under unequal weights the function subtracts the parameters that the regression spent from the effective count. This approximates the law of a weighted sum of squared residuals.

Arguments

  • cnt: The count and divisor of the variance estimator, or nothing when it states none.
  • edof: The edof of the loadings block, or nothing when the regression recorded none.
  • T: Number of observations of the residuals.

Returns

  • counts::ResidualVarianceCounts: The degrees of freedom and the divisor of each variance. Both are nothing when the variance estimator states no count, because neither fact is then known. edof alone is nothing when the regression recorded no count.

Related

source
PortfolioOptimisers.factor_residual_config — Function
factor_residual_config(pe::AbstractPriorEstimator) -> Option{<:ResidualBlockConfig}

Declare how a prior estimator adds a residual block to the covariance it lifts.

HighOrderFactorPriorEstimator subtracts the residual block to recover the systematic covariance, because its residual cokurtosis correction is defined on that covariance. It cannot read ve and the repair of mp off the fields of the estimator that it wraps. Its pe field has the bound AbstractLowOrderPriorEstimator_F_AF, and in that bound only FactorPrior and FactorBlackLittermanPrior carry both fields. The other members wrap a prior, pool priors, or build the asset moments with no residual block, as AugmentedBlackLittermanPrior does. A field access on one of them raises a FieldError.

The declaration states whether the estimator added a block, and how to re-condition the covariance after a consumer removes the block. The block itself is on the loadings result, at rr.esigma, and the fit measured it under its own weights. A prior that embeds this estimator can give the fit weights other than those of ve, for example the w of a caller on an entropy pooling prior, or the pooled weights of an opinion pool. So a consumer reads the block off rr, and does not measure it again under ve.

Each estimator declares its answer in a method beside its own definition:

  • The two estimators that add a residual block report it.
  • A wrapper forwards the answer of the estimator that it wraps. A pooling estimator is a wrapper here, and it forwards the one estimator that its moments come from.
  • An estimator that adds no residual block returns nothing from an explicit method.
  • CrossSectionalFactorPrior throws an ArgumentError. Its block is the idiosyncratic covariance that it measured, which a variance under ve does not rebuild.

An answer of nothing and an answer whose rsd is false both mean that the estimator added no residual block, so the consumer leaves the covariance as it is.

There is no default. A fallback that returns nothing cannot tell an estimator that adds no residual block from an estimator whose author forgot the method. In the second case the consumer does not subtract a residual block that the covariance holds. So the method for an undeclared type throws an ArgumentError that names the type. range_tails uses the same rule for a per-type declaration whose absence is a defect.

Arguments

  • pe: Prior estimator.

Validation

  • Throws an ArgumentError when the type of pe declares no method.

Returns

  • nothing: The estimator adds no residual block.
  • cfg::ResidualBlockConfig: The variance estimator that sizes the residual block, the positive definite matrix estimator that re-conditions the covariance after a consumer removes the block, and whether the estimator adds the block. A factor prior gives pdm from matrix_processing_posdef when rsd is true, and nothing when rsd is false, because no consumer repairs a covariance that has no residual block.

Related

source
PortfolioOptimisers.assert_factor_residual_config — Function
assert_factor_residual_config(pe::AbstractPriorEstimator, cfg) -> Nothing

Check the shape of a factor_residual_config answer before a consumer reads it.

A consumer reads ve, pdm and rsd off the answer by property access, which checks no shape. A declaration that returns a wrong value would raise a FieldError deep inside the correction, far from the declaration that caused it. This function checks the two shapes that the contract permits, and its error names the estimator that answered.

Arguments

  • pe: Prior estimator that answered.
  • cfg: The answer of factor_residual_config(pe).

Validation

Related

source
PortfolioOptimisers.coverage_regression — Function
coverage_regression(re::AbstractTimeSeriesRegressionEstimator, cmsk::Nothing)
coverage_regression(re::AbstractTimeSeriesRegressionEstimator, cmsk::BitVector)

Takes the view of a regression estimator at the assets that a factor prior fits.

FactorPrior, FactorBlackLittermanPrior and AugmentedBlackLittermanPrior fit the regression on the columns of the returns that a mask keeps. A regression can hold one value per asset, such as the factor set of each asset in included of a StepwiseRegression. That value must describe the same columns as the returns that the fit reads, so the prior takes the view of the regression with the same mask.

Algorithm

The method that Julia selects is the algorithm.

  1. If cmsk is nothing, every asset is in the fit, so return re.
  2. If cmsk is a mask, return port_opt_view of re at findall(cmsk).

Arguments

  • re: Regression estimator.
  • cmsk: The mask of the assets in the fit, or nothing.

Returns

  • re::AbstractTimeSeriesRegressionEstimator: The regression on the assets in the fit.

Related

source
PortfolioOptimisers.show_fields — Method
show_fields(_::FactorPrior) -> NTuple{7, Symbol}

Renders every field of a FactorPrior except cache.

The state in cache belongs to an incremental fit and is not part of the configuration, and it would print under the estimator at every site that renders one. Set set_show_nothing_fields!(:FactorPrior, true) to render it.

Arguments

  • ::FactorPrior: Prior estimator, read for its type alone.

Returns

  • fields::Tuple: The field names to render, which is (:pe, :mp, :re, :ve, :rsd, :mtx_sqrt, :x_src).

Related

source