Factor Prior

PortfolioOptimisers.FactorPrior — Type
struct FactorPrior{__T_pe, __T_mp, __T_re, __T_ve, __T_rsd, __T_mtx_sqrt, __T_cache, __T_x_src} <: AbstractLowOrderPriorEstimator_F

Factor-based prior estimator for asset returns.

FactorPrior is a low order prior estimator that computes the mean and covariance of asset returns from a factor model. pe fits the factor moments on the factor returns, and re fits the loadings of each asset on the factors. The estimator projects the factor moments through the loadings, and processes the covariance with mp. When rsd is true, it adds to the covariance the residual variances that ve measures.

The covariance is the plug-in estimator of the factor model that Fan et al. [151] state. It is the loadings times the factor covariance times the transposed loadings, plus the diagonal matrix of the residual variances. The library differs from the source in three ways:

  • The mean adds the regression intercepts. The model of the source has no intercepts.
  • The default re is stepwise, so an asset can drop a factor. The source fits every factor by least squares.
  • ve sizes the residual variances. The default SimpleVariance divides by T - 1, and the source divides by T.

Fields

  • pe: Prior estimator.
  • mp: Matrix processing estimator.
  • re: Regression estimator.
  • ve: Variance estimator.
  • rsd: Whether to include residual variance in the posterior covariance.
  • mtx_sqrt: Square-root algorithm of the factor covariance, and of the idiosyncratic covariance when that block is a full matrix, whose square roots make up chol. The default EigenFallbackSquareRoot takes the Cholesky factor where it exists, and the eigen square root of a singular positive semidefinite matrix otherwise. RidgeCholeskySquareRoot takes the Cholesky factor of the matrix with a small ridge added, and nothing takes the plain Cholesky factor, which raises a LinearAlgebra.PosDefException on a matrix that is not positive definite. matrix_square_root states each algorithm.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it. factory keeps it unchanged, because a factory call resolves the configuration, not the sample. port_opt_view copies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone. obs_weights_view removes it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.
  • x_src: The source of the returns that the prior fits on, nothing or an AbstractReturnsSource. nothing and DataSource read the asset returns rd.X. ExcessSource reads the returns in excess of the benchmark, rd.X - rd.B, the Excess Frame, and refuses returns data with no benchmark. nothing means that the caller stated no source, so an optimiser that holds a reader of the Excess Frame warns about a prior at nothing, and not about a prior at DataSource(). A prior reads no prior, so it refuses PriorSource. A prior inside another estimator, such as the pe of a FactorPrior, keeps nothing, and the outer estimator states the source.

Constructors

FactorPrior(;    pe::AbstractLowOrderPriorEstimator_A_AF = EmpiricalPrior(),    mp::AbstractMatrixProcessingEstimator = MatrixProcessing(),    re::AbstractTimeSeriesRegressionEstimator = StepwiseRegression(),    ve::AbstractVarianceEstimator = SimpleVariance(),    rsd::Bool = true,    mtx_sqrt::Option{<:AbstractMatrixSquareRootAlgorithm} = EigenFallbackSquareRoot(),    cache::Option{<:AbstractPartialFitState} = nothing,    x_src::Option{<:AbstractReturnsSource} = nothing) -> FactorPrior

Keywords correspond to the struct's fields.

Validation

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • pe: Recursively updated through factory.
  • re: Recursively updated through factory.
  • ve: Recursively updated through factory.
  • cache: Carried unchanged through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

The incremental fit

This prior has no exact incremental recursion, so the online step refits from a sample buffer. Online seeds cache, and partial_fit! appends each observation to it. The one-argument prior runs the batch verb of this estimator over the rows in the buffer. The answer equals a batch fit over those rows, and a max_history on the wrapper cuts the whole fit.

obs_weights_view drops cache, because a partial-fit state has no slice on the observation axis. show_fields does not render it, because a running buffer is not part of the configuration.

Composition: what this estimator forwards

This estimator lifts a factor-axis prior onto the asset axis. It rebuilds X as F * transpose(M) .+ transpose(b), so it builds its prior result itself and does not forward one along its own axis. The forwarding rule still governs each field. This estimator does not change the factor distribution. FactorBlackLittermanPrior is this estimator with views on the factor block before the lift.

  • The factor block fpr is the result of the wrapped factor prior, forwarded whole. It needs no reconstruction, because the asset moments are its projection and not an update of it.
  • mu and sigma are that block projected through the loadings, so mu == rr.M * fpr.mu + rr.b holds for every investable asset. When rsd is true, sigma also holds the residual variances.
  • chol is not forwarded. The lift rebuilds it on the asset axis from M * matrix_square_root(mtx_sqrt, fpr.sigma), and adds the residual block when rsd is true. It factorises sigma before matrix processing, see factor_lift. An mp that denoises or detones moves sigma and leaves chol as it was.
  • w is the observation weights of the factor prior. This estimator wraps only a factor prior, and posterior_X has the rows of F, so these are the only weights, and they are on the correct axis. The ens, kld and ow of the factor prior come with them.

Examples

julia> FactorPrior()FactorPrior        pe ┼ EmpiricalPrior           │           ce ┼ PortfolioOptimisersCovariance           │              │   ce ┼ Covariance           │              │      │    me ┼ SimpleExpectedReturns           │              │      │       │   w ┴ nothing           │              │      │    ce ┼ GeneralCovariance           │              │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)           │              │      │       │    w ┴ nothing           │              │      │   alg ┼ FullMoment()           │              │      │     w ┴ nothing           │              │   mp ┼ MatrixProcessing           │              │      │     pdm ┼ Posdef           │              │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton           │              │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()           │              │      │      dn ┼ nothing           │              │      │      dt ┼ nothing           │              │      │     alg ┼ nothing           │              │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)           │           me ┼ SimpleExpectedReturns           │              │   w ┴ nothing           │      horizon ┼ nothing           │   fill_limit ┼ nothing           │        x_src ┴ nothing        mp ┼ MatrixProcessing           │     pdm ┼ Posdef           │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton           │         │   kwargs ┴ @NamedTuple{}: NamedTuple()           │      dn ┼ nothing           │      dt ┼ nothing           │     alg ┼ nothing           │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)        re ┼ StepwiseRegression           │       crit ┼ PValue           │            │   t ┴ Float64: 0.05           │        alg ┼ ForwardSelection()           │        tgt ┼ LinearModel           │            │   kwargs ┴ @NamedTuple{}: NamedTuple()           │     choice ┼ BatchChoice()           │   included ┼ nothing           │      x_src ┴ nothing        ve ┼ SimpleVariance           │          me ┼ SimpleExpectedReturns           │             │   w ┴ nothing           │           w ┼ nothing           │   corrected ┴ Bool: true       rsd ┼ Bool: true  mtx_sqrt ┼ EigenFallbackSquareRoot()     x_src ┴ nothing

Related

References

  • [151] J. Fan, Y. Fan and J. Lv. High dimensional covariance matrix estimation using a factor model. Journal of Econometrics 147, 186–197 (2008). Section 1.2, Equations 1.3 and 1.4. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
PortfolioOptimisers.prior — Function
prior(pe::FactorPrior, X::MatNum, F::MatNum, pnl::Option{<:AssetPanel} = nothing;
      dims::Int = 1, strict::Bool = false, ts::Option{<:VecDate} = nothing,
      active_mask::Option{<:AbstractMatrix{<:Bool}} = nothing, cache_reg::Bool = false,
      kwargs...)

Compute the prior moments of asset returns from a factor model.

pe.pe fits the factor moments on F, and pe.re fits the loadings of the assets on the factors. factor_lift projects the factor moments onto the assets through the loadings. When pe.rsd is true, the lift adds the residual variances to the covariance. The method returns a LowOrderPrior.

Mathematical definition

The factor model maps the factor moments to the assets through the loadings matrix $\mathbf{B}$ and the intercepts $\boldsymbol{\alpha}$:

\[\begin{align} \hat{\boldsymbol{\mu}} &= \mathbf{B} \hat{\boldsymbol{f}} + \boldsymbol{\alpha}\,, \\ \hat{\mathbf{\Sigma}} &= \mathbf{B} \mathbf{\Sigma}_f \mathbf{B}^\intercal + \mathbf{\Sigma}_\varepsilon\,. \end{align}\]

Where:

  • $\mathbf{B}$: $N \times K$ factor loadings matrix, rr.M.
  • $\hat{\boldsymbol{f}}$: $K \times 1$ vector of factor expected returns.
  • $\boldsymbol{\alpha}$: $N \times 1$ vector of regression intercepts, rr.b.
  • $\mathbf{\Sigma}_f$: $K \times K$ factor covariance matrix.
  • $\mathbf{\Sigma}_\varepsilon$: $N \times N$ diagonal matrix of residual variances, present only when rsd is true.

pe.pe fits $\hat{\boldsymbol{f}}$ and $\mathbf{\Sigma}_f$ on F, and pe.re fits the loadings on (X, F). factor_lift computes the two equations.

Algorithm

  1. Orient X and F with dims_oriented, to observations × assets and observations × factors.
  2. Fit the wrapped prior pe.pe on F to get f_prior, the factor-axis prior result. The method passes strict to it, because pe.pe can be a BlackLittermanPrior or an EntropyPoolingPrior, which resolve view names against a universe. The method also passes cache_reg, so that such a prior stores the cache that a refit under new scenario weights reads from fpr.
  3. Reduce X to its Coverage Universe with coverage_reduction. When the caller gives active_mask, the reduction reads it in place of the active mask of pnl. Take the view of pe.re at the Coverage Universe with coverage_regression. Fit the loadings and rebuild the asset returns with factor_reconstruction to get rr and posterior_X.
  4. Project f_prior.mu and f_prior.sigma through rr with factor_lift to get mu, sigma, chol, esigma, edof and ediv. The lift processes the covariance at the count of f_prior.sigma, from moment_observation_count.
  5. Write esigma, edof and ediv onto rr with set_idiosyncratic_covariance. Under pe.rsd = true the fields hold the residual variances that the lift measured and their counts. Under pe.rsd = false, esigma and ediv hold nothing, because the lift added no residual block.
  6. Let r be the rows that f_prior.X carries, which are the last size(f_prior.X, 1) observations. A Scenario Cap of pe.pe keeps only the last rows of F. The asset side keeps the same rows, so the two blocks describe the same observations. With no cap, r is every observation. Steps 2 to 4 fit the moments over every observation, so the cut changes no moment.
  7. Write r and ts[r] onto rr as its row key with set_row_key, or nothing for ts when ts is nothing.
  8. Assemble a LowOrderPrior over the rows r of posterior_X, with the rows r of the oriented X under o_X, both cut by scenario_window. It holds the three lifted moments, the w, ens, kld and ow of the factor prior, the regression result under rr, and f_prior itself under fpr. It holds no Feature Matrix Z, because a Feature Matrix comes from the Asset Panel on the returns data, or from a producer on the distance that reads the loadings off this result.

Arguments

  • pe: Factor prior estimator.
  • X: Asset returns matrix (observations × assets).
  • F: Factor returns matrix (observations × factors).
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it. The nested prior fits the factors, and no panel describes the factor axis, so the method does not pass the panel on.
  • dims: Dimension along which the computation runs.
  • strict: Whether to throw an error if sets does not contain the desired value in sets.dict[key].
  • ts: Timestamp vector. It has one entry for each observation of X, and the loadings result records the entries of the rows the prior result carries, or nothing. The method that takes a ReturnsResult passes rd.ts.
  • active_mask: The active mask of X, of the shape of X, or nothing. When it is not nothing, it takes the place of the active mask of pnl.
  • cache_reg: If true, a prior stores on its result the cache that its refit under new scenario weights reads: a CrossSectionalFactorPrior the active mask and the Return Forecast history on its block, and an OpinionPoolingPrior the rows of pe1, its pooled weights and the cache of pe2 in pc. The refit of an entropy pooling prior around it then runs no regression and no opinion. Every other prior forwards it to the prior that it wraps. Default is false.
  • kwargs...: Additional keyword arguments passed to matrix processing and estimators.

Validation

  • dims in (1, 2).
  • If ts is not nothing, length(ts) equals the number of observations of X. Otherwise the method raises a DimensionMismatch.

Returns

  • pr::LowOrderPrior: The prior result. It holds the posterior asset returns, the mean vector, the covariance matrix, the square-root factor chol, the regression result and the factor prior result.

Related

source
PortfolioOptimisers.prior — Method
prior(pe::FactorPrior, rd::ReturnsResult; strict::Bool = false, dims::Int = 1,
      cache_reg::Bool = false) -> LowOrderPrior

Fit a Factor Prior from a ReturnsResult.

This method passes the fields of rd to the returns-matrix method, which runs the fit. It replaces prior(pe::AbstractPriorEstimator, rd::ReturnsResult) for this estimator, so that the timestamps rd.ts reach the fit. The loadings result then records the timestamp of each observation that the prior result carries.

Algorithm

  1. Check that rd carries asset returns and factor returns.
  2. Check that every UniverseSets of the tree of pe names the columns that it reads, with assert_universe_axis_order. A universe in the factor prior names the factors of rd, because that prior reads the factor returns.
  3. Drop the observations of a Shared Series gap with shared_series_view: a gap of the factor returns, and a gap of a vector benchmark under pe.x_src = ExcessSource(). The factor prior, the regression and the reconstruction then read the same observations.
  4. Read the asset returns with benchmark_relative_returns under pe.x_src, giving X.
  5. Call the returns-matrix method with X, rd.F and rd.pnl, forwarding rd.iv, rd.ivpa, rd.ne, rd.E, rd.ts, strict and cache_reg as keyword arguments, and dims = 1, because a ReturnsResult holds its observations along the rows.
  6. Record the observations that step 3 dropped with shared_series_record, and pe.x_src with stamp_returns_source.

Arguments

  • pe: Factor prior estimator.
  • rd: The returns result to use. It must carry asset returns in rd.X and factor returns in rd.F.
  • strict: Passed to the returns-matrix method of the prior, where it decides if a scenario fill or a coverage peel raises in place of a warning.
  • dims: Accepted and ignored. A ReturnsResult holds its observations along the rows, so the call passes dims = 1. The method takes no other keyword that it does not list: an unknown keyword raises the MethodError of the language, and no estimator swallows it.
  • cache_reg: If true, a prior stores on its result the cache that its refit under new scenario weights reads: a CrossSectionalFactorPrior the active mask and the Return Forecast history on its block, and an OpinionPoolingPrior the rows of pe1, its pooled weights and the cache of pe2 in pc. The refit of an entropy pooling prior around it then runs no regression and no opinion. Every other prior forwards it to the prior that it wraps. Default is false.

Validation

Returns

  • pr::LowOrderPrior: The prior the returns-matrix method fitted.

Related

source

References

[151]
J. Fan, Y. Fan and J. Lv. High dimensional covariance matrix estimation using a factor model. Journal of Econometrics 147, 186–197 (2008).