Dimension reduction regression: private API
PortfolioOptimisers.DimensionReductionTarget — Type
abstract type DimensionReductionTarget <: AbstractRegressionAlgorithmAbstract supertype for all dimension reduction target types.
A subtype sets the method that reduces the factors of a DimensionReductionRegression, for example PCA or PPCA. A target must answer StatsAPI.fit(tgt, X) with a model that StatsAPI.predict and dimension_reduction_map both accept. The generic method of dimension_reduction_map reads MultivariateStats.projection. That matrix is correct for a model whose predict applies the transpose of that projection.
Related
PortfolioOptimisers.ReducedBasisFit — Type
struct ReducedBasisFit{__T_beta, __T_edof} <: AbstractResultHolds the fit of one asset in the reduced basis of a DimensionReductionRegression, with its coefficients mapped back to the original factors.
This type is an implementation detail and is not intended for direct use. _regression(::DimensionReductionRegression, ::VecNum, ::VecNum, ::VecNum, ::MatNum, ::MatNum) makes it, and regression(::DimensionReductionRegression, ::MatNum, ::MatNum) collects one for each asset into a Regression.
Fields
beta:beta: Regression coefficients in the original factor space, with the intercept as the first element.
edof:edof: Residual count of the fit in the reduced space, the number of rows of the projected design less its number of columns under an unpenalised target.
Related
PortfolioOptimisers._regression — Method
_regression(re::DimensionReductionRegression, y::VecNum, mu::VecNum,
sigma::VecNum, x1::MatNum, Vp::MatNum) -> ReducedBasisFitFits one asset in the reduced basis and maps its coefficients back to the original factors.
The method discards the intercept of the reduced space and builds a new one from the mean of the response. So a fit and its recovery agree only while mu is the mean under the weights that the fit used. When the weights match, the two paths predict the same values, with or without weights. When the standardisation uses an unweighted mean and the fit uses weights, the predictions of the two paths are different.
Mathematical definition
\[\begin{align} \hat{y} &= \hat{\beta}_{0,\mathrm{pc}} + \mathbf{X}_1 \hat{\boldsymbol{\beta}}_{\mathrm{pc}}\,, \\ \hat{\boldsymbol{\beta}} &= \mathbf{V}_p \hat{\boldsymbol{\beta}}_{\mathrm{pc}} \oslash \boldsymbol{\sigma}\,, \\ \hat{\beta}_0 &= \bar{y} - \hat{\boldsymbol{\beta}}^{\intercal} \boldsymbol{\mu}\,. \end{align}\]
Where:
- $\hat{y}$: Fitted response.
- $\hat{\beta}_{0,\mathrm{pc}}$: Intercept of the fit in the reduced space, which this method discards.
- $\hat{\boldsymbol{\beta}}_{\mathrm{pc}}$: Regression coefficients in the reduced space.
- $\hat{\boldsymbol{\beta}}$: Regression coefficients in the original factor space.
- $\hat{\beta}_0$: Intercept in the original factor space.
- $\mathbf{X}_1$: Projected factor matrix in the reduced space, with its leading column of ones.
- $\mathbf{V}_p$: Projection matrix of the dimension reduction model.
- $\boldsymbol{\sigma}$: Factor standard deviations.
- $\boldsymbol{\mu}$: Factor means.
- $\bar{y}$: Mean of the response.
- $\oslash$: Element-wise division.
Algorithm
- Read the weights of
re.retgtwithregression_target_weights, and call themw. Take the mean ofy, weighted bywwhen it is notnothing, and call itmean_y. - Fit
re.retgttox1andy, with the leading column of ones named to it byfactory(re.retgt, InterceptColumn(1)), and drop the leading coefficient. Call the restbeta_pc. - Map
beta_pcthroughVpand divide bysigma. Call the resultbeta, the coefficients in the original factor space. - Subtract the
mu-weighted sum ofbetafrommean_y, and call the resultbeta0. - Prepend
beta0tobeta. - Read the residual count of the fit of step 2 with
residual_count, and call itedof.
Arguments
re: Dimension reduction regression.y: Response vectorobservations × 1.mu: Mean vector of the original factors. It must be the mean that standardised them. That is whyprep_dim_red_regreturns it.sigma: Standard deviation vector of the original factors. It must be the scale that standardised them, for the same reason.x1: Projected factor matrix with an intercept column, fromprep_dim_red_reg.Vp: Projection matrix of the fitted dimension reduction model.
Returns
fit::ReducedBasisFit: The coefficientsbetain the original factor space, with the intercept first, and the residual countedofof the fit in the reduced space.
Related
References
- [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Pages 7 and 21-22, Equations 20-23 and 34-36.
PortfolioOptimisers.prep_dim_red_reg — Function
prep_dim_red_reg(re::DimensionReductionRegression{<:Any, <:Any, <:Any, <:Any, Nothing},
X::MatNum)
prep_dim_red_reg(re::DimensionReductionRegression{<:Any, <:Any, <:Any, <:Any, <:MatNum},
X::MatNum)Standardises the factors, fits the dimension reduction model, and projects the factors into the reduced basis.
It returns the two statistics of the standardisation with the projection, because the caller must undo that same scale. Equations 11 and 12 of [73] hold only when the standardisation and the recovery use the same statistics.
When re.proj holds the components, the method makes no reduction. It centres the factors at their mean and multiplies them by re.proj. It returns re.proj as the projection and a scale of ones, so the caller recovers the coefficients with the same formula. A constant shift of the components changes only the intercept of the fit in the reduced basis, and _regression discards that intercept. So the components need no centre of their own.
Algorithm
The method that Julia selects is the algorithm.
- Read the expected returns estimator from
re.ve.me, and call itme. UseSimpleExpectedReturns()when it isnothing. - Take the mean of each column of
Xunderme, and call itmu. re.projisnothing:- Take the standard deviation of each column of
Xunderre.ve, and call itsigma. Raise every entry to at leasteps(eltype(sigma)), so that a constant factor does not cause a division by zero. - Centre
Xwithdemean_returnsatmu, divide each column by its entry ofsigma, and transpose. Call the resultX_std. - Fit
re.drtgttoX_std, and call the fitmodel. - Project
X_stdthroughmodeland transpose. Call the resultXp, the factors in the reduced basis. - Read the matrix of the projection of
modelwithdimension_reduction_map, and call itVp.
- Take the standard deviation of each column of
re.projis a matrix:- Centre
Xwithdemean_returnsatmu, and multiply it byre.proj. Call the resultXp. - Take
Vp = re.proj, and a vector of ones assigma.
- Centre
- Prepend a column of ones to
Xp, and call the resultx1.
Arguments
re: Dimension reduction regression estimator. Itsvegives the standard deviation, and itsve.megives the mean. Whenve.meisnothing, the method usesSimpleExpectedReturns().X: Factor matrixobservations × factors, which the method reduces.
Validation
- If
re.projis a matrix,size(re.proj, 1) == size(X, 2). The method raises aDimensionMismatchotherwise.
Returns
x1::MatNum: Projected factor matrixobservations × components, with an intercept column prepended.Vp::MatNum: Projection matrixfactors × components. Its transpose maps one standardised observation to its components.mu::VecNum: Factor means that centreX.sigma::VecNum: Factor standard deviations that scaleX, or ones whenre.projholds the components.
Related
DimensionReductionRegressionPCAPPCAdemean_returnsdimension_reduction_map_regression(::DimensionReductionRegression, ::VecNum, ::VecNum, ::VecNum, ::MatNum, ::MatNum)
References
- [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Pages 4-5 and 21, Equations 8, 13-17 and 34.
PortfolioOptimisers.dimension_reduction_map — Function
dimension_reduction_map(model)
dimension_reduction_map(model::MultivariateStats.PPCA)Returns the matrix whose transpose maps one centred, standardised observation of the factors to its components, as StatsAPI.predict of model maps it.
prep_dim_red_reg regresses each asset on the components that StatsAPI.predict gives, and _regression maps the coefficients back to the factors through this matrix. The recovery is exact only when this matrix is the one that the prediction applies.
Mathematical definition
The prediction of a probabilistic PCA is the posterior mean of its latent components, Equation 9 of [74]:
\[\begin{align} \mathbb{E}[\mathbf{z} \mid \mathbf{x}] &= \mathbf{M}^{-1} \mathbf{W}^{\intercal} (\mathbf{x} - \boldsymbol{\mu}_{\mathrm{pp}})\,, \\ \mathbf{M} &= \mathbf{W}^{\intercal} \mathbf{W} + \sigma^2 \mathbf{I}\,. \end{align}\]
Where:
- $\mathbf{z}$: Latent components of one observation.
- $\mathbf{x}$: One standardised observation of the factors.
- $\mathbf{W}$: Weights of the model,
factors × components. - $\sigma^2$: Noise variance of the model.
- $\boldsymbol{\mu}_{\mathrm{pp}}$: Mean of the model.
- $\mathbf{M}$: The $q \times q$ matrix that
Statistics.covof the model gives, with $q$ the number of components. - $\mathbf{I}$: Identity matrix.
So the matrix is $\mathbf{W} \mathbf{M}^{-1}$, which differs from MultivariateStats.projection, the left singular vectors of $\mathbf{W}$.
Algorithm
The method that Julia selects is the algorithm.
- Any model: return
MultivariateStats.projection(model). For a PCA this is the matrix that the prediction applies. - A probabilistic PCA: return $\mathbf{W} \mathbf{M}^{-1}$, computed as
model.W / Statistics.cov(model).
Arguments
model: Fitted dimension reduction model, fromStatsAPI.fitof aDimensionReductionTarget.
Returns
Vp::MatNum: Matrixfactors × components.
Related
References
- [74] M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999). Section 3.4, Equation 9.
PortfolioOptimisers.pin_regression_choice — Function
pin_regression_choice(re::DimensionReductionRegression{<:Any, <:Any, <:Any,
<:PinnedChoice, Nothing},
X::MatNum, F::MatNum, w::Option{<:ObsWeights} = nothing)Writes the components of the first fit into proj of a dimension reduction regression under a PinnedChoice.
The online step of a prior calls it after the fold, over the rows of the buffer of the prior. The reduction reads F alone, so the components that it writes are the components that the fit over the same rows uses. A prior that refits the regression under observation weights, as OpinionPoolingPrior refits pe2, gives those weights as w, and the reduction reads them as that fit does.
The method writes the components in the units of the original factors. That is the matrix of the projection, with each row divided by the scale that standardised its factor. A later fit centres the factors at their mean over its own rows and multiplies them by proj. So the components keep the weights of the first fit, and the intercept of each asset is still its mean less the part that the factors explain. A regression whose proj is a matrix, and a regression under BatchChoice, get the generic method, which returns re unchanged.
Algorithm
- Return
rewhenFholds fewer than two rows. The regression refuses one row, so no fit happens yet. - Reduce
Fwithprep_dim_red_reg, and keepVpandsigma. Whenwis notnothing, the reduction is the reduction offactory(re, w). - Return
rerebuilt withproj = Vp ./ sigma.
Arguments
re: Dimension reduction regression under aPinnedChoice, whoseprojisnothing.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.w: The observation weights under which the batch fit makes the choice that the method pins, ornothingwhen that fit is unweighted.
Validation
- The rules of
prep_dim_red_reg.
Returns
re::DimensionReductionRegression: The regression with the components of the first fit inproj.
Related
PortfolioOptimisers.reader_frames — Method
reader_frames(re::Union{StepwiseRegression, DimensionReductionRegression},
path::AbstractString, refit::Bool)Returns the frames that a regression reader adds to the group of the optimiser that holds it.
A regression reader at x_src = nothing takes the loadings of the Prior Result when it holds them, and then it reads the frame of the prior. It fits on the raw returns rd.X only when the Prior Result holds none. So it counts as a raw reader only when refit is true. refits_regression finds refit from the prior estimator of the optimiser. A regression reader is outermost when it is in a constraint space or in a factor risk optimiser. A regression inside a prior keeps x_src = nothing, and the prior counts for it.
Arguments
re: Regression reader of the returns.path: The path of the value from the head, for example"pe","cle"or"r[2].pe". A warning names a reader by its path.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.
Returns
fr::ReturnsFrames: The frames ofre, fromsource_frameswithraw = refit.
Related
References
- [73]
- B. D. Fekedulegn, J. J. Colbert, R. R. Hicks Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Technical Report NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002).
- [74]
- M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999).