Dimension reduction regression
PortfolioOptimisers.PCA — Type
struct PCA{__T_kwargs} <: DimensionReductionTargetReplaces the factors with the principal components of their standardised covariance.
The principal components are those of [71] and [72]. No copy of either work was found, so the components of the standardised factors follow Equation 34 of [73].
The fit passes the kwargs field to MultivariateStats.fit(MultivariateStats.PCA, X; kwargs...), and only this field sets the retained width. pratio sets the share of the variance that the retained components must explain, and maxoutdim caps their number. The default kwargs = (;) takes the defaults of that library. On a factor matrix of full rank, these defaults retain every component and reduce nothing. On a sample of five factors of full rank, PCA() retained five components, PCA(; kwargs = (; pratio = 0.8)) retained four, and PCA(; kwargs = (; maxoutdim = 2)) retained two.
Fields
kwargs: Keyword arguments that the fit passes tofit(MultivariateStats.PCA, X; kwargs...).
Constructors
PCA(; kwargs::NamedTuple = (;)) -> PCAKeywords correspond to the struct's fields.
Examples
julia> PCA()PCA kwargs ┴ @NamedTuple{}: NamedTuple()Related
References
- [71] K. Pearson. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2, 559–572 (1901).
- [72] H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24, 417–441 (1933).
- [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Equation 34. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.PPCA — Type
struct PPCA{__T_kwargs} <: DimensionReductionTargetReplaces the factors with the latent components of a Gaussian latent-variable model.
The model is the maximum-likelihood factor analyser with an isotropic noise variance. Its latent directions span the same subspace as the principal components of PCA, and they are equal to them in the limit of zero noise. The fit is the maximum-likelihood estimate of Section 3.2 of [74]. The weights are those of Equation 7 with the rotation set to the identity. The noise variance is that of Equation 8, the mean of the discarded eigenvalues. MultivariateStats takes the eigenvalues from the sample covariance with divisor $N - 1$, where Equation 5 divides by $N$. The subspace is the same, and the eigenvalues, the weights and the noise variance carry that divisor.
The fit passes the kwargs field to MultivariateStats.fit(MultivariateStats.PPCA, X; kwargs...). The default width is one fewer than the width of PCA, because that library caps a latent-variable model at one less than the number of input dimensions. On five factors of full rank, PCA() retained five components and PPCA() retained four. maxoutdim can lower that width, and it must stay below the factor count.
StatsAPI.fit(::PPCA, ::MatNum) checks that bound before it calls MultivariateStats, and it raises a DomainError that names maxoutdim. Without the check, the fit of that library accepts the full width and returns weights of NaN. StatsAPI.predict in prep_dim_red_reg then raises an ArgumentError that names neither the cause nor the keyword. The constructor cannot make the check, because it does not see the factor matrix.
The latent components are the posterior means that StatsAPI.predict gives. So the regression maps its coefficients back through the matrix that dimension_reduction_map reads off that prediction, not through MultivariateStats.projection.
Fields
kwargs: Keyword arguments that the fit passes tofit(MultivariateStats.PPCA, X; kwargs...).
Constructors
PPCA(; kwargs::NamedTuple = (;)) -> PPCAKeywords correspond to the struct's fields.
Examples
julia> PPCA()PPCA kwargs ┴ @NamedTuple{}: NamedTuple()Related
References
- [74] M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999). Section 3.2, Equations 7-8.
PortfolioOptimisers.DimensionReductionRegression — Type
struct DimensionReductionRegression{__T_ve, __T_drtgt, __T_retgt, __T_choice, __T_proj, __T_x_src} <: AbstractTimeSeriesRegressionEstimatorEstimates a loadings matrix by regressing each asset on the leading components of the factors.
drtgt maps the standardised factors to a basis of orthogonal components, and retgt fits each asset in that basis. The basis is smaller than the factor set only when drtgt retains fewer components than factors. The method then maps the coefficients back to the original factors. ve gives the mean, and the standard deviation by which that map divides. The mean comes from the expected returns estimator ve.me, or from SimpleExpectedReturns() when ve.me is nothing. Every asset keeps every factor, which is not true of StepwiseRegression.
The standardisation and the recovery read the same statistics. prep_dim_red_reg computes them from ve, and _regression recovers the coefficients with the pair that it returns. So a weighted ve, which factory builds from the observation weights, gives its weights to both steps. Equations 8, 11, 12, 35 and 36 of [73] need this. That work fits the regression on the principal components of the standardised factors, with the eigenvectors of their correlation matrix. A probabilistic target puts the matrix of Equation 9 of [74] in place of the eigenvectors, and neither source states this composition.
A penalised retgt, such as ElasticNet, keeps the fit and the recovery in agreement. The estimator names the column of ones with InterceptColumn, so the target does not penalise the intercept. The intercept is then the weighted mean of the residual of the slopes. The components have a weighted mean of zero, so the intercept is the weighted mean of the response, which the recovery rebuilds. The condition is the same as for an unpenalised target: ve and retgt carry the same weights, as factory gives them. The penalty shrinks the coefficients of the components, which are in the units of the standardised factors. The degrees of freedom of the residuals count one for each column of the reduced design, and a penalised fit has fewer.
The reduction reads every observation, so a fit over more observations can reduce to other components. proj fixes the components as linear combinations of the original factors, and the fit then makes no reduction. choice states whether the online step of a prior that fits this regression writes the components of the first fit into proj. A batch fit gives the same answer under both rules.
Fields
ve: Variance estimator.
drtgt: Dimension reduction target.
retgt: Regression model target.
choice: Choice Rule of the components. UnderBatchChoice, the default, each fit reduces the factors again over every observation. UnderPinnedChoice, the online step of a prior that fits this regression writes the components of the first fit intoprojof the estimator that it returns. The two agree in a batch fit. An entropy pooling prior refits the prior under the posterior weights. It holds the regression of the fit that its views read, so its refits agree too.
proj: Components as linear combinations of the original factors,factors × components, ornothing. Columnkholds the weight of each factor in componentk, after the fit subtracts the mean of the factors. A matrix makes the fit regress each asset on these components and make no reduction.nothingmakes the fit reduce the factors withdrtgt.
x_src: The source of the loadings that the factor regression gives,nothingor anAbstractReturnsSource.nothingtakes the loadings of the Prior Result when it holds them, and otherwise fits the regression on the asset returnsrd.X.DataSourcealways fits onrd.X.ExcessSourcealways fits on the returns in excess of the benchmark,rd.X - rd.B, the Excess Frame.PriorSourcetakes the loadings of the Prior Result only, and refuses a Prior Result that holds none. A regression inside a prior keepsnothing, and the prior states the source.
Constructors
DimensionReductionRegression(; ve::AbstractVarianceEstimator = SimpleVariance(), drtgt::DimensionReductionTarget = PCA(), retgt::AbstractRegressionTarget = LinearModel(), choice::AbstractChoiceRule = BatchChoice(), proj::Option{<:MatNum} = nothing, x_src::Option{<:AbstractReturnsSource} = nothing) -> DimensionReductionRegressionKeywords correspond to the struct's fields.
Validation
retgtstates its observation weights throughregression_target_weights, which returnsretgt.kwargs.weightsfor aLinearModelor aGeneralisedLinearModel. The constructor raises anArgumentErrorfor a target of the caller that has no such method.- If
regression_target_weights(retgt)is notnothing, it must be anObsWeightsand, when it is a vector, not empty. - If
projis notnothing,!isempty(proj), and every entry ofprojis finite, asassert_nonempty_finite_valchecks. x_src in ALL_RETURNS_SOURCES, asassert_returns_sourcechecks. A regression reader admits every source.
Propagated parameters
factory passes its arguments on to these @fprop fields:
ve: Recursively updated throughfactory.drtgt: Recursively updated throughfactory.retgt: Recursively updated throughfactory.
View parameters
port_opt_view takes the view of these @vprop fields at the selected indices:
ve: Recursively viewed throughport_opt_view.
Examples
julia> DimensionReductionRegression()DimensionReductionRegression ve ┼ SimpleVariance │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ w ┼ nothing │ corrected ┴ Bool: true drtgt ┼ PCA │ kwargs ┴ @NamedTuple{}: NamedTuple() retgt ┼ LinearModel │ kwargs ┴ @NamedTuple{}: NamedTuple() choice ┼ BatchChoice() proj ┼ nothing x_src ┴ nothingRelated
AbstractTimeSeriesRegressionEstimatorAbstractVarianceEstimatorDimensionReductionTargetAbstractRegressionTargetregression_target_weightsStepwiseRegressionRegressionPinnedChoiceAbstractReturnsSourceresolve_factor_regressionfactoryport_opt_view
References
- [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Pages 4-7 and 21-22, Equations 8, 11-12, 20-23 and 34-36.
- [74] M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999). Section 3.4, Equation 9.
StatsAPI.fit — Method
StatsAPI.fit(drtgt::PCA, X::MatNum)Fits a principal component analysis model of MultivariateStats to the data matrix X, with the keyword arguments of drtgt.
Algorithm
- Read
drtgt.kwargs, which sets the retained width throughpratioandmaxoutdim. - Call
MultivariateStats.fitonMultivariateStats.PCAwithXand those keyword arguments, and return the fitted model.
Arguments
drtgt:PCAtarget that holds the keyword arguments of the fit.X: Data matrixfactors × observations, which the caller standardises.
Returns
model::MultivariateStats.PCA: The fitted model.
Related
StatsAPI.fit — Method
StatsAPI.fit(drtgt::PPCA, X::MatNum)Fits a probabilistic principal component analysis model of MultivariateStats to the data matrix X, with the keyword arguments of drtgt.
Algorithm
- Read
drtgt.kwargs, which sets the retained width throughmaxoutdim. - If
maxoutdimis present, check it against the number of factors,size(X, 1). - Call
MultivariateStats.fitonMultivariateStats.PPCAwithXand those keyword arguments, and return the fitted model.
Arguments
drtgt:PPCAtarget that holds the keyword arguments of the fit.X: Data matrixfactors × observations, which the caller standardises.
Validation
- If
drtgt.kwargsholds amaxoutdimentry,0 < drtgt.kwargs.maxoutdim < size(X, 1)must hold.MultivariateStatscaps a probabilistic PCA at one latent dimension fewer than the number of factors. Its own fit accepts the full width and returns a model whose weights areNaN. Without this check, the failure reaches the caller later, insideprep_dim_red_reg, with a message that names neither the cause nor the keyword.
Returns
model::MultivariateStats.PPCA: The fitted model.
Related
PortfolioOptimisers.regression — Method
regression(re::DimensionReductionRegression, X::MatNum, F::MatNum)Reduces the factors once and regresses every asset on the same reduced basis.
The method fits the reduction on F alone. So the reduction does not depend on the assets, and the response of one asset cannot change the loadings of another.
Algorithm
- Allocate
rr, a denseassets × (factors + 1)buffer of zeros. - Reduce
Fwithprep_dim_red_reg, and call the resultsf1,Vp,muandsigma. - For each asset
i, fit that column ofXin the reduced basis, map the coefficients back, and write the result into rowiofrr. Write the residual count of the fit intoedof[i]. - Take the first column of
rrasband its remaining columns asM. - Undo the rescaling and the projection of
Min turn. Call the resultL, the coefficients in the reduced basis. - Negate each column of
Lwhose sum is negative. The sign of a component is arbitrary, and a build flag such as--check-bounds=yescan flip it. After this step, the equal-weight portfolio has a non-negative exposure to each component.Mdoes not change, because a component and its coefficient change sign together. - Build a
Regressionfromb,M,Landedof.
Arguments
re: Dimension reduction regression estimator that gives the variance estimator, the dimension reduction target and the regression target.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Validation
- The rules of
assert_finite_factor_rowsforF.
Returns
reg::Regression: Regression result that holds:b: Intercept of each asset, a view of the first column ofrr.M: Coefficient of each asset and factor in the original factor space, a view of the remaining columns ofrr. Every asset keeps every factor, soMhas no structural zero.L: Coefficient of each asset and retained component, $(\mathbf{M} \odot \boldsymbol{\sigma}^{\intercal}) \mathbf{V}_p^{+\intercal}$. It reproduces the coefficients of the fits of step 3 in the reduced space, up to the sign of each component that step 6 sets. On a 200×5 sample the two agreed to2.2e-16.size(L, 2)is the number of retained components, which is the width of a risk decomposition.edof: Degrees of freedom of the residuals of each asset, whichresidual_countreads off the fit of the asset. Under an unpenalised target it is the number of observations less the intercept and the retained components, the same integer for every asset. Under a penalised target it is the real count of each asset thatElasticNetFitstates.
Related
References
- [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Pages 4-7 and 21-22, Equations 8, 11-12, 20-23 and 34-36.
PortfolioOptimisers.returns_source — Method
returns_source(re::Union{StepwiseRegression, DimensionReductionRegression})Returns the source that a regression reader states, which is its x_src field.
A prior that holds a regression reads the returns data and gives the regression a matrix. So the prior refuses a regression whose source is not nothing. assert_inner_reader makes that check, and it reads the source with this method.
Arguments
re: Regression reader of the returns.
Returns
x_src::Option{<:AbstractReturnsSource}: The source of the regression.
Related
References
- [71]
- K. Pearson. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2, 559–572 (1901).
- [72]
- H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24, 417–441 (1933).
- [73]
- B. D. Fekedulegn, J. J. Colbert, R. R. Hicks Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Technical Report NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002).
- [74]
- M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999).