Dimension reduction regression

PortfolioOptimisers.PCA — Type
struct PCA{__T_kwargs} <: DimensionReductionTarget

Replaces the factors with the principal components of their standardised covariance.

The principal components are those of [71] and [72]. No copy of either work was found, so the components of the standardised factors follow Equation 34 of [73].

The fit passes the kwargs field to MultivariateStats.fit(MultivariateStats.PCA, X; kwargs...), and only this field sets the retained width. pratio sets the share of the variance that the retained components must explain, and maxoutdim caps their number. The default kwargs = (;) takes the defaults of that library. On a factor matrix of full rank, these defaults retain every component and reduce nothing. On a sample of five factors of full rank, PCA() retained five components, PCA(; kwargs = (; pratio = 0.8)) retained four, and PCA(; kwargs = (; maxoutdim = 2)) retained two.

Fields

  • kwargs: Keyword arguments that the fit passes to fit(MultivariateStats.PCA, X; kwargs...).

Constructors

PCA(;    kwargs::NamedTuple = (;)) -> PCA

Keywords correspond to the struct's fields.

Examples

julia> PCA()PCA  kwargs ┴ @NamedTuple{}: NamedTuple()

Related

References

  • [71] K. Pearson. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2, 559–572 (1901).
  • [72] H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24, 417–441 (1933).
  • [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Equation 34. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
PortfolioOptimisers.PPCA — Type
struct PPCA{__T_kwargs} <: DimensionReductionTarget

Replaces the factors with the latent components of a Gaussian latent-variable model.

The model is the maximum-likelihood factor analyser with an isotropic noise variance. Its latent directions span the same subspace as the principal components of PCA, and they are equal to them in the limit of zero noise. The fit is the maximum-likelihood estimate of Section 3.2 of [74]. The weights are those of Equation 7 with the rotation set to the identity. The noise variance is that of Equation 8, the mean of the discarded eigenvalues. MultivariateStats takes the eigenvalues from the sample covariance with divisor $N - 1$, where Equation 5 divides by $N$. The subspace is the same, and the eigenvalues, the weights and the noise variance carry that divisor.

The fit passes the kwargs field to MultivariateStats.fit(MultivariateStats.PPCA, X; kwargs...). The default width is one fewer than the width of PCA, because that library caps a latent-variable model at one less than the number of input dimensions. On five factors of full rank, PCA() retained five components and PPCA() retained four. maxoutdim can lower that width, and it must stay below the factor count.

StatsAPI.fit(::PPCA, ::MatNum) checks that bound before it calls MultivariateStats, and it raises a DomainError that names maxoutdim. Without the check, the fit of that library accepts the full width and returns weights of NaN. StatsAPI.predict in prep_dim_red_reg then raises an ArgumentError that names neither the cause nor the keyword. The constructor cannot make the check, because it does not see the factor matrix.

The latent components are the posterior means that StatsAPI.predict gives. So the regression maps its coefficients back through the matrix that dimension_reduction_map reads off that prediction, not through MultivariateStats.projection.

Fields

  • kwargs: Keyword arguments that the fit passes to fit(MultivariateStats.PPCA, X; kwargs...).

Constructors

PPCA(;    kwargs::NamedTuple = (;)) -> PPCA

Keywords correspond to the struct's fields.

Examples

julia> PPCA()PPCA  kwargs ┴ @NamedTuple{}: NamedTuple()

Related

References

  • [74] M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999). Section 3.2, Equations 7-8.
source
PortfolioOptimisers.DimensionReductionRegression — Type
struct DimensionReductionRegression{__T_ve, __T_drtgt, __T_retgt, __T_choice, __T_proj, __T_x_src} <: AbstractTimeSeriesRegressionEstimator

Estimates a loadings matrix by regressing each asset on the leading components of the factors.

drtgt maps the standardised factors to a basis of orthogonal components, and retgt fits each asset in that basis. The basis is smaller than the factor set only when drtgt retains fewer components than factors. The method then maps the coefficients back to the original factors. ve gives the mean, and the standard deviation by which that map divides. The mean comes from the expected returns estimator ve.me, or from SimpleExpectedReturns() when ve.me is nothing. Every asset keeps every factor, which is not true of StepwiseRegression.

The standardisation and the recovery read the same statistics. prep_dim_red_reg computes them from ve, and _regression recovers the coefficients with the pair that it returns. So a weighted ve, which factory builds from the observation weights, gives its weights to both steps. Equations 8, 11, 12, 35 and 36 of [73] need this. That work fits the regression on the principal components of the standardised factors, with the eigenvectors of their correlation matrix. A probabilistic target puts the matrix of Equation 9 of [74] in place of the eigenvectors, and neither source states this composition.

A penalised retgt, such as ElasticNet, keeps the fit and the recovery in agreement. The estimator names the column of ones with InterceptColumn, so the target does not penalise the intercept. The intercept is then the weighted mean of the residual of the slopes. The components have a weighted mean of zero, so the intercept is the weighted mean of the response, which the recovery rebuilds. The condition is the same as for an unpenalised target: ve and retgt carry the same weights, as factory gives them. The penalty shrinks the coefficients of the components, which are in the units of the standardised factors. The degrees of freedom of the residuals count one for each column of the reduced design, and a penalised fit has fewer.

The reduction reads every observation, so a fit over more observations can reduce to other components. proj fixes the components as linear combinations of the original factors, and the fit then makes no reduction. choice states whether the online step of a prior that fits this regression writes the components of the first fit into proj. A batch fit gives the same answer under both rules.

Fields

  • ve: Variance estimator.
  • drtgt: Dimension reduction target.
  • retgt: Regression model target.
  • choice: Choice Rule of the components. Under BatchChoice, the default, each fit reduces the factors again over every observation. Under PinnedChoice, the online step of a prior that fits this regression writes the components of the first fit into proj of the estimator that it returns. The two agree in a batch fit. An entropy pooling prior refits the prior under the posterior weights. It holds the regression of the fit that its views read, so its refits agree too.
  • proj: Components as linear combinations of the original factors, factors × components, or nothing. Column k holds the weight of each factor in component k, after the fit subtracts the mean of the factors. A matrix makes the fit regress each asset on these components and make no reduction. nothing makes the fit reduce the factors with drtgt.
  • x_src: The source of the loadings that the factor regression gives, nothing or an AbstractReturnsSource. nothing takes the loadings of the Prior Result when it holds them, and otherwise fits the regression on the asset returns rd.X. DataSource always fits on rd.X. ExcessSource always fits on the returns in excess of the benchmark, rd.X - rd.B, the Excess Frame. PriorSource takes the loadings of the Prior Result only, and refuses a Prior Result that holds none. A regression inside a prior keeps nothing, and the prior states the source.

Constructors

DimensionReductionRegression(;    ve::AbstractVarianceEstimator = SimpleVariance(),    drtgt::DimensionReductionTarget = PCA(),    retgt::AbstractRegressionTarget = LinearModel(),    choice::AbstractChoiceRule = BatchChoice(),    proj::Option{<:MatNum} = nothing,    x_src::Option{<:AbstractReturnsSource} = nothing) -> DimensionReductionRegression

Keywords correspond to the struct's fields.

Validation

  • retgt states its observation weights through regression_target_weights, which returns retgt.kwargs.weights for a LinearModel or a GeneralisedLinearModel. The constructor raises an ArgumentError for a target of the caller that has no such method.
  • If regression_target_weights(retgt) is not nothing, it must be an ObsWeights and, when it is a vector, not empty.
  • If proj is not nothing, !isempty(proj), and every entry of proj is finite, as assert_nonempty_finite_val checks.
  • x_src in ALL_RETURNS_SOURCES, as assert_returns_source checks. A regression reader admits every source.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • ve: Recursively updated through factory.
  • drtgt: Recursively updated through factory.
  • retgt: Recursively updated through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Examples

julia> DimensionReductionRegression()DimensionReductionRegression      ve ┼ SimpleVariance         │          me ┼ SimpleExpectedReturns         │             │   w ┴ nothing         │           w ┼ nothing         │   corrected ┴ Bool: true   drtgt ┼ PCA         │   kwargs ┴ @NamedTuple{}: NamedTuple()   retgt ┼ LinearModel         │   kwargs ┴ @NamedTuple{}: NamedTuple()  choice ┼ BatchChoice()    proj ┼ nothing   x_src ┴ nothing

Related

References

  • [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Pages 4-7 and 21-22, Equations 8, 11-12, 20-23 and 34-36.
  • [74] M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999). Section 3.4, Equation 9.
source
StatsAPI.fit — Method
StatsAPI.fit(drtgt::PCA, X::MatNum)

Fits a principal component analysis model of MultivariateStats to the data matrix X, with the keyword arguments of drtgt.

Algorithm

  1. Read drtgt.kwargs, which sets the retained width through pratio and maxoutdim.
  2. Call MultivariateStats.fit on MultivariateStats.PCA with X and those keyword arguments, and return the fitted model.

Arguments

  • drtgt: PCA target that holds the keyword arguments of the fit.
  • X: Data matrix factors × observations, which the caller standardises.

Returns

  • model::MultivariateStats.PCA: The fitted model.

Related

source
StatsAPI.fit — Method
StatsAPI.fit(drtgt::PPCA, X::MatNum)

Fits a probabilistic principal component analysis model of MultivariateStats to the data matrix X, with the keyword arguments of drtgt.

Algorithm

  1. Read drtgt.kwargs, which sets the retained width through maxoutdim.
  2. If maxoutdim is present, check it against the number of factors, size(X, 1).
  3. Call MultivariateStats.fit on MultivariateStats.PPCA with X and those keyword arguments, and return the fitted model.

Arguments

  • drtgt: PPCA target that holds the keyword arguments of the fit.
  • X: Data matrix factors × observations, which the caller standardises.

Validation

  • If drtgt.kwargs holds a maxoutdim entry, 0 < drtgt.kwargs.maxoutdim < size(X, 1) must hold. MultivariateStats caps a probabilistic PCA at one latent dimension fewer than the number of factors. Its own fit accepts the full width and returns a model whose weights are NaN. Without this check, the failure reaches the caller later, inside prep_dim_red_reg, with a message that names neither the cause nor the keyword.

Returns

  • model::MultivariateStats.PPCA: The fitted model.

Related

source
PortfolioOptimisers.regression — Method
regression(re::DimensionReductionRegression, X::MatNum, F::MatNum)

Reduces the factors once and regresses every asset on the same reduced basis.

The method fits the reduction on F alone. So the reduction does not depend on the assets, and the response of one asset cannot change the loadings of another.

Algorithm

  1. Allocate rr, a dense assets × (factors + 1) buffer of zeros.
  2. Reduce F with prep_dim_red_reg, and call the results f1, Vp, mu and sigma.
  3. For each asset i, fit that column of X in the reduced basis, map the coefficients back, and write the result into row i of rr. Write the residual count of the fit into edof[i].
  4. Take the first column of rr as b and its remaining columns as M.
  5. Undo the rescaling and the projection of M in turn. Call the result L, the coefficients in the reduced basis.
  6. Negate each column of L whose sum is negative. The sign of a component is arbitrary, and a build flag such as --check-bounds=yes can flip it. After this step, the equal-weight portfolio has a non-negative exposure to each component. M does not change, because a component and its coefficient change sign together.
  7. Build a Regression from b, M, L and edof.

Arguments

  • re: Dimension reduction regression estimator that gives the variance estimator, the dimension reduction target and the regression target.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Validation

Returns

  • reg::Regression: Regression result that holds:

    • b: Intercept of each asset, a view of the first column of rr.
    • M: Coefficient of each asset and factor in the original factor space, a view of the remaining columns of rr. Every asset keeps every factor, so M has no structural zero.
    • L: Coefficient of each asset and retained component, $(\mathbf{M} \odot \boldsymbol{\sigma}^{\intercal}) \mathbf{V}_p^{+\intercal}$. It reproduces the coefficients of the fits of step 3 in the reduced space, up to the sign of each component that step 6 sets. On a 200×5 sample the two agreed to 2.2e-16. size(L, 2) is the number of retained components, which is the width of a risk decomposition.
    • edof: Degrees of freedom of the residuals of each asset, which residual_count reads off the fit of the asset. Under an unpenalised target it is the number of observations less the intercept and the retained components, the same integer for every asset. Under a penalised target it is the real count of each asset that ElasticNetFit states.

Related

References

  • [73] B. D. Fekedulegn, J. J. Colbert, R. R. Hicks, Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Research Paper NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002). Pages 4-7 and 21-22, Equations 8, 11-12, 20-23 and 34-36.
source
PortfolioOptimisers.returns_source — Method
returns_source(re::Union{StepwiseRegression, DimensionReductionRegression})

Returns the source that a regression reader states, which is its x_src field.

A prior that holds a regression reads the returns data and gives the regression a matrix. So the prior refuses a regression whose source is not nothing. assert_inner_reader makes that check, and it reads the source with this method.

Arguments

  • re: Regression reader of the returns.

Returns

  • x_src::Option{<:AbstractReturnsSource}: The source of the regression.

Related

source

References

[71]
K. Pearson. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2, 559–572 (1901).
[72]
H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24, 417–441 (1933).
[73]
B. D. Fekedulegn, J. J. Colbert, R. R. Hicks Jr. and M. E. Schuckers. Coping with multicollinearity: an example on application of principal components regression in dendroecology. Technical Report NE-RP-721 (U.S. Department of Agriculture, Forest Service, Northeastern Research Station, 2002).
[74]
M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61, 611–622 (1999).