Stepwise Regression

PortfolioOptimisers.PValue — Type
struct PValue{__T_t} <: AbstractStepwiseRegressionCriterion

Selects factors by the statistical significance of their coefficients.

A candidate model is admissible when each p-value of its factor coefficients is at or below t. The test does not read the p-value of the intercept. This criterion reads the fitted coefficients and not one score of the whole model, so it is not a MinMaxValStepwiseRegressionCriterion, and it has its own stepwise methods. The selection is never empty under either algorithm. When no factor clears t, add_best_factor_after_pval_failure! adds the factor whose one-factor model has the smallest p-value, and warns.

Fields

  • t: Threshold value.

Constructors

PValue(;    t::Number = 0.05) -> PValue

Keywords correspond to the struct's fields.

Validation

  • 0 < t < 1.

Examples

julia> PValue()PValue  t ┴ Float64: 0.05

Related

References

  • [68] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203. Pages 191-203. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
PortfolioOptimisers.ForwardSelection — Type
struct ForwardSelection <: AbstractStepwiseRegressionAlgorithm

Grows the factor set from empty, adding the factor that most improves the criterion.

At each step the algorithm fits one model per excluded factor and keeps the best of them. It stops when no addition improves on the score of the set that it holds. Under a MinMaxValStepwiseRegressionCriterion the starting score is regression_threshold, the worst value that the criterion can take, so the first addition always happens. Under PValue the step admits a candidate whose p-values all clear t.

The selection is never empty under either criterion. Under a MinMaxValStepwiseRegressionCriterion, BackwardElimination can return an empty selection, and this tag cannot. The two methods that run the steps state them: _regression(::StepwiseRegression{<:PValue, <:ForwardSelection}, ::VecNum, ::MatNum) and _regression(::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion, <:ForwardSelection}, ::VecNum, ::MatNum).

Related

References

  • [68] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203. Pages 191-203. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
PortfolioOptimisers.BackwardElimination — Type
struct BackwardElimination <: AbstractStepwiseRegressionAlgorithm

Shrinks the factor set from full, removing the factor whose removal most improves the criterion.

At each step the algorithm fits one model per included factor, each model without that factor, and removes the factor whose reduced model scores best. It stops when no removal improves on the score of the set that it holds. The starting score is the score of the full model, not regression_threshold. Under PValue the step removes the factor with the largest p-value while a p-value exceeds t.

Under a MinMaxValStepwiseRegressionCriterion the selection can be empty. When each removal improves the score, the search removes every factor. The asset then gets an intercept-only model, and its row of the loadings matrix is all zeros. stepwise_factor_set warns and names the asset when the search selects no factor. The two methods that run the steps state them: _regression(::StepwiseRegression{<:PValue, <:BackwardElimination}, ::VecNum, ::MatNum) and _regression(::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion, <:BackwardElimination}, ::VecNum, ::MatNum).

Related

References

  • [68] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203. Pages 191-203. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
PortfolioOptimisers.StepwiseRegression — Type
struct StepwiseRegression{__T_crit, __T_alg, __T_tgt, __T_choice, __T_included, __T_x_src} <: AbstractTimeSeriesRegressionEstimator

Estimates a loadings matrix by selecting a factor subset per asset, one factor at a time.

crit scores a candidate model, alg sets the direction in which the factor set moves, and tgt fits each model. Each asset gets its own subset, so a factor that an asset does not select has an exact zero in the row of that asset in the loadings matrix.

A penalised target, such as ElasticNet, runs under :aic, :aicc and :bic alone. These criteria read the effective degrees of freedom that the fit states in StatsAPI.dof, see ElasticNetFit. The check of the inner constructor, assert_stepwise_regression, has two methods, one for a MinValStepwiseRegressionCriterion with any target, and one for every other criterion with an AbstractUnpenalisedRegressionTarget. So a refused pair is a MethodError.

CriterionUnpenalised targetPenalised targetReason
:aic, :aiccValidValidEquation (2.5) of [69] states both for a linear smoother, with the effective degrees of freedom in place of the number of columns.
:bicValidValidEquation (2.16) of [70] proposes BIC with the effective degrees of freedom when sparsity is the aim, and does not derive it.
:r2ValidRefusedIt reads no count, so it gives no penalty for the size of the model. Under MiddleOfPath each candidate has its own strength. No source treats it as a criterion for a penalised fit.
:adjr2ValidRefusedIt needs the residual count of the fit, and no source states it for this criterion under a penalty.
PValueValidRefusedA penalty makes the p-values of the coefficients invalid.
A criterion of the callerValidRefusedThe library cannot tell whether the criterion reads a count that a penalty changes.

The search names its first column, the column of ones, to the target with factory(tgt, InterceptColumn(1)), so a penalised target does not shrink the intercept. An unpenalised target ignores the call. The penalty of a fit can set the coefficient of a factor that the search selected to zero. That factor stays in the factor set of the asset, because the factor set records the search and not the active set of the last fit, and the row of M holds the zero. A ridge, ElasticNet(; theta = 0, strength = FixedStrength(; gamma)), sets no coefficient to zero, so for it the search is the only selection. The effective degrees of freedom ignores the search, as the count of a LinearModel does.

The search reads every observation, so a fit over more observations can choose another factor set. included fixes the factor set of each asset, and the fit then runs no search for that asset. choice states whether the online step of a prior that fits this regression writes the factor set of the first fit into included. A batch fit gives the same answer under both rules.

Fields

  • alg: Regression algorithm.
  • tgt: Regression model target.
  • choice: Choice Rule of the factor set of each asset. Under BatchChoice, the default, each fit searches again over every observation. Under PinnedChoice, the online step of a prior that fits this regression writes the factor set of the first fit of each asset into included of the estimator that it returns. The two agree in a batch fit. An entropy pooling prior refits the prior under the posterior weights. It holds the regression of the fit that its views read, so its refits agree too.
  • included: Factor set of each asset, one entry per column of the returns, or nothing. An entry holds the indices of the factors that the fit of that asset uses, in the order in which the search added them. The fit then runs no search for that asset. An entry of nothing makes the fit search for that asset.
  • x_src: The source of the loadings that the factor regression gives, nothing or an AbstractReturnsSource. nothing takes the loadings of the Prior Result when it holds them, and otherwise fits the regression on the asset returns rd.X. DataSource always fits on rd.X. ExcessSource always fits on the returns in excess of the benchmark, rd.X - rd.B, the Excess Frame. PriorSource takes the loadings of the Prior Result only, and refuses a Prior Result that holds none. A regression inside a prior keeps nothing, and the prior states the source.

Constructors

StepwiseRegression(;    crit::Union{Symbol, MinMaxValStepwiseRegressionCriterion,                AbstractStepwiseRegressionCriterion} = PValue(),    alg::AbstractStepwiseRegressionAlgorithm = ForwardSelection(),    tgt::AbstractRegressionTarget = LinearModel(),    choice::AbstractChoiceRule = BatchChoice(),    included::Option{<:AbstractVector{<:Option{<:AbstractVector{<:Integer}}}} = nothing,    x_src::Option{<:AbstractReturnsSource} = nothing) -> StepwiseRegression

Keywords correspond to the struct's fields.

Validation

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • tgt: Recursively updated through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

  • included: The factor sets of the selected assets.

Examples

julia> StepwiseRegression()StepwiseRegression      crit ┼ PValue           │   t ┴ Float64: 0.05       alg ┼ ForwardSelection()       tgt ┼ LinearModel           │   kwargs ┴ @NamedTuple{}: NamedTuple()    choice ┼ BatchChoice()  included ┼ nothing     x_src ┴ nothing

Related

References

  • [68] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203. Pages 191-203. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [69] C. M. Hurvich, J. S. Simonoff and C.-L. Tsai. Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 60, 271–293 (1998). Equation (2.5).
  • [70] H. Zou, T. Hastie and R. Tibshirani. On the "degrees of freedom" of the lasso. The Annals of Statistics 35, 2173–2192 (2007). Equation (2.16).
source
PortfolioOptimisers.regression — Method
regression(re::StepwiseRegression, X::MatNum, F::MatNum)

Runs one stepwise search per asset and assembles the loadings matrix from the fits.

Each asset has its own search. The search of one asset reads only the column of that asset and F, so one asset cannot change the factor set of another. An asset whose entry of re.included holds a factor set takes that set, and no search runs for it.

Algorithm

  1. Allocate rr, a dense assets × (factors + 1) buffer of zeros. A factor that an asset does not select keeps its zero.
  2. Name the first column of each design, the column of ones, to the target with factory(re.tgt, InterceptColumn(1)), and call the result tgt.
  3. For each asset i, fit it with stepwise_asset_fit!, which writes row i of rr and returns the residual count of the asset. Collect the counts in edof, whose element type is the type of the counts.
  4. Build a Regression from the first column of rr, its remaining columns, and edof.

Arguments

  • re: Stepwise regression estimator that gives the criterion, the algorithm and the regression target.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Validation

Returns

  • reg::Regression: Regression result that holds:

    • b: Intercept of each asset, a view of the first column of rr.
    • M: Coefficient of each asset and factor, a view of the remaining columns of rr. A factor that the asset does not use has an exact zero. A whole row is zero when the asset uses no factor. An empty entry of re.included gives such a row with no warning. A search that selects no factor gives it too, and stepwise_factor_set warns. Only BackwardElimination under a MinMaxValStepwiseRegressionCriterion can select no factor.
    • L: Not set. The regression runs in the original factor basis, so reg.L returns reg.M through the swap(L, M) property rule of the result, and size(reg.L, 2) is the number of columns of F.
    • edof: Degrees of freedom of the residuals of each asset, which residual_count reads off the fit of the asset. Under an unpenalised target it is the number of observations less the intercept and the factors that the search selected for that asset, an integer. Under a penalised target it is the real count that ElasticNetFit states. The count reads the selected factors alone, and not the search that selected them.

Related

References

  • [12] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 4.1, Equations 4.2-4.3. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source

References

[12]
D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025).
[68]
M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.
[69]
C. M. Hurvich, J. S. Simonoff and C.-L. Tsai. Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 60, 271–293 (1998).
[70]
H. Zou, T. Hastie and R. Tibshirani. On the ``degrees of freedom'' of the lasso. The Annals of Statistics 35, 2173–2192 (2007), arXiv:0712.0881.