Stepwise Regression

struct PValue{__T_t} <: AbstractStepwiseRegressionCriterion

Selects factors by the statistical significance of their coefficients.

A candidate model is admissible when every one of its coefficient p-values is at or below t, the intercept excluded. This is the only criterion that reads the fitted coefficients rather than one model-wide score, so it is not a MinMaxValStepwiseRegressionCriterion and it takes its own stepwise methods. Under either algorithm the selection never returns an empty factor set: when no factor clears t, add_best_factor_after_pval_failure! adds the single best one and warns.

Fields

• t: Threshold value.

Constructors

PValue(;    t::Number = 0.05) -> PValue

Keywords correspond to the struct's fields.

Validation

• 0 < t < 1.

Examples

julia> PValue()PValue  t ┴ Float64: 0.05

Related

• AbstractStepwiseRegressionCriterion
• StepwiseRegression
• add_best_factor_after_pval_failure!

References

• [33] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.

struct ForwardSelection <: AbstractStepwiseRegressionAlgorithm

Grows the factor set from empty, adding the factor that most improves the criterion.

At each step the algorithm fits one model per excluded factor, keeps the best of them, and stops when no addition improves on the score of the set it already holds. Under a MinMaxValStepwiseRegressionCriterion the starting score is regression_threshold, the worst value that criterion can take, so the first addition always happens; under PValue the step instead admits a candidate whose p-values all clear t. The selection is therefore never empty under either criterion, which is the one behaviour that separates this tag from BackwardElimination. The steps are stated on the two methods that run them, _regression(::StepwiseRegression{<:PValue, <:ForwardSelection}, ::VecNum, ::MatNum) and _regression(::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion, <:ForwardSelection}, ::VecNum, ::MatNum).

Related

• AbstractStepwiseRegressionAlgorithm
• BackwardElimination
• StepwiseRegression
• regression_threshold

References

• [33] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.

struct BackwardElimination <: AbstractStepwiseRegressionAlgorithm

Shrinks the factor set from full, removing the factor whose removal most improves the criterion.

At each step the algorithm fits one model per included factor, each with that factor dropped, and removes the factor whose reduced model scores best. It stops when no removal improves on the score of the set it already holds. The starting score is the score of the full model, not regression_threshold; under PValue the step instead drops the factor with the largest p-value while any exceeds t. Under a MinMaxValStepwiseRegressionCriterion the selection can therefore empty, because a criterion that rewards every removal removes every factor; the asset then gets an intercept-only model, and its row of the loadings matrix is all zeros. regression warns when that happens, naming the asset, so an unexplained asset is never silent. The steps are stated on the two methods that run them, _regression(::StepwiseRegression{<:PValue, <:BackwardElimination}, ::VecNum, ::MatNum) and _regression(::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion, <:BackwardElimination}, ::VecNum, ::MatNum).

Related

• AbstractStepwiseRegressionAlgorithm
• ForwardSelection
• StepwiseRegression

References

• [33] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.

struct StepwiseRegression{__T_crit, __T_alg, __T_tgt} <: AbstractTimeSeriesRegressionEstimator

Estimates a loadings matrix by selecting a factor subset per asset, one factor at a time.

crit scores a candidate model, alg sets the direction the factor set moves in, and tgt fits it. Each asset gets its own subset, so a factor a given asset never selected carries an exact zero in that row of the loadings matrix.

Fields

• crit: Factor selection criterion. A PValue, or a Val of one symbol of STEPWISE_REGRESSION_CRITERIA.

• alg: Regression algorithm.

• tgt: Regression model target.

Constructors

StepwiseRegression(;    crit::Union{Symbol, MinMaxValStepwiseRegressionCriterion,                AbstractStepwiseRegressionCriterion} = PValue(),    alg::AbstractStepwiseRegressionAlgorithm = ForwardSelection(),    tgt::AbstractRegressionTarget = LinearModel()) -> StepwiseRegression

Keywords correspond to the struct's fields.

Validation

• If crit is a Symbol, crit in STEPWISE_REGRESSION_CRITERIA. The constructor stores Val(crit).
• If crit is Val(:adjr2), tgt is a GeneralisedLinearModel and tgt.variant is set, tgt.variant in ADJUSTED_PSEUDO_R2_VARIANTS.
• If tgt.kwargs carries a weights entry, it must be an ObsWeights and, when it is a vector, !isempty(tgt.kwargs.weights).

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

• tgt: Recursively updated via factory.

Examples

julia> StepwiseRegression()StepwiseRegression  crit ┼ PValue       │   t ┴ Float64: 0.05   alg ┼ ForwardSelection()   tgt ┼ LinearModel       │   kwargs ┴ @NamedTuple{}: NamedTuple()

Related

• AbstractStepwiseRegressionCriterion
• AbstractStepwiseRegressionAlgorithm
• AbstractRegressionTarget
• Regression
• DimensionReductionRegression
• factory

References

• [33] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.
• [34] R. R. Hocking. The analysis and selection of variables in linear regression. Biometrics 32, 1–49 (1976).

regression(re::StepwiseRegression, X::MatNum, F::MatNum)

Runs one stepwise search per asset and assembles the loadings matrix from the fits.

Each asset takes its own search, so the searches see one another only through the buffer they write into.

Algorithm

1. Allocate rr, a dense assets × (factors + 1) buffer of zeros. A factor an asset never selected keeps its zero.
2. For each asset i, do steps 3 to 5.
3. Run the stepwise search of re on column i of X, giving included.
4. Fit re.tgt to an intercept column and the columns included of F, and read its coefficients, giving params. Warn, naming the asset, and fit the intercept column alone when included is empty.
5. Write params[1] into rr[i, 1], and the remaining coefficients into the columns of rr that included names, in the order included holds them.
6. Build a Regression from the first column of rr and its remaining columns.

Arguments

• re: Stepwise regression estimator that supplies the criterion, the algorithm and the regression target.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Returns

• reg::Regression: Regression result carrying:

  • b: Intercept of each asset, a view of the first column of rr.
  • M: Coefficient of each asset and factor, a view of the remaining columns of rr. An unselected factor is an exact zero. A whole row is zero when the search selected no factor for that asset, which only BackwardElimination under a MinMaxValStepwiseRegressionCriterion can do, and which the loop warns about.
  • L: Left unset. The regression runs in the original factor basis, so reg.L reads back as reg.M through the result's swap(L, M) property rule, and size(reg.L, 2) is the number of columns of F.

Related

• StepwiseRegression
• regression
• Regression

References

• [5] D. Cajas. Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach (Springer Nature Switzerland, 2025). Section 4.1, Equations 4.2-4.3.