Stepwise Regression: private API

PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:PValue, <:ForwardSelection}, x::VecNum,
           F::MatNum)

Grows a factor set from empty, and admits a model only when its p-values all clear the threshold.

A candidate is admissible only when each factor coefficient of the candidate model is significant. So the method rejects a factor that is significant alone when it makes a factor of the set insignificant.

Algorithm

  1. Set included empty and val to zero, so the loop runs at least once.
  2. While val does not exceed re.crit.t, do steps 3 to 6.
  3. Take the factors that included does not hold, and call them excluded.
  4. For each i of excluded, fit re.tgt to an intercept column and the columns [included; i], and read the p-value of column i. Call it test_pval. Keep i as new_factor when test_pval is the smallest so far and no p-value of that candidate model exceeds re.crit.t. Keep the p-values of that model as pvals.
  5. Stop the loop when step 4 kept no factor. Otherwise push new_factor onto included.
  6. Set val to the largest entry of pvals.
  7. Call add_best_factor_after_pval_failure!, which acts only when included is still empty.

Arguments

  • re: Stepwise regression estimator with a PValue criterion and the ForwardSelection algorithm.
  • x: Response vector, observations × 1.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Returns

  • included::Vector{Int}: Indices of the selected factors, in the order in which the loop added them, so the vector is not sorted. It holds at least one index.

Related

source
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion,
                                  <:ForwardSelection}, x::VecNum, F::MatNum)

Grows a factor set from empty, adding the factor whose model scores best.

The starting score is the worst value that the criterion can take, so the first addition always happens and the selection is never empty. get_forward_reg_incl_excl! reads the direction from regression_polarity, so one loop works for a minimised and for a maximised criterion.

Algorithm

  1. Set included empty and excluded to every factor index, in ascending order. Name the first column of each design, the column of ones, to the target with factory(re.tgt, InterceptColumn(1)), and call the result tgt.
  2. Read the criterion from re.crit and re.tgt with regression_criterion_func, and call it criterion_func. Read the starting score from regression_threshold, and call it t.
  3. Fill value with that same worst score, one entry per factor.
  4. Do steps 5 to 7 at most once per observation.
  5. For each i of excluded, fit tgt to an intercept column and the columns [included; i], and write criterion_func of that fit into value[i].
  6. Call get_forward_reg_incl_excl!, which moves the best factor and returns the new t.
  7. Stop when step 6 moved no factor.

Arguments

  • re: Stepwise regression estimator with a MinMaxValStepwiseRegressionCriterion criterion and the ForwardSelection algorithm.
  • x: Response vector, observations × 1.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Returns

  • included::Vector{Int}: Indices of the selected factors, in the order in which the loop added them, so the vector is not sorted. It holds at least one index.

Related

source
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:PValue, <:BackwardElimination}, x::VecNum,
           F::MatNum)

Shrinks a factor set from full, and removes the factor of largest p-value while a p-value exceeds the threshold.

Algorithm

  1. Fit re.tgt to an intercept column and every column of F, read the p-values of the factor coefficients, and set val to the largest of them.
  2. Set included to every factor index and excluded empty.
  3. While val exceeds re.crit.t, do steps 4 to 6.
  4. Set included to the factors that excluded does not hold. Stop the loop when included is empty.
  5. Fit re.tgt to an intercept column and the columns included, and read the p-values of the factor coefficients. Call them pvals.
  6. Set val to the largest entry of pvals, and push the factor of that entry onto excluded. Only step 4 reads excluded, so the push has an effect only when step 3 runs the loop again. The push of the last iteration does not change the result.
  7. Call add_best_factor_after_pval_failure!, which acts only when included is empty.

Arguments

  • re: Stepwise regression estimator with a PValue criterion and the BackwardElimination algorithm.
  • x: Response vector, observations × 1.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Returns

  • included::VecInt: Indices of the selected factors, in ascending order. It holds at least one index. It is the 1:size(F, 2) range itself when the full model already passes the threshold, and a Vector{Int} otherwise.

Related

source
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion,
                                  <:BackwardElimination}, x::VecNum, F::MatNum)

Shrinks a factor set from full, removing the factor whose reduced model scores best.

value[j] is the score of the model that omits j. So "best" has the same meaning as in the forward direction, and not the inverse meaning. Under a minimised criterion the method removes the factor of lowest value, and under a maximised criterion it removes the factor of highest value.

Algorithm

  1. Set included to every factor index, in ascending order. Name the first column of each design, the column of ones, to the target with factory(re.tgt, InterceptColumn(1)), and call the result tgt.
  2. Read the criterion from re.crit and re.tgt with regression_criterion_func, and call it criterion_func.
  3. Fit tgt to an intercept column and every column of F, and set t to criterion_func of that full model.
  4. Fill value with the worst score the criterion can take, one entry per factor.
  5. Do steps 6 to 8 at most once per observation.
  6. For each factor of included, fit tgt to an intercept column and the other columns of included, and write criterion_func of that fit into that factor's entry of value. Fit the intercept column alone when included holds one factor.
  7. Call get_backward_reg_incl!, which removes the best factor and returns the new t.
  8. Stop when step 7 removed no factor.

Arguments

  • re: Stepwise regression estimator with a MinMaxValStepwiseRegressionCriterion criterion and the BackwardElimination algorithm.
  • x: Response vector, observations × 1.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Returns

  • included::Vector{Int}: Indices of the selected factors, in ascending order. It can be empty. When each removal improves the score, the search removes every factor. This happens often under an information criterion on a response that the factors do not explain. stepwise_factor_set warns and names the asset on an empty return, and regression then fits the intercept column alone.

Related

source
PortfolioOptimisers.add_best_factor_after_pval_failure! — Function
add_best_factor_after_pval_failure!(tgt::AbstractRegressionTarget,
                                    included::VecInt, F::MatNum,
                                    x::VecNum)

Adds the factor of smallest p-value when a p-value search selected none.

Both PValue methods of _regression call it last, so a p-value search never returns an empty factor set. It does nothing when included already holds a factor. It warns each time that it adds a factor, because that factor failed the threshold in the search.

Algorithm

  1. Return at once when included is not empty.
  2. For each factor i of F, fit tgt to an intercept column and column i, and read the p-value of column i. Call it test_pval.
  3. Keep the smallest test_pval of step 2 as best_pval, and its factor as new_factor.
  4. Warn, and name new_factor and best_pval.
  5. Push new_factor onto included.

Arguments

  • tgt: Regression target that fits each candidate model.
  • included: Indices of the factors that the search selected. The function writes into it.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.
  • x: Response vector, observations × 1.

Returns

  • nothing. included gains one index when it is empty, and it does not change otherwise.

Related

source
PortfolioOptimisers.get_forward_reg_incl_excl! — Function
get_forward_reg_incl_excl!(crit::MinMaxValStepwiseRegressionCriterion,
                           value::VecNum, excluded::VecInt,
                           included::VecInt, t::Number)

Moves the best excluded factor into included when it improves the criterion.

regression_polarity states the direction. A minimised criterion uses findmin and <, and a maximised criterion uses findmax and >. The search reads the whole value vector, and it still finds the best excluded factor. The entry of an included factor holds the score of the step that added it. t holds the score of the last step, and each step improves on the step before it, so no entry of an included factor improves on t. Thus a score that improves on t belongs to an excluded factor, and searchsortedfirst finds its index in excluded.

Algorithm

  1. Read best and improves from regression_polarity.
  2. Find the best entry of value with best. Call it val, and its index idx.
  3. Return t unchanged when val does not improve on t.
  4. Otherwise find the position of idx in excluded, move that entry to the end of included, and set t to val.

Arguments

  • crit: Stepwise regression criterion. Its polarity states which direction is better.
  • value: Criterion score of each factor. Entry i is the score of the model that adds factor i to included.
  • excluded: Indices of the factors outside the model, in ascending order. The function writes into it.
  • included: Indices of the factors inside the model, in the order in which the search added them. The function writes into it.
  • t: Threshold value.

Returns

  • t::Number: The score of the factor that moved, or the input t when no factor moved.

Related

source
PortfolioOptimisers.get_backward_reg_incl! — Function
get_backward_reg_incl!(crit::MinMaxValStepwiseRegressionCriterion, value::VecNum,
                       included::VecInt, t::Number)

Removes the best included factor from included when its removal improves the criterion.

regression_polarity states the direction. A minimised criterion uses findmin and <, and a maximised criterion uses findmax and >. The search reads the whole value vector, and it still finds the best included factor. The entry of a removed factor holds the score of the step that removed it. t holds the score of the last step, and each step improves on the step before it, so no entry of a removed factor improves on t. Thus a score that improves on t belongs to an included factor, and searchsortedfirst finds its index in included.

Algorithm

  1. Read best and improves from regression_polarity.
  2. Find the best entry of value with best. Call it val, and its index idx. This entry is the best model that one removal can make.
  3. Return t unchanged when val does not improve on t.
  4. Otherwise find the position of idx in included, remove that entry, and set t to val.

Arguments

  • crit: Stepwise regression criterion. Its polarity states which direction is better.
  • value: Criterion score of each factor. Entry j is the score of the model that omits factor j, so the best entry names the removal that improves the score most.
  • included: Indices of the factors inside the model, in ascending order. The function writes into it.
  • t: Threshold value.

Returns

  • t::Number: The score of the model that the removal leaves, or the input t when the function removed no factor.

Related

source
PortfolioOptimisers.assert_stepwise_included — Function
assert_stepwise_included(included::Nothing)
assert_stepwise_included(included::AbstractVector)
assert_stepwise_included(included::Nothing, X::MatNum, F::MatNum)
assert_stepwise_included(included::AbstractVector, X::MatNum, F::MatNum)

Checks the factor sets that the included field of a StepwiseRegression holds.

The one-argument form is the check of the constructor, which sees no data. The three-argument form is the check of the fit. It compares the field with the returns and the factor returns.

Algorithm

The method that Julia selects is the algorithm. included of nothing passes both forms.

Arguments

  • included: The factor sets, one entry per asset, or nothing.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.

Validation

  • One argument: !isempty(included), or the check raises an IsEmptyError. Each entry is nothing or a vector of positive factor indices with no repeated index, or the check raises a DomainError.
  • Three arguments: length(included) == size(X, 2), and every index of each entry is at most size(F, 2). The check raises a DimensionMismatch otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_stepwise_regression — Function
assert_stepwise_regression(crit::MinValStepwiseRegressionCriterion,
                           tgt::AbstractRegressionTarget,
                           included::Option{<:AbstractVector})
assert_stepwise_regression(crit::Union{MaxValStepwiseRegressionCriterion,
                                       AbstractStepwiseRegressionCriterion},
                           tgt::AbstractUnpenalisedRegressionTarget,
                           included::Option{<:AbstractVector})

Checks the criterion, the target and the factor sets of a StepwiseRegression.

The inner constructor calls it. The two methods state the valid pairs of criterion and target in their signatures, so a refused pair, a penalised target under a criterion other than a MinValStepwiseRegressionCriterion, finds no method and throws a MethodError. The pair rule sits here and not in two methods of the inner constructor, because @concrete adds a generic positional constructor, and a call that matches no inner method reaches that constructor with no check.

Algorithm

Each method holds the checks that its pairs reach, so the copies of the rule on the weights agree.

  1. Under a MaxValStepwiseRegressionCriterion, check the variant of a GeneralisedLinearModel under :adjr2. The method of a MinValStepwiseRegressionCriterion has no such step, because :adjr2 is a maximised criterion.
  2. Check the weights entry of the kwargs of a LinearModel or a GeneralisedLinearModel. Both methods hold this step.
  3. Check included with assert_stepwise_included.

Arguments

  • crit: The criterion, as the crit field holds it.
  • tgt: The regression target.
  • included: The factor sets, one entry per asset, or nothing.

Validation

  • If crit is Val(:adjr2), tgt is a GeneralisedLinearModel and tgt.variant is set, tgt.variant in ADJUSTED_PSEUDO_R2_VARIANTS.
  • If tgt is a LinearModel or a GeneralisedLinearModel and tgt.kwargs holds a weights entry, the entry must be an ObsWeights, or an ArgumentError is thrown. When it is a vector, !isempty(tgt.kwargs.weights). The check does not read a target that the caller defines. The fit reads the weights of that target through its own factory and StatsAPI.fit methods, and the # Interfaces section of AbstractRegressionTarget states those methods.
  • The rules of assert_stepwise_included for included alone. An entry can be empty, and the asset then gets an intercept-only model.

Returns

  • nothing.

Related

source
PortfolioOptimisers.stepwise_factor_set — Function
stepwise_factor_set(re::StepwiseRegression, x::VecNum, F::MatNum, i::Integer)

Returns the factor set of one asset, which re.included fixes or the stepwise search selects.

Algorithm

  1. Take entry i of re.included. Return it when it is not nothing.
  2. Otherwise run the stepwise search of re on x, and call the result included. The method also takes this step when re.included is nothing.
  3. Warn, and name the asset, when included is empty. The asset then gets an intercept-only model, and its row of the loadings matrix is all zeros.
  4. Return included.

Arguments

  • re: Stepwise regression estimator.
  • x: Returns of asset i, observations × 1.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.
  • i: Index of the asset, which names it in the warning.

Returns

  • included::AbstractVector{<:Integer}: Indices of the factors of the asset, in the order of the search or of re.included.

Related

source
PortfolioOptimisers.stepwise_asset_fit! — Function
stepwise_asset_fit!(rr::MatNum, re::StepwiseRegression, tgt::AbstractRegressionTarget,
                    x::VecNum, F::MatNum, i::Integer)

Fits one asset of a StepwiseRegression, writes its coefficients into row i of rr, and returns its residual count.

Algorithm

  1. Take the factor set of asset i with stepwise_factor_set, and call it included.
  2. Fit tgt to a column of ones and the columns included of F, and read its coefficients. Call them params. Fit the intercept column alone when included is empty.
  3. Write params[1] into rr[i, 1], and the remaining coefficients into the columns of rr that included names, in the order of included. A selected factor whose coefficient the penalty of a penalised target set to zero writes that zero.
  4. Return residual_count of the fit and its design. This is the number of degrees of freedom that the fit leaves in the residuals of the asset.

Arguments

  • rr: The buffer assets × (factors + 1) of regression. The function writes into row i.
  • re: Stepwise regression estimator, which gives the factor set.
  • tgt: The target of re, with its first column named by factory(re.tgt, InterceptColumn(1)).
  • x: Returns of asset i, observations × 1.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.
  • i: Index of the asset.

Returns

  • nu::Real: The residual count of the fit of the asset.

Related

source
PortfolioOptimisers.pin_regression_choice — Function
pin_regression_choice(re::StepwiseRegression{<:Any, <:Any, <:Any, <:PinnedChoice},
                      X::MatNum, F::MatNum, w::Option{<:ObsWeights} = nothing)

Writes the factor set of the first fit of each asset into included of a stepwise regression under a PinnedChoice.

The online step of a prior calls it after the fold, over the rows of the buffer of the prior. The search of one asset reads only the column of that asset and F. So the factor set that it writes is the set that the fit over the same rows takes, whichever other assets the prior gives to that fit. A prior that refits the regression under observation weights, as OpinionPoolingPrior refits pe2, gives those weights as w, and the search reads them as that fit does. The fit does not cover an asset whose column holds a value that is not finite. That asset keeps nothing, and a later step writes its set at the first fit that covers it. Under BatchChoice the generic method returns re unchanged.

Algorithm

  1. Return re when F holds fewer than two rows. The regression refuses one row, so no fit happens yet.
  2. Check re.included with assert_stepwise_included. Copy it into a vector with one entry per column of X, and call it included. A field of nothing gives a vector of nothing.
  3. For each asset j whose entry is nothing and whose column of X is finite, run the stepwise search of re on that column, and write the result into entry j. When w is not nothing, the search is the search of factory(re, w).
  4. Return re rebuilt with included.

Arguments

  • re: Stepwise regression estimator under a PinnedChoice.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.
  • w: The observation weights under which the batch fit makes the choice that the method pins, or nothing when that fit is unweighted.

Validation

Returns

  • re::StepwiseRegression: The regression with the factor set of each covered asset in included.

Related

source