Stepwise Regression: private API
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:PValue, <:ForwardSelection}, x::VecNum,
F::MatNum)Grows a factor set from empty, and admits a model only when its p-values all clear the threshold.
A candidate is admissible only when each factor coefficient of the candidate model is significant. So the method rejects a factor that is significant alone when it makes a factor of the set insignificant.
Algorithm
- Set
includedempty andvalto zero, so the loop runs at least once. - While
valdoes not exceedre.crit.t, do steps 3 to 6. - Take the factors that
includeddoes not hold, and call themexcluded. - For each
iofexcluded, fitre.tgtto an intercept column and the columns[included; i], and read the p-value of columni. Call ittest_pval. Keepiasnew_factorwhentest_pvalis the smallest so far and no p-value of that candidate model exceedsre.crit.t. Keep the p-values of that model aspvals. - Stop the loop when step 4 kept no factor. Otherwise push
new_factorontoincluded. - Set
valto the largest entry ofpvals. - Call
add_best_factor_after_pval_failure!, which acts only whenincludedis still empty.
Arguments
re: Stepwise regression estimator with aPValuecriterion and theForwardSelectionalgorithm.x: Response vector,observations × 1.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Returns
included::Vector{Int}: Indices of the selected factors, in the order in which the loop added them, so the vector is not sorted. It holds at least one index.
Related
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion,
<:ForwardSelection}, x::VecNum, F::MatNum)Grows a factor set from empty, adding the factor whose model scores best.
The starting score is the worst value that the criterion can take, so the first addition always happens and the selection is never empty. get_forward_reg_incl_excl! reads the direction from regression_polarity, so one loop works for a minimised and for a maximised criterion.
Algorithm
- Set
includedempty andexcludedto every factor index, in ascending order. Name the first column of each design, the column of ones, to the target withfactory(re.tgt, InterceptColumn(1)), and call the resulttgt. - Read the criterion from
re.critandre.tgtwithregression_criterion_func, and call itcriterion_func. Read the starting score fromregression_threshold, and call itt. - Fill
valuewith that same worst score, one entry per factor. - Do steps 5 to 7 at most once per observation.
- For each
iofexcluded, fittgtto an intercept column and the columns[included; i], and writecriterion_funcof that fit intovalue[i]. - Call
get_forward_reg_incl_excl!, which moves the best factor and returns the newt. - Stop when step 6 moved no factor.
Arguments
re: Stepwise regression estimator with aMinMaxValStepwiseRegressionCriterioncriterion and theForwardSelectionalgorithm.x: Response vector,observations × 1.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Returns
included::Vector{Int}: Indices of the selected factors, in the order in which the loop added them, so the vector is not sorted. It holds at least one index.
Related
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:PValue, <:BackwardElimination}, x::VecNum,
F::MatNum)Shrinks a factor set from full, and removes the factor of largest p-value while a p-value exceeds the threshold.
Algorithm
- Fit
re.tgtto an intercept column and every column ofF, read the p-values of the factor coefficients, and setvalto the largest of them. - Set
includedto every factor index andexcludedempty. - While
valexceedsre.crit.t, do steps 4 to 6. - Set
includedto the factors thatexcludeddoes not hold. Stop the loop whenincludedis empty. - Fit
re.tgtto an intercept column and the columnsincluded, and read the p-values of the factor coefficients. Call thempvals. - Set
valto the largest entry ofpvals, and push the factor of that entry ontoexcluded. Only step 4 readsexcluded, so the push has an effect only when step 3 runs the loop again. The push of the last iteration does not change the result. - Call
add_best_factor_after_pval_failure!, which acts only whenincludedis empty.
Arguments
re: Stepwise regression estimator with aPValuecriterion and theBackwardEliminationalgorithm.x: Response vector,observations × 1.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Returns
included::VecInt: Indices of the selected factors, in ascending order. It holds at least one index. It is the1:size(F, 2)range itself when the full model already passes the threshold, and aVector{Int}otherwise.
Related
PortfolioOptimisers._regression — Method
_regression(re::StepwiseRegression{<:MinMaxValStepwiseRegressionCriterion,
<:BackwardElimination}, x::VecNum, F::MatNum)Shrinks a factor set from full, removing the factor whose reduced model scores best.
value[j] is the score of the model that omits j. So "best" has the same meaning as in the forward direction, and not the inverse meaning. Under a minimised criterion the method removes the factor of lowest value, and under a maximised criterion it removes the factor of highest value.
Algorithm
- Set
includedto every factor index, in ascending order. Name the first column of each design, the column of ones, to the target withfactory(re.tgt, InterceptColumn(1)), and call the resulttgt. - Read the criterion from
re.critandre.tgtwithregression_criterion_func, and call itcriterion_func. - Fit
tgtto an intercept column and every column ofF, and setttocriterion_funcof that full model. - Fill
valuewith the worst score the criterion can take, one entry per factor. - Do steps 6 to 8 at most once per observation.
- For each factor of
included, fittgtto an intercept column and the other columns ofincluded, and writecriterion_funcof that fit into that factor's entry ofvalue. Fit the intercept column alone whenincludedholds one factor. - Call
get_backward_reg_incl!, which removes the best factor and returns the newt. - Stop when step 7 removed no factor.
Arguments
re: Stepwise regression estimator with aMinMaxValStepwiseRegressionCriterioncriterion and theBackwardEliminationalgorithm.x: Response vector,observations × 1.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Returns
included::Vector{Int}: Indices of the selected factors, in ascending order. It can be empty. When each removal improves the score, the search removes every factor. This happens often under an information criterion on a response that the factors do not explain.stepwise_factor_setwarns and names the asset on an empty return, andregressionthen fits the intercept column alone.
Related
PortfolioOptimisers.add_best_factor_after_pval_failure! — Function
add_best_factor_after_pval_failure!(tgt::AbstractRegressionTarget,
included::VecInt, F::MatNum,
x::VecNum)Adds the factor of smallest p-value when a p-value search selected none.
Both PValue methods of _regression call it last, so a p-value search never returns an empty factor set. It does nothing when included already holds a factor. It warns each time that it adds a factor, because that factor failed the threshold in the search.
Algorithm
- Return at once when
includedis not empty. - For each factor
iofF, fittgtto an intercept column and columni, and read the p-value of columni. Call ittest_pval. - Keep the smallest
test_pvalof step 2 asbest_pval, and its factor asnew_factor. - Warn, and name
new_factorandbest_pval. - Push
new_factorontoincluded.
Arguments
tgt: Regression target that fits each candidate model.included: Indices of the factors that the search selected. The function writes into it.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.x: Response vector,observations × 1.
Returns
nothing.includedgains one index when it is empty, and it does not change otherwise.
Related
PortfolioOptimisers.get_forward_reg_incl_excl! — Function
get_forward_reg_incl_excl!(crit::MinMaxValStepwiseRegressionCriterion,
value::VecNum, excluded::VecInt,
included::VecInt, t::Number)Moves the best excluded factor into included when it improves the criterion.
regression_polarity states the direction. A minimised criterion uses findmin and <, and a maximised criterion uses findmax and >. The search reads the whole value vector, and it still finds the best excluded factor. The entry of an included factor holds the score of the step that added it. t holds the score of the last step, and each step improves on the step before it, so no entry of an included factor improves on t. Thus a score that improves on t belongs to an excluded factor, and searchsortedfirst finds its index in excluded.
Algorithm
- Read
bestandimprovesfromregression_polarity. - Find the best entry of
valuewithbest. Call itval, and its indexidx. - Return
tunchanged whenvaldoes not improve ont. - Otherwise find the position of
idxinexcluded, move that entry to the end ofincluded, and setttoval.
Arguments
crit: Stepwise regression criterion. Its polarity states which direction is better.value: Criterion score of each factor. Entryiis the score of the model that adds factoritoincluded.excluded: Indices of the factors outside the model, in ascending order. The function writes into it.included: Indices of the factors inside the model, in the order in which the search added them. The function writes into it.t: Threshold value.
Returns
t::Number: The score of the factor that moved, or the inputtwhen no factor moved.
Related
PortfolioOptimisers.get_backward_reg_incl! — Function
get_backward_reg_incl!(crit::MinMaxValStepwiseRegressionCriterion, value::VecNum,
included::VecInt, t::Number)Removes the best included factor from included when its removal improves the criterion.
regression_polarity states the direction. A minimised criterion uses findmin and <, and a maximised criterion uses findmax and >. The search reads the whole value vector, and it still finds the best included factor. The entry of a removed factor holds the score of the step that removed it. t holds the score of the last step, and each step improves on the step before it, so no entry of a removed factor improves on t. Thus a score that improves on t belongs to an included factor, and searchsortedfirst finds its index in included.
Algorithm
- Read
bestandimprovesfromregression_polarity. - Find the best entry of
valuewithbest. Call itval, and its indexidx. This entry is the best model that one removal can make. - Return
tunchanged whenvaldoes not improve ont. - Otherwise find the position of
idxinincluded, remove that entry, and setttoval.
Arguments
crit: Stepwise regression criterion. Its polarity states which direction is better.value: Criterion score of each factor. Entryjis the score of the model that omits factorj, so the best entry names the removal that improves the score most.included: Indices of the factors inside the model, in ascending order. The function writes into it.t: Threshold value.
Returns
t::Number: The score of the model that the removal leaves, or the inputtwhen the function removed no factor.
Related
PortfolioOptimisers.assert_stepwise_included — Function
assert_stepwise_included(included::Nothing)
assert_stepwise_included(included::AbstractVector)
assert_stepwise_included(included::Nothing, X::MatNum, F::MatNum)
assert_stepwise_included(included::AbstractVector, X::MatNum, F::MatNum)Checks the factor sets that the included field of a StepwiseRegression holds.
The one-argument form is the check of the constructor, which sees no data. The three-argument form is the check of the fit. It compares the field with the returns and the factor returns.
Algorithm
The method that Julia selects is the algorithm. included of nothing passes both forms.
Arguments
included: The factor sets, one entry per asset, ornothing.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Validation
- One argument:
!isempty(included), or the check raises anIsEmptyError. Each entry isnothingor a vector of positive factor indices with no repeated index, or the check raises aDomainError. - Three arguments:
length(included) == size(X, 2), and every index of each entry is at mostsize(F, 2). The check raises aDimensionMismatchotherwise.
Returns
nothing.
Related
PortfolioOptimisers.assert_stepwise_regression — Function
assert_stepwise_regression(crit::MinValStepwiseRegressionCriterion,
tgt::AbstractRegressionTarget,
included::Option{<:AbstractVector})
assert_stepwise_regression(crit::Union{MaxValStepwiseRegressionCriterion,
AbstractStepwiseRegressionCriterion},
tgt::AbstractUnpenalisedRegressionTarget,
included::Option{<:AbstractVector})Checks the criterion, the target and the factor sets of a StepwiseRegression.
The inner constructor calls it. The two methods state the valid pairs of criterion and target in their signatures, so a refused pair, a penalised target under a criterion other than a MinValStepwiseRegressionCriterion, finds no method and throws a MethodError. The pair rule sits here and not in two methods of the inner constructor, because @concrete adds a generic positional constructor, and a call that matches no inner method reaches that constructor with no check.
Algorithm
Each method holds the checks that its pairs reach, so the copies of the rule on the weights agree.
- Under a
MaxValStepwiseRegressionCriterion, check the variant of aGeneralisedLinearModelunder:adjr2. The method of aMinValStepwiseRegressionCriterionhas no such step, because:adjr2is a maximised criterion. - Check the
weightsentry of thekwargsof aLinearModelor aGeneralisedLinearModel. Both methods hold this step. - Check
includedwithassert_stepwise_included.
Arguments
crit: The criterion, as thecritfield holds it.tgt: The regression target.included: The factor sets, one entry per asset, ornothing.
Validation
- If
critisVal(:adjr2),tgtis aGeneralisedLinearModelandtgt.variantis set,tgt.variant in ADJUSTED_PSEUDO_R2_VARIANTS. - If
tgtis aLinearModelor aGeneralisedLinearModelandtgt.kwargsholds aweightsentry, the entry must be anObsWeights, or anArgumentErroris thrown. When it is a vector,!isempty(tgt.kwargs.weights). The check does not read a target that the caller defines. The fit reads the weights of that target through its ownfactoryandStatsAPI.fitmethods, and the# Interfacessection ofAbstractRegressionTargetstates those methods. - The rules of
assert_stepwise_includedforincludedalone. An entry can be empty, and the asset then gets an intercept-only model.
Returns
nothing.
Related
PortfolioOptimisers.stepwise_factor_set — Function
stepwise_factor_set(re::StepwiseRegression, x::VecNum, F::MatNum, i::Integer)Returns the factor set of one asset, which re.included fixes or the stepwise search selects.
Algorithm
- Take entry
iofre.included. Return it when it is notnothing. - Otherwise run the stepwise search of
reonx, and call the resultincluded. The method also takes this step whenre.includedisnothing. - Warn, and name the asset, when
includedis empty. The asset then gets an intercept-only model, and its row of the loadings matrix is all zeros. - Return
included.
Arguments
re: Stepwise regression estimator.x: Returns of asseti,observations × 1.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.i: Index of the asset, which names it in the warning.
Returns
included::AbstractVector{<:Integer}: Indices of the factors of the asset, in the order of the search or ofre.included.
Related
PortfolioOptimisers.stepwise_asset_fit! — Function
stepwise_asset_fit!(rr::MatNum, re::StepwiseRegression, tgt::AbstractRegressionTarget,
x::VecNum, F::MatNum, i::Integer)Fits one asset of a StepwiseRegression, writes its coefficients into row i of rr, and returns its residual count.
Algorithm
- Take the factor set of asset
iwithstepwise_factor_set, and call itincluded. - Fit
tgtto a column of ones and the columnsincludedofF, and read its coefficients. Call themparams. Fit the intercept column alone whenincludedis empty. - Write
params[1]intorr[i, 1], and the remaining coefficients into the columns ofrrthatincludednames, in the order ofincluded. A selected factor whose coefficient the penalty of a penalised target set to zero writes that zero. - Return
residual_countof the fit and its design. This is the number of degrees of freedom that the fit leaves in the residuals of the asset.
Arguments
rr: The bufferassets × (factors + 1)ofregression. The function writes into rowi.re: Stepwise regression estimator, which gives the factor set.tgt: The target ofre, with its first column named byfactory(re.tgt, InterceptColumn(1)).x: Returns of asseti,observations × 1.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.i: Index of the asset.
Returns
nu::Real: The residual count of the fit of the asset.
Related
PortfolioOptimisers.pin_regression_choice — Function
pin_regression_choice(re::StepwiseRegression{<:Any, <:Any, <:Any, <:PinnedChoice},
X::MatNum, F::MatNum, w::Option{<:ObsWeights} = nothing)Writes the factor set of the first fit of each asset into included of a stepwise regression under a PinnedChoice.
The online step of a prior calls it after the fold, over the rows of the buffer of the prior. The search of one asset reads only the column of that asset and F. So the factor set that it writes is the set that the fit over the same rows takes, whichever other assets the prior gives to that fit. A prior that refits the regression under observation weights, as OpinionPoolingPrior refits pe2, gives those weights as w, and the search reads them as that fit does. The fit does not cover an asset whose column holds a value that is not finite. That asset keeps nothing, and a later step writes its set at the first fit that covers it. Under BatchChoice the generic method returns re unchanged.
Algorithm
- Return
rewhenFholds fewer than two rows. The regression refuses one row, so no fit happens yet. - Check
re.includedwithassert_stepwise_included. Copy it into a vector with one entry per column ofX, and call itincluded. A field ofnothinggives a vector ofnothing. - For each asset
jwhose entry isnothingand whose column ofXis finite, run the stepwise search ofreon that column, and write the result into entryj. Whenwis notnothing, the search is the search offactory(re, w). - Return
rerebuilt withincluded.
Arguments
re: Stepwise regression estimator under aPinnedChoice.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.w: The observation weights under which the batch fit makes the choice that the method pins, ornothingwhen that fit is unweighted.
Validation
- The rules of
assert_stepwise_includedforre.included,XandF.
Returns
re::StepwiseRegression: The regression with the factor set of each covered asset inincluded.
Related