Base regression: private API
PortfolioOptimisers.PSEUDO_R2_VARIANTS — Constant
PSEUDO_R2_VARIANTSTuple of the pseudo-$R^2$ variants StatsAPI.r2 accepts for a fitted GeneralisedLinearModel.
The members are :McFadden, :CoxSnell, :Nagelkerke and :devianceratio. The constructor of GeneralisedLinearModel checks its variant field against this tuple. A generalised linear model has no classical $R^2$, so each member scores the fitted model against the intercept-only model of the same family instead.
Mathematical definition
\[\begin{align} R^2_{\mathrm{McF}} &= 1 - \frac{\ln\hat{L}}{\ln\hat{L}_{0}}\,,\\ R^2_{\mathrm{CS}} &= 1 - \left(\frac{\hat{L}_{0}}{\hat{L}}\right)^{2/T}\,,\\ R^2_{\mathrm{N}} &= \frac{R^2_{\mathrm{CS}}}{1 - \hat{L}_{0}^{\,2/T}}\,,\\ R^2_{\mathrm{dev}} &= 1 - \frac{D}{D_{0}}\,. \end{align}\]
Where:
- $R^2_{\mathrm{McF}}$:
:McFadden. - $R^2_{\mathrm{CS}}$:
:CoxSnell. - $R^2_{\mathrm{N}}$:
:Nagelkerke. - $R^2_{\mathrm{dev}}$:
:devianceratio. - $\hat{L}$: Maximum likelihood of the fitted model.
- $\hat{L}_{0}$: Maximum likelihood of the intercept-only model of the same family.
- $D$: Deviance of the fitted model.
- $D_{0}$: Deviance of the intercept-only model of the same family.
- $T$: Number of observations.
The Normal family is the default of GeneralisedLinearModel, and three facts hold for it. Its deviance is the residual sum of squares, so $R^2_{\mathrm{dev}}$ is the classical $R^2$ of the same fit. Its maximum likelihood carries the fitted dispersion, so $\left(\hat{L}_{0}/\hat{L}\right)^{2/T} = D/D_{0}$ and $R^2_{\mathrm{CS}}$ equals $R^2_{\mathrm{dev}}$.
Its likelihood is a density and not a probability, so $\hat{L}_{0}$ can be above or below one. The two forms on the log-likelihood, $R^2_{\mathrm{McF}}$ and $R^2_{\mathrm{N}}$, can then leave $[0, 1]$ in either direction. A change of the scale of the response alone moves both from above one to below zero, and $R^2_{\mathrm{dev}}$ does not change. Only $R^2_{\mathrm{dev}}$ is continuous with the LinearModel path, so default_regression_criterion_variant returns :devianceratio.
Related
ADJUSTED_PSEUDO_R2_VARIANTSGeneralisedLinearModeldefault_regression_criterion_variantregression_criterion_func
References
- [291] D. McFadden. Conditional logit analysis of qualitative choice behavior. In: Frontiers in Econometrics, edited by P. Zarembka (Academic Press, 1974); pp. 105–142. Equation 30, page 121.
- [292] D. R. Cox and E. J. Snell. Analysis of Binary Data. 2 Edition (Chapman and Hall, 1989). Pages 208-209. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [293] N. J. Nagelkerke. A note on a general definition of the coefficient of determination. Biometrika 78, 691–692 (1991). Pages 691-692. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [67] J. A. Nelder and R. W. Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 370–384 (1972). Pages 370-384. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.ADJUSTED_PSEUDO_R2_VARIANTS — Constant
ADJUSTED_PSEUDO_R2_VARIANTSTuple of the pseudo-$R^2$ variants StatsAPI.adjr2 accepts for a fitted GeneralisedLinearModel.
The members are :McFadden and :devianceratio, a strict subset of PSEUDO_R2_VARIANTS. GeneralisedLinearModel cannot check its variant against this tuple, because it does not know which criterion reads the variant. StepwiseRegression checks it, because it is the first type that holds the criterion and the target together. StatsAPI.adjr2 raises an ArgumentError on each variant that this tuple omits.
Mathematical definition
Each member discounts its unadjusted form of PSEUDO_R2_VARIANTS by the number of parameters of the model. [291] states the unadjusted form alone, and the discount of each member is the convention of the GLM package. This docstring cites no original source for the discount.
\[\begin{align} \bar{R}^2_{\mathrm{McF}} &= 1 - \frac{\ln\hat{L} - k}{\ln\hat{L}_{0}}\,,\\ \bar{R}^2_{\mathrm{dev}} &= 1 - \frac{D\,(T - 1)}{D_{0}\,(T - k)}\,. \end{align}\]
Where:
- $\bar{R}^2_{\mathrm{McF}}$:
:McFadden. - $\bar{R}^2_{\mathrm{dev}}$:
:devianceratio. - $\hat{L}$: Maximum likelihood of the fitted model.
- $\hat{L}_{0}$: Maximum likelihood of the intercept-only model of the same family.
- $D$: Deviance of the fitted model.
- $D_{0}$: Deviance of the intercept-only model of the same family.
- $k$: Number of estimated parameters, which is
StatsAPI.dofof the fitted model. It counts the regression coefficients, the intercept, and the dispersion when the family estimates one. The Normal family estimates it, and the binomial and Poisson families do not. - $T$: Number of observations.
This $k$ is the count that :aic, :aicc and :bic read. It is not the predictor count that :adjr2 reads on a fitted LinearModel. STEPWISE_REGRESSION_CRITERIA states both counts.
Related
References
- [291] D. McFadden. Conditional logit analysis of qualitative choice behavior. In: Frontiers in Econometrics, edited by P. Zarembka (Academic Press, 1974); pp. 105–142. Equation 30, page 121.
- [67] J. A. Nelder and R. W. Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 370–384 (1972). Pages 370-384. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.MIN_VAL_STEPWISE_REGRESSION_CRITERIA — Constant
MIN_VAL_STEPWISE_REGRESSION_CRITERIATuple of the symbols that name a stepwise regression criterion for which a lower value is better.
The members are :aic, :aicc and :bic. The library builds MinValStepwiseRegressionCriterion from this tuple, and STEPWISE_REGRESSION_CRITERIA states what each symbol computes.
Related
PortfolioOptimisers.MAX_VAL_STEPWISE_REGRESSION_CRITERIA — Constant
MAX_VAL_STEPWISE_REGRESSION_CRITERIATuple of the symbols that name a stepwise regression criterion for which a higher value is better.
The members are :r2 and :adjr2. The library builds MaxValStepwiseRegressionCriterion from this tuple, and STEPWISE_REGRESSION_CRITERIA states what each symbol computes.
Related
PortfolioOptimisers.STEPWISE_REGRESSION_CRITERIA — Constant
STEPWISE_REGRESSION_CRITERIATuple of the symbols that name the stepwise regression criteria with one score for a fitted model.
StepwiseRegression accepts each symbol of this tuple in its crit field, and stores it as a Val. regression_criterion_func, regression_polarity, regression_threshold and the get_*_reg_incl*! helpers dispatch on that Val. The constructor rejects a symbol outside the tuple.
PValue is not a member. It reads the p-values of the coefficients of the fitted model, and not one score, so it is a type with its own stepwise methods. :aic, :aicc and :bic score a fitted LinearModel and a fitted GeneralisedLinearModel the same way. GLM defines :r2 and :adjr2 for a fitted LinearModel only, so under the other target they read a named pseudo-$R^2$ variant.
Mathematical definition
:aic, the Akaike information criterion
The criterion trades the fitted likelihood against the number of estimated parameters. A lower value is a better model.
\[\begin{align} \mathrm{AIC} &= 2k - 2\ln\hat{L}\,. \end{align}\]
Where:
- $k$: Number of estimated parameters, which is
StatsAPI.dofof the fitted model. It counts the regression coefficients, the intercept and the residual variance. - $\hat{L}$: Maximum likelihood of the model.
:aicc, the Akaike information criterion with the small-sample correction
The correction term grows as $T$ comes near $k$. On a short sample, :aicc then gives a large model a larger penalty than :aic does.
\[\begin{align} \mathrm{AICC} &= \mathrm{AIC} + \frac{2k(k+1)}{T - k - 1}\,. \end{align}\]
Where:
- $k$: Number of estimated parameters, as for
:aic. - $T$: Number of observations.
The correction term divides by $T - k - 1$. The criterion has no value when $T = k + 1$, and the term changes sign below it. So :aicc needs a sample longer than the largest model that the search can reach.
:bic, the Bayesian information criterion
The penalty is $k \ln T$ and not $2k$. It is larger than the penalty of :aic for $T \geq 8$, so the search usually stops with fewer factors. [294] maximises the log-likelihood less $\tfrac{1}{2} k \ln T$. The form below multiplies that score by minus two, which keeps the order of the models and makes a lower value better.
\[\begin{align} \mathrm{BIC} &= k\ln T - 2\ln\hat{L}\,. \end{align}\]
Where:
- $k$: Number of estimated parameters, as for
:aic. - $T$: Number of observations.
- $\hat{L}$: Maximum likelihood of the model.
:r2, the coefficient of determination
The criterion is the share of the variance of the response that the model explains. A higher value is a better fit.
\[\begin{align} R^2 &= 1 - \frac{\mathrm{SS}_{\mathrm{res}}}{\mathrm{SS}_{\mathrm{tot}}} = 1 - \frac{\sum_t (y_t - \hat{y}_t)^2}{\sum_t (y_t - \bar{y})^2}\,. \end{align}\]
Where:
- $\mathrm{SS}_{\mathrm{res}}$: Residual sum of squares.
- $\mathrm{SS}_{\mathrm{tot}}$: Total sum of squares.
- $y_t$: Observed response at time $t$.
- $\hat{y}_t$: Fitted response at time $t$.
- $\bar{y}$: Mean of the observed responses.
$R^2$ never falls when the model gets one more factor, so it gives no penalty for the size of the model. Under ForwardSelection it admits every factor, and under BackwardElimination it removes no factor. Use :adjr2, :aic, :aicc or :bic when the criterion must penalise the size of the model.
:adjr2, the adjusted coefficient of determination
The adjustment discounts $R^2$ by the degrees of freedom of the predictors. Unlike the score of :r2, the score of :adjr2 can fall when the model gets one more factor, and the search can stop on it.
\[\begin{align} \bar{R}^2 &= 1 - (1 - R^2) \frac{T - 1}{T - k - 1}\,. \end{align}\]
Where:
- $k$: Number of predictors, without the intercept.
:aic,:aiccand:bicuse $k$ for a different count, which adds the intercept and the residual variance to the predictors. - $T$: Number of observations.
Related
MinValStepwiseRegressionCriterionMaxValStepwiseRegressionCriterionStepwiseRegressionPSEUDO_R2_VARIANTSregression_criterion_funcdefault_regression_criterion_variantregression_thresholdPValue
References
- [295] H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19, 716–723 (1974). Section V, page 719.
- [296] C. M. Hurvich and C.-L. Tsai. Regression and time series model selection in small samples. Biometrika 76, 297–307 (1989). Pages 297-307. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [294] G. Schwarz. Estimating the dimension of a model. The Annals of Statistics 6, 461–464 (1978). Section 1, page 461, and the Proposition of Section 3, page 462.
- [297] H. Theil. Economic Forecasts and Policy. 2 Edition (North-Holland, 1961). Page 213. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.AbstractRegressionEstimator — Type
abstract type AbstractRegressionEstimator <: AbstractEstimatorAbstract supertype of every regression estimator, over both the time-series family and the cross-sectional family.
The type is an umbrella over the two families. It declares no interface of its own, because the two families fit different models and implement different verbs. A time-series estimator fits one model per asset over the observations and implements regression. A cross-sectional estimator fits one model per observation across the assets and implements cross_sectional_regression. Subtype the child that names the family, never this root. Then a consumer of one family never receives a value of the other.
Related
PortfolioOptimisers.AbstractRegressionResult — Type
abstract type AbstractRegressionResult <: AbstractResultAbstract supertype of every regression result, over both the loadings family and the cross-sectional family.
The type is an umbrella over two children, and an asset index means a different thing in each child. A loadings result holds one row of M per asset, so port_opt_view slices its rows. A cross-sectional result holds one row per observation and one column per asset, so the same index slices its columns. Subtype the child that states what the result carries, never this root.
Related
PortfolioOptimisers.AbstractLoadingsRegressionResult — Type
abstract type AbstractLoadingsRegressionResult <: AbstractRegressionResultAbstract supertype for all regression result types that carry a loadings matrix.
A subtype carries a loadings matrix M, with one row per asset and one column per factor. The root states what a member carries, and not how the library fitted it. Every consumer that re-bases a constraint or decomposes risk in the factor basis reads M, so it binds this type and not the umbrella. The shape of the fit does not decide membership. A result fitted per asset over the observations and a result fitted per observation across the assets both belong here when they carry M.
Related
PortfolioOptimisers.AbstractCrossSectionalRegressionResult — Type
abstract type AbstractCrossSectionalRegressionResult <: AbstractRegressionResultAbstract supertype for all cross-sectional regression result types.
A subtype holds the output of a regression fitted per observation across the assets. A member carries no loadings matrix, because the exposures are the input of the regression, and an Exposure Estimator makes them.
Related
PortfolioOptimisers.AbstractFactorFamilyBasis — Type
abstract type AbstractFactorFamilyBasis <: AbstractResultAbstract supertype for all Factor Family Basis result types.
A subtype holds the change of basis of a re-based Factor Family. The root lets CrossSectionalFactorModel bind its fcb field to a type and not to Any. This file declares the root, and not the file of its member, because the library loads the block that has the field before the member that fills it.
Related
PortfolioOptimisers.AbstractRegressionAlgorithm — Type
abstract type AbstractRegressionAlgorithm <: AbstractAlgorithmAbstract supertype for all regression algorithm types.
A subtype names one choice of a regression estimator, such as the stepwise search, the criterion of the search or the target that fits each model. The estimator holds the subtype in a field.
Related
PortfolioOptimisers.AbstractStepwiseRegressionAlgorithm — Type
abstract type AbstractStepwiseRegressionAlgorithm <: AbstractRegressionAlgorithmAbstract supertype for all stepwise regression algorithm types.
A subtype decides whether the search adds factors to the factor set or removes them. An AbstractStepwiseRegressionCriterion decides which move is an improvement.
Related
AbstractRegressionAlgorithmAbstractStepwiseRegressionCriterionAbstractRegressionTargetForwardSelectionBackwardElimination
References
- [68] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203. Pages 191-203. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.AbstractStepwiseRegressionCriterion — Type
abstract type AbstractStepwiseRegressionCriterion <: AbstractRegressionAlgorithmAbstract supertype for all stepwise regression criterion types.
A subtype scores a fitted model, and the stepwise search keeps the move that improves the score.
Related
PortfolioOptimisers.MinValStepwiseRegressionCriterion — Type
MinValStepwiseRegressionCriterionUnion of the Val types that name a stepwise regression criterion for which a lower value is better.
The library builds the members from MIN_VAL_STEPWISE_REGRESSION_CRITERIA, so the union is Union{Val{:aic}, Val{:aicc}, Val{:bic}}. StepwiseRegression stores Val(crit) in its first field, crit. So a member of this union is the first type parameter of the estimator when the criterion is a minimisation criterion.
Related
PortfolioOptimisers.MaxValStepwiseRegressionCriterion — Type
MaxValStepwiseRegressionCriterionUnion of the Val types that name a stepwise regression criterion for which a higher value is better.
The library builds the members from MAX_VAL_STEPWISE_REGRESSION_CRITERIA, so the union is Union{Val{:r2}, Val{:adjr2}}. Both members read a pseudo-$R^2$ variant under a GeneralisedLinearModel target.
Related
PortfolioOptimisers.MinMaxValStepwiseRegressionCriterion — Type
MinMaxValStepwiseRegressionCriterionUnion of every Val type that names a stepwise regression criterion with one score for a fitted model.
This is the union of MinValStepwiseRegressionCriterion and MaxValStepwiseRegressionCriterion. The crit field of StepwiseRegression holds a member of this union when the criterion is one score, and not a PValue.
Related
PortfolioOptimisers.RegE_Reg — Type
const RegE_Reg = Union{AbstractLoadingsRegressionResult,
AbstractTimeSeriesRegressionEstimator}Alias for a loadings regression result or a time-series regression estimator.
The alias matches an AbstractLoadingsRegressionResult, which is a precomputed result that carries the loadings matrix M. It also matches an AbstractTimeSeriesRegressionEstimator, whose verb returns such a result. The factor models and the regression-based risk code dispatch on it. Every consumer of the alias reads M, so the alias names the two ways to get M, and not the umbrella.
The two arms of the union use different rules, on purpose. A result carries data, so the result arm names what it carries. It admits each loadings result, whatever the shape of its fit. An estimator carries no data, so the estimator arm names the family whose verb, regression, returns a loadings result. A cross-sectional estimator implements cross_sectional_regression, whose result carries no loadings, so the alias excludes it.
Related
PortfolioOptimisers.set_idiosyncratic_covariance — Method
set_idiosyncratic_covariance(re::Regression, esigma::Option{<:VecNum_MatNum},
edof::Option{<:VecNum}, ediv::Option{<:VecNum})Return a Regression that carries esigma, edof and ediv, with every other field unchanged, the row key included.
A regression estimator fits loadings alone, so the block that a fit returns carries no idiosyncratic covariance. The prior that lifts the factor moments measures the residual variances in the lift. It writes them here with their counts, so that no consumer computes them again. FactorPrior and FactorBlackLittermanPrior are the two callers. Each passes what factor_lift returns, which is the variances and their counts under rsd = true, and nothing for the variances and the divisors under rsd = false.
This method reads L and b with getfield, as port_opt_view does. The swap(L, M) rule of Regression makes re.L return re.M when L is unset. So a read through the property writes M under L in the new result, and isnothing(getfield(re, :L)) is then false.
Arguments
re: The regression result to copy.esigma: The idiosyncratic covariance to write, ornothingto leave the field unset.edof: The degrees of freedom of each residual series, ornothingto leave the field unset.ediv: The divisor of each idiosyncratic variance, ornothingto leave the field unset.
Returns
re::Regression: A new result that carriesesigma,edofandediv. The constructor runs all its guards again.
Related
PortfolioOptimisers.set_row_key — Method
set_row_key(re::Regression, idx::Option{<:VecInt}, ts::Option{<:VecDate})Return a Regression that carries the row key idx and ts, with every other field unchanged.
A regression estimator fits loadings alone, and does not record the observations that it read. FactorPrior writes the key after the fit, because it knows which observations its prior result describes. map_row_positions maps the key through a selection of observations, and forget_row_positions drops the positions of the key. Both write the new key with this method. The method reads L and b with getfield, for the reason that set_idiosyncratic_covariance states.
Arguments
re: The regression result to copy.idx: The position of each observation in the returns data that the prior read, ornothing.ts: The timestamp of each observation, ornothing.
Returns
re::Regression: A new result that carriesidxandts. The constructor runs all its guards again.
Related
PortfolioOptimisers.has_family_rebasis — Method
has_family_rebasis(
_::AbstractLoadingsRegressionResult
) -> Bool
Return whether a loadings result comes from a fit in a re-based Factor Family.
The fit re-bases a Factor Family whose one-hot exposures are collinear with a global factor. It drops one factor of the family, and writes the family in an equivalent basis of full column rank. A result that carries such a re-basis keeps the raw loadings in M and the re-based loadings in L. Its raw factor axis is then a linear image of a smaller axis, so a factor covariance on that axis is singular. A consumer that inverts or factorises such a covariance reads this trait, and refuses the result before the factorisation fails.
The method of the root returns false, so a member that cannot carry a re-basis needs no method. The value false is correct for a Regression, because a fit per asset over the observations re-bases nothing. Only CrossSectionalFactorModel has a method of its own.
Arguments
rr: The loadings regression result.
Returns
val::Bool:truewhen the result carries a family re-basis, andfalseotherwise. Under a re-basis, the raw factor axis is rank deficient.
Examples
julia> PortfolioOptimisers.has_family_rebasis(Regression(; M = [1 2; 3 4], b = [1, 2]))falseRelated
PortfolioOptimisers.default_regression_criterion_variant — Function
default_regression_criterion_variant(crit::MaxValStepwiseRegressionCriterion)Return the pseudo-$R^2$ variant a maximisation criterion reads when the target names none.
StatsAPI.r2 and StatsAPI.adjr2 both accept :McFadden and :devianceratio. Of the two, only :devianceratio gives the classical $R^2$ of a fitted LinearModel on a Normal family. With this default, the score stays continuous with the LinearModel path under each criterion.
Arguments
crit: The maximisation criterion, as theValthat thecritfield ofStepwiseRegressionholds.
Returns
variant::Symbol::devianceratio.
Related
MaxValStepwiseRegressionCriterionGeneralisedLinearModel: A target whosevariantis notnothingoverrides this default.regression_criterion_func: The only caller. It reads the default only when the target names no variant.PSEUDO_R2_VARIANTSADJUSTED_PSEUDO_R2_VARIANTS
PortfolioOptimisers.regression_criterion_func — Function
regression_criterion_func(crit::MinMaxValStepwiseRegressionCriterion,
tgt::AbstractRegressionTarget)Return the function that scores a fitted model under a stepwise regression criterion.
The method dispatches on the Val that names the criterion and on the regression target, because the two maximisation criteria read a different quantity under each target. The map is:
| Criterion | GeneralisedLinearModel | Every other target |
|---|---|---|
:aic | StatsAPI.aic | StatsAPI.aic |
:aicc | StatsAPI.aicc | StatsAPI.aicc |
:bic | StatsAPI.bic | StatsAPI.bic |
:r2 | model -> StatsAPI.r2(model, variant) | StatsAPI.r2 |
:adjr2 | model -> StatsAPI.adjr2(model, variant) | StatsAPI.adjr2 |
StatsAPI.aic, StatsAPI.aicc and StatsAPI.bic accept a fitted model of each library target, so each minimisation criterion has one method. StatsAPI.r2 and StatsAPI.adjr2 accept a fitted LinearModel without a variant. A fitted GeneralisedLinearModel needs a named pseudo-$R^2$ variant, so each maximisation criterion has two methods. The method for every other target serves LinearModel and a target of the caller alike. It returns the StatsAPI verb, and the fit of a target of the caller implements that verb, as the # Interfaces section of AbstractRegressionTarget states. PValue has no method here, because it reads the p-values of the coefficients and not one score.
Algorithm
The five methods that return a StatsAPI function run no steps. The two that close over a variant run these steps:
- Read
tgt.variant. When it isnothing, takedefault_regression_criterion_variantofcrit. Call the resultvariant. - Build a closure over
variantthat callsStatsAPI.r2(model, variant), orStatsAPI.adjr2(model, variant)under:adjr2.
Arguments
crit: The criterion, as theValthat thecritfield ofStepwiseRegressionholds.tgt: The regression target that fits the candidate models.
Returns
f::Function: The function that computes the criterion value for a fitted model.
Related
PortfolioOptimisers.RegressionPolarity — Type
struct RegressionPolarity{__T_best, __T_improves, __T_worst}Holds the three functions that state which direction of a stepwise criterion is better.
This type is an implementation detail and is not intended for direct use. regression_polarity makes it, and regression_threshold, get_forward_reg_incl_excl!, get_backward_reg_incl! and the two _regression methods that fill a score vector read it. The fields best and worst are functions of the same kind, so a positional swap of them is silent, and the type names them.
Fields
best:best:findminunder a minimised criterion,findmaxunder a maximised one. Returns the best entry of a score vector and its index.
improves:improves:<under a minimised criterion,>under a maximised one. Returns whether the first score is better than the second.
worst:worst:typemaxunder a minimised criterion,typeminunder a maximised one. Returns the worst score of the type it is given.
Related
PortfolioOptimisers.regression_polarity — Function
regression_polarity(crit::MinMaxValStepwiseRegressionCriterion) -> RegressionPolarityReturn the three functions that state which direction of a stepwise criterion is better.
A stepwise search asks the same three questions of every criterion: which entry of a score vector is the best one, whether a candidate score improves on the score in hand, and what the worst score of a type is. Each answer is one function under a minimised criterion and its opposite under a maximised one. This method pair is the only place in the library that states the pairing. regression_threshold, get_forward_reg_incl_excl!, get_backward_reg_incl! and the two _regression methods that fill a score vector all read it, and none of them states the pairing again.
Arguments
crit: The criterion, as theValthat thecritfield ofStepwiseRegressionholds.
Returns
polarity::RegressionPolarity: The three functionsbest,improvesandworst.
Related
PortfolioOptimisers.regression_threshold — Function
regression_threshold(crit::MinMaxValStepwiseRegressionCriterion)Return the threshold that a forward stepwise regression search starts from.
The value is the worst score the criterion can take, so the first candidate model always improves on it. regression_polarity states which of typemax and typemin that is, and typemax(Float64) is Inf. Only ForwardSelection reads it. BackwardElimination starts from the score of the full model, because its first move must improve on a model that already exists.
Arguments
crit: The criterion, as theValthat thecritfield ofStepwiseRegressionholds.
Returns
t::Number:Inffor a minimisation criterion, and-Inffor a maximisation criterion.
Related
regression_polarity: The pairing that this method reads.MinValStepwiseRegressionCriterionMaxValStepwiseRegressionCriterionSTEPWISE_REGRESSION_CRITERIAForwardSelection: The only caller.BackwardElimination: It starts from the score of the full model, and never from this value.
PortfolioOptimisers.pin_regression_choice — Function
pin_regression_choice(re::AbstractTimeSeriesRegressionEstimator, X::MatNum, F::MatNum,
w::Option{<:ObsWeights} = nothing)Return a regression estimator that has no choice to pin, unchanged.
The online step of a prior that fits a regression calls pin_regression_choice after the fold, over the rows of its buffer. A regression that makes a choice over every observation, such as the factor set of StepwiseRegression or the components of DimensionReductionRegression, has a method of its own under a PinnedChoice. That method writes the choice of the first fit into the configuration of the estimator. This method serves every other regression, and a regression under BatchChoice.
Arguments
re: The regression estimator.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.w: The observation weights under which the batch fit makes the choice that the method pins, ornothingwhen that fit is unweighted.
Returns
re::AbstractTimeSeriesRegressionEstimator: The input estimator, unchanged.
Related
PortfolioOptimisers.glm_weights — Function
glm_weights(w::StatsBase.AbstractWeights) -> StatsBase.FrequencyWeights
glm_weights(w::Nothing) -> NothingReturn the frequency weights that GLM reads as the observation weights w, or nothing for a fit with no weights.
GLM reads every weight vector as frequency weights, so the number of observations of a weighted fit is $\sum_t w_t$. StatsBase.FrequencyWeights state a count, so the method returns them unchanged. Every other kind of weights states the relative weight of each observation and no count. For example, GLM reads probability weights that sum to one as one observation. The fit then has a residual degree of freedom at or below zero, and a p-value of one for every coefficient. To prevent this, the method scales the other kinds to sum to their Kish effective sample size.
The scale does not change the coefficients, because a weighted fit does not change under a constant scale of its weights. The Kish count is the correct count for observations that are drawn alike and then reweighted. The scenario probabilities and the decay weights of the library are such weights. The count is $T$ under uniform weights, so the fit equals the unweighted fit. It is $k$ under equal weights on $k$ of the rows, so the fit equals the fit of those rows alone.
A caller who wants another count states it with StatsBase.fweights.
Mathematical definition
\[\begin{align} T_{e} &= \frac{\left(\sum_{t} w_{t}\right)^2}{\sum_{t} w_{t}^2}\,,\\ \tilde{w}_{t} &= w_{t} \frac{T_{e}}{\sum_{s} w_{s}} = w_{t} \frac{\sum_{s} w_{s}}{\sum_{s} w_{s}^2}\,. \end{align}\]
Where:
- $w_{t}$: Observation weight of observation $t$.
- $T_{e}$: Effective sample size, which is Kish's when the observation weights are stated.
- $\tilde{w}_{t}$: Frequency weight of observation $t$. The frequency weights sum to $T_{e}$.
Arguments
w: The resolved observation weights, one per row of the design, ornothing.
Returns
w::Option{StatsBase.FrequencyWeights}:witself when it isStatsBase.FrequencyWeightsornothing, otherwise $\tilde{w}$.
Related
PortfolioOptimisers.glm_fit_kwargs — Function
glm_fit_kwargs(kwargs::NamedTuple, X::MatNum) -> NamedTupleReturn the keyword arguments that a library regression target passes to GLM.
LinearModel and GeneralisedLinearModel keep their observation weights under the weights key of their kwargs, where factory puts them. Both targets resolve and scale the entry here, before the fit. A DynamicAbstractWeights holds no values before the method resolves it against the design, and GLM reads every weight vector as a count.
Algorithm
- Read the
weightsentry ofkwargs. When it is not anObsWeights, returnkwargsunchanged. - Resolve the entry against
Xwithget_observation_weights, and scale it withglm_weights. - Return
kwargswith the scaled weights under theweightskey.
Arguments
kwargs: Thekwargsfield of the regression target.X: The design matrixobservations × factors.
Returns
kwargs::NamedTuple: The keyword arguments of the call toGLM.
Related
PortfolioOptimisers.assert_finite_factor_rows — Function
assert_finite_factor_rows(F::MatNum) -> NothingRefuse a factor returns matrix with a non-finite entry, for a regression on matrices.
A factor return is a regressor of every asset, so a gap at one observation leaves that observation with no regressor for any asset. A fit through a ReturnsResult drops such an observation with shared_series_view before it reaches a matrix. A caller of a matrix method gives the observations that it chose, and the method refuses a gap in them. It does not return NaN loadings.
Arguments
F: Data matrixobservations × factorsif thedimskeyword does not exist ordims = 1,factors × observationswhendims = 2.
Validation
- Every entry of
Fis finite. Otherwise, the method throws anIsNonFiniteErrorthat names the first observations with a gap.
Returns
nothing.
Related
References
- [67]
- J. A. Nelder and R. W. Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 370–384 (1972).
- [68]
- M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.
- [291]
- D. McFadden. Conditional logit analysis of qualitative choice behavior. In: Frontiers in Econometrics, edited by P. Zarembka (Academic Press, New York, 1974); pp. 105–142.
- [292]
- D. R. Cox and E. J. Snell. Analysis of Binary Data. 2 Edition (Chapman and Hall, London, 1989).
- [293]
- N. J. Nagelkerke. A note on a general definition of the coefficient of determination. Biometrika 78, 691–692 (1991).
- [294]
- G. Schwarz. Estimating the dimension of a model. The Annals of Statistics 6, 461–464 (1978).
- [295]
- H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19, 716–723 (1974).
- [296]
- C. M. Hurvich and C.-L. Tsai. Regression and time series model selection in small samples. Biometrika 76, 297–307 (1989).
- [297]
- H. Theil. Economic Forecasts and Policy. 2 Edition (North-Holland, 1961).