GitHub

Regression: private API

PortfolioOptimisers.PSEUDO_R2_VARIANTS — Constant
PSEUDO_R2_VARIANTS

Tuple of the pseudo-$R^2$ variants StatsAPI.r2 accepts for a fitted GeneralisedLinearModel.

The members are :McFadden, :CoxSnell, :Nagelkerke and :devianceratio. The variant field of GeneralisedLinearModel is checked against this tuple at construction. A generalised linear model has no classical $R^2$, so each member scores the fitted model against the intercept-only model of the same family instead.

Mathematical definition

\[\begin{align} R^2_{\mathrm{McF}} &= 1 - \frac{\ln\hat{L}}{\ln\hat{L}_{0}}\,,\\ R^2_{\mathrm{CS}} &= 1 - \left(\frac{\hat{L}_{0}}{\hat{L}}\right)^{2/T}\,,\\ R^2_{\mathrm{N}} &= \frac{R^2_{\mathrm{CS}}}{1 - \hat{L}_{0}^{\,2/T}}\,,\\ R^2_{\mathrm{dev}} &= 1 - \frac{D}{D_{0}}\,. \end{align}\]

Where:

  • $R^2_{\mathrm{McF}}$: :McFadden.
  • $R^2_{\mathrm{CS}}$: :CoxSnell.
  • $R^2_{\mathrm{N}}$: :Nagelkerke.
  • $R^2_{\mathrm{dev}}$: :devianceratio.
  • $\hat{L}$: Maximum likelihood of the fitted model.
  • $\hat{L}_{0}$: Maximum likelihood of the intercept-only model of the same family.
  • $D$: Deviance of the fitted model.
  • $D_{0}$: Deviance of the intercept-only model of the same family.
  • $T$: Number of observations.

Three consequences follow for the Normal family, which is the default of GeneralisedLinearModel. Its deviance is the residual sum of squares, so $R^2_{\mathrm{dev}}$ is the classical $R^2$ of the same fit. Its maximum likelihood carries the fitted dispersion, so $\left(\hat{L}_{0}/\hat{L}\right)^{2/T} = D/D_{0}$ and $R^2_{\mathrm{CS}}$ equals $R^2_{\mathrm{dev}}$ exactly. Its likelihood is a density rather than a probability, so $\hat{L}_{0}$ sits on either side of one and the two forms built on the log-likelihood, $R^2_{\mathrm{McF}}$ and $R^2_{\mathrm{N}}$, leave $[0, 1]$ in either direction: rescaling the response alone moves both from above one to below zero, while $R^2_{\mathrm{dev}}$ does not move at all. Only $R^2_{\mathrm{dev}}$ is continuous with the LinearModel path, which is why default_regression_criterion_variant returns :devianceratio.

Related

References

  • [147] D. McFadden. Conditional logit analysis of qualitative choice behavior. In: Frontiers in Econometrics, edited by P. Zarembka (Academic Press, 1974); pp. 105–142.
  • [148] D. R. Cox and E. J. Snell. Analysis of Binary Data. 2 Edition (Chapman and Hall, 1989).
  • [149] N. J. Nagelkerke. A note on a general definition of the coefficient of determination. Biometrika 78, 691–692 (1991).
  • [32] J. A. Nelder and R. W. Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 370–384 (1972).
source
PortfolioOptimisers.ADJUSTED_PSEUDO_R2_VARIANTS — Constant
ADJUSTED_PSEUDO_R2_VARIANTS

Tuple of the pseudo-$R^2$ variants StatsAPI.adjr2 accepts for a fitted GeneralisedLinearModel.

The members are :McFadden and :devianceratio, a strict subset of PSEUDO_R2_VARIANTS. GeneralisedLinearModel cannot check against this tuple, because it does not know which criterion will read its variant. StepwiseRegression checks it instead: it is the first type that holds the criterion and the target together. StatsAPI.adjr2 raises an ArgumentError on either variant this tuple omits.

Mathematical definition

Each member discounts its unadjusted form of PSEUDO_R2_VARIANTS by the parameters the model consumes.

\[\begin{align} \bar{R}^2_{\mathrm{McF}} &= 1 - \frac{\ln\hat{L} - k}{\ln\hat{L}_{0}}\,,\\ \bar{R}^2_{\mathrm{dev}} &= 1 - \frac{D\,(T - 1)}{D_{0}\,(T - k)}\,. \end{align}\]

Where:

  • $\bar{R}^2_{\mathrm{McF}}$: :McFadden.
  • $\bar{R}^2_{\mathrm{dev}}$: :devianceratio.
  • $\hat{L}$: Maximum likelihood of the fitted model.
  • $\hat{L}_{0}$: Maximum likelihood of the intercept-only model of the same family.
  • $D$: Deviance of the fitted model.
  • $D_{0}$: Deviance of the intercept-only model of the same family.
  • $k$: Number of estimated parameters, which is StatsAPI.dof of the fitted model: the regression coefficients, the intercept, and the dispersion.
  • $T$: Number of observations.

The $k$ here is the one :aic, :aicc and :bic read, not the predictor count :adjr2 reads on a fitted LinearModel. See STEPWISE_REGRESSION_CRITERIA, which states both.

Related

References

  • [147] D. McFadden. Conditional logit analysis of qualitative choice behavior. In: Frontiers in Econometrics, edited by P. Zarembka (Academic Press, 1974); pp. 105–142.
  • [32] J. A. Nelder and R. W. Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 370–384 (1972).
source
PortfolioOptimisers.STEPWISE_REGRESSION_CRITERIA — Constant
STEPWISE_REGRESSION_CRITERIA

Tuple of the symbols that name a stepwise regression criterion scoring a fitted model with one number.

StepwiseRegression accepts any symbol of this tuple in its crit field and stores it as a Val, which is what regression_criterion_func, regression_threshold and the get_*_reg_incl*! helpers dispatch on. A symbol outside the tuple is rejected at construction. PValue is not a member: it reads the coefficient p-values of the fitted model instead of one score, so it stays a type and takes its own stepwise methods. :aic, :aicc and :bic score a fitted LinearModel and a fitted GeneralisedLinearModel alike, while :r2 and :adjr2 are defined for a fitted LinearModel only and read a named pseudo-$R^2$ variant under the other target.

Mathematical definition

:aic — Akaike Information Criterion

Trades the fitted likelihood against the number of estimated parameters, so a lower value is a better model.

\[\begin{align} \mathrm{AIC} &= 2k - 2\ln\hat{L}\,. \end{align}\]

Where:

  • $k$: Number of estimated parameters, which is StatsAPI.dof of the fitted model: the regression coefficients, the intercept, and the residual variance.
  • $\hat{L}$: Maximum likelihood of the model.

:aicc — Akaike Information Criterion corrected for a small sample

The correction term grows as $T$ approaches $k$, so :aicc penalises a large model more heavily than :aic does on a short sample.

\[\begin{align} \mathrm{AICC} &= \mathrm{AIC} + \frac{2k(k+1)}{T - k - 1}\,. \end{align}\]

Where:

  • $k$: Number of estimated parameters, as for :aic.
  • $T$: Number of observations.

The correction term divides by $T - k - 1$. The criterion is undefined when $T = k + 1$ and changes sign below it, so :aicc needs a sample longer than the largest model the search can reach.

:bic — Bayesian Information Criterion

The penalty is $k \ln T$ rather than $2k$, which is heavier than the penalty of :aic for $T \geq 8$, so the search usually stops with fewer factors.

\[\begin{align} \mathrm{BIC} &= k\ln T - 2\ln\hat{L}\,. \end{align}\]

Where:

  • $k$: Number of estimated parameters, as for :aic.
  • $T$: Number of observations.
  • $\hat{L}$: Maximum likelihood of the model.

:r2 — coefficient of determination

The share of the response variance the model explains, so a higher value is a better fit.

\[\begin{align} R^2 &= 1 - \frac{\mathrm{SS}_{\mathrm{res}}}{\mathrm{SS}_{\mathrm{tot}}} = 1 - \frac{\sum_t (y_t - \hat{y}_t)^2}{\sum_t (y_t - \bar{y})^2}\,. \end{align}\]

Where:

  • $\mathrm{SS}_{\mathrm{res}}$: Residual sum of squares.
  • $\mathrm{SS}_{\mathrm{tot}}$: Total sum of squares.
  • $y_t$: Observed response at time $t$.
  • $\hat{y}_t$: Fitted response at time $t$.
  • $\bar{y}$: Mean of observed responses.

$R^2$ never falls when a factor is added, so it penalises no complexity at all. Under ForwardSelection it admits every factor and under BackwardElimination it removes none. Use :adjr2, :aic, :aicc or :bic when the criterion must pay for size.

:adjr2 — coefficient of determination adjusted for the model size

The adjustment discounts $R^2$ by the degrees of freedom the predictors consume, so unlike :r2 the score can fall when a factor is added. This makes it usable as a stopping rule.

\[\begin{align} \bar{R}^2 &= 1 - (1 - R^2) \frac{T - 1}{T - k - 1}\,. \end{align}\]

Where:

  • $k$: Number of predictors, excluding the intercept. :aic, :aicc and :bic write $k$ for a different count: the predictors, plus the intercept, plus the residual variance.
  • $T$: Number of observations.

Related

References

  • [150] H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19, 716–723 (1974).
  • [151] C. M. Hurvich and C.-L. Tsai. Regression and time series model selection in small samples. Biometrika 76, 297–307 (1989).
  • [152] G. Schwarz. Estimating the dimension of a model. The Annals of Statistics 6, 461–464 (1978).
  • [34] R. R. Hocking. The analysis and selection of variables in linear regression. Biometrics 32, 1–49 (1976).
  • [153] H. Theil. Economic Forecasts and Policy. 2 Edition (North-Holland, 1961).
source
PortfolioOptimisers.AbstractRegressionEstimator — Type
abstract type AbstractRegressionEstimator <: AbstractEstimator

Abstract supertype of every regression estimator, over both the time-series family and the cross-sectional family.

The type is an umbrella and declares no interface of its own, because the two families fit different models and answer different verbs. A time-series estimator fits one model per asset over the observations and answers regression. A cross-sectional estimator fits one model per observation across the assets and answers cross_sectional_regression. Subtype the child that names the family, never this root, so a consumer of one family never receives a value of the other.

Related

source
PortfolioOptimisers.AbstractRegressionResult — Type
abstract type AbstractRegressionResult <: AbstractResult

Abstract supertype of every regression result, over both the loadings family and the cross-sectional family.

The type is an umbrella, and the two children disagree on what an asset index means. A loadings result holds one row of M per asset, so port_opt_view slices its rows. A cross-sectional result holds one row per observation and one column per asset, so the same index slices its columns. Subtype the child that states what the result carries, never this root.

Related

source
PortfolioOptimisers.AbstractLoadingsRegressionResult — Type
abstract type AbstractLoadingsRegressionResult <: AbstractRegressionResult

Abstract supertype for all regression result types that carry a loadings matrix.

All concrete and/or abstract types representing the output of a regression that carries a loadings matrix M, one row per asset and one column per factor, should be subtypes of AbstractLoadingsRegressionResult. The root states what a member carries, not how it was fitted, because every consumer that re-bases a constraint or decomposes risk in the factor basis reads M and binds this type rather than the umbrella. The fitting geometry is not the criterion: a result fitted per asset over the observations and a result fitted per observation across the assets both belong here when they carry M.

Related

source
PortfolioOptimisers.AbstractCrossSectionalRegressionResult — Type
abstract type AbstractCrossSectionalRegressionResult <: AbstractRegressionResult

Abstract supertype for all cross-sectional regression result types.

All concrete and/or abstract types representing the output of a regression fitted per observation across the assets should be subtypes of AbstractCrossSectionalRegressionResult. A member carries no loadings matrix, because the exposures are the regression's input and an Exposure Estimator produces them.

Related

source
PortfolioOptimisers.AbstractFactorFamilyBasis — Type
abstract type AbstractFactorFamilyBasis <: AbstractResult

Abstract supertype for all Factor Family Basis result types.

All concrete and/or abstract types representing the change of basis that a re-based Factor Family is written in should be subtypes of AbstractFactorFamilyBasis. The root exists so that CrossSectionalFactorModel can bind its fcb slot to a type rather than to Any, and it is declared here rather than beside its member because the block that carries the slot is loaded before the member that fills it.

Related

source
PortfolioOptimisers.AbstractRegressionAlgorithm — Type
abstract type AbstractRegressionAlgorithm <: AbstractAlgorithm

Abstract supertype for all regression algorithm types.

All concrete and/or abstract types implementing specific regression algorithms should be subtypes of AbstractRegressionAlgorithm.

These types are used to specify the algorithm when constructing a regression estimator.

Related

source
PortfolioOptimisers.AbstractStepwiseRegressionAlgorithm — Type
abstract type AbstractStepwiseRegressionAlgorithm <: AbstractRegressionAlgorithm

Abstract supertype for all stepwise regression algorithm types.

All concrete and/or abstract types implementing stepwise regression algorithms should be subtypes of AbstractStepwiseRegressionAlgorithm. A stepwise algorithm decides the direction the factor set moves in, and an AbstractStepwiseRegressionCriterion decides which move is an improvement.

Related

References

  • [33] M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.
source
PortfolioOptimisers.AbstractStepwiseRegressionCriterion — Type
abstract type AbstractStepwiseRegressionCriterion <: AbstractRegressionAlgorithm

Abstract supertype for all stepwise regression criterion types.

All concrete and/or abstract types representing criteria for stepwise regression algorithms should be subtypes of AbstractStepwiseRegressionCriterion. A criterion scores a fitted model, and the stepwise algorithm keeps the move that improves the score.

Related

References

  • [34] R. R. Hocking. The analysis and selection of variables in linear regression. Biometrics 32, 1–49 (1976).
source
PortfolioOptimisers.MinValStepwiseRegressionCriterion — Type
MinValStepwiseRegressionCriterion

Union of the Val types naming a stepwise regression criterion that a lower value scores better.

The members are built from MIN_VAL_STEPWISE_REGRESSION_CRITERIA, so the union is Union{Val{:aic}, Val{:aicc}, Val{:bic}}. StepwiseRegression stores Val(crit) in its crit field, so a member of this union is the first type parameter of the estimator whenever the criterion minimises.

Related

source
PortfolioOptimisers.MaxValStepwiseRegressionCriterion — Type
MaxValStepwiseRegressionCriterion

Union of the Val types naming a stepwise regression criterion that a higher value scores better.

The members are built from MAX_VAL_STEPWISE_REGRESSION_CRITERIA, so the union is Union{Val{:r2}, Val{:adjr2}}. Both members read a pseudo-$R^2$ variant under a GeneralisedLinearModel target.

Related

source
PortfolioOptimisers.MinMaxValStepwiseRegressionCriterion — Type
MinMaxValStepwiseRegressionCriterion

Union of every Val type naming a stepwise regression criterion that scores a fitted model with one number.

This is the union of MinValStepwiseRegressionCriterion and MaxValStepwiseRegressionCriterion. It is the bound the crit field of StepwiseRegression takes when the criterion is one score rather than a PValue.

Related

source
PortfolioOptimisers.RegE_Reg — Type
const RegE_Reg = Union{<:AbstractLoadingsRegressionResult,
                       <:AbstractTimeSeriesRegressionEstimator}

Alias for a loadings regression result or a time-series regression estimator.

Matches either an AbstractLoadingsRegressionResult (a pre-computed result that carries the loadings matrix M) or an AbstractTimeSeriesRegressionEstimator (a specification whose verb produces one). Used for dispatch in factor model and regression-based risk routines. Every consumer of the alias reads M, so the alias names the two ways a consumer obtains it rather than the umbrella.

The two arms state different criteria, and the asymmetry is deliberate. A result carries a payload, so the result arm names what it carries and admits any loadings result whatever its fitting geometry. An estimator carries no payload, so the estimator arm names the family whose verb, regression, returns a loadings result. A cross-sectional estimator answers cross_sectional_regression, whose result carries no loadings, so it stays outside the alias.

Related

source
PortfolioOptimisers.set_idiosyncratic_covariance — Method
set_idiosyncratic_covariance(re::Regression, esigma::Option{<:VecNum_MatNum})

Return a Regression that carries esigma, with every other field unchanged.

A regression estimator fits loadings alone, so the block a fit returns carries no idiosyncratic covariance. The prior that lifts the factor moments measures the residual variances on the way, and it writes them here rather than making every consumer recompute them. FactorPrior and FactorBlackLittermanPrior are the two callers, and each passes what factor_lift returned: the variances under rsd = true, and nothing under rsd = false.

Accessors.@set cannot do this. It reads the fields through property access, and the swap(L, M) rule of Regression makes re.L return re.M when L is unset, so the rebuilt result would carry a copy of M under L and isnothing(getfield(re, :L)) would stop being true. This method reads L and b with getfield for that reason, as port_opt_view does.

Arguments

  • re: The regression result to rewrite.
  • esigma: The idiosyncratic covariance to write, or nothing to leave the field unset.

Returns

  • re::Regression: A new result carrying esigma, which re-runs every guard of the constructor.

Related

source
PortfolioOptimisers.has_family_rebasis — Method
has_family_rebasis(
    _::AbstractLoadingsRegressionResult
) -> Bool

State whether a loadings result was fitted in a re-based factor family.

A Factor Family whose one-hot exposures are collinear with a global factor is re-based before the fit: one factor of the family is dropped, and the family is rewritten in an equivalent basis of full column rank. A result that carries such a re-basis keeps the raw loadings in M and the re-based ones in L, so its raw factor axis is a linear image of a smaller one, and a factor covariance stated on that axis is singular by construction. A consumer that inverts or factorises such a covariance reads this trait and refuses the result, rather than letting the factorisation fail on a matrix it was handed.

The root answers false, and a member that cannot state a re-basis needs no method. That fallback is an answer rather than a missing declaration: a Regression fits one model per asset over the observations and re-bases nothing, so false is true of it. Only CrossSectionalFactorModel overrides it.

Arguments

  • rr: Loadings regression result.

Returns

  • val::Bool: true when the result carries a family re-basis, so its raw factor axis is rank deficient; false otherwise.

Examples

julia> PortfolioOptimisers.has_family_rebasis(Regression(; M = [1 2; 3 4], b = [1, 2]))false

Related

source
PortfolioOptimisers.default_regression_criterion_variant — Function
default_regression_criterion_variant(crit::MaxValStepwiseRegressionCriterion)

Return the pseudo-$R^2$ variant a maximisation criterion reads when the target names none.

:devianceratio is the only member of PSEUDO_R2_VARIANTS that both StatsAPI.r2 and StatsAPI.adjr2 accept and that also reproduces the classical $R^2$ of a fitted LinearModel on a Normal family. The default therefore keeps the score continuous with the LinearModel path under either criterion.

Arguments

Returns

  • variant::Symbol: :devianceratio.

Related

source
PortfolioOptimisers.regression_criterion_func — Function
regression_criterion_func(crit::MinMaxValStepwiseRegressionCriterion,
                          tgt::AbstractRegressionTarget)

Return the function that scores a fitted model under a stepwise regression criterion.

The method dispatches on the Val naming the criterion and on the regression target, because the two maximisation criteria read a different quantity under each target. The map is:

CriterionLinearModelGeneralisedLinearModel
:aicStatsAPI.aicStatsAPI.aic
:aiccStatsAPI.aiccStatsAPI.aicc
:bicStatsAPI.bicStatsAPI.bic
:r2StatsAPI.r2model -> StatsAPI.r2(model, variant)
:adjr2StatsAPI.adjr2model -> StatsAPI.adjr2(model, variant)

StatsAPI.aic, StatsAPI.aicc and StatsAPI.bic accept a fitted model of either target, so the three minimisation criteria take one method each. StatsAPI.r2 and StatsAPI.adjr2 accept a fitted LinearModel without a variant, and a fitted GeneralisedLinearModel needs a named pseudo-$R^2$ variant, so the two maximisation criteria take two methods each. PValue has no method here: it reads the coefficient p-values of the fitted model rather than one score, so its stepwise methods are separate.

Algorithm

The five methods that return a StatsAPI function run no steps. The two that close over a variant run these:

  1. Read tgt.variant. When it is nothing, take default_regression_criterion_variant of crit instead, giving variant.
  2. Build a closure over variant that calls StatsAPI.r2(model, variant), or StatsAPI.adjr2(model, variant) under :adjr2.

Arguments

  • crit: Criterion, as the Val the crit field of StepwiseRegression holds.
  • tgt: Regression target the candidate models are fitted with.

Returns

  • f::Function: The function that computes the criterion value for a fitted model.

Related

source
PortfolioOptimisers.regression_polarity — Function
regression_polarity(crit::MinMaxValStepwiseRegressionCriterion)

Return the three functions that state which direction of a stepwise criterion is better.

A stepwise search asks the same three questions of every criterion: which entry of a score vector is the best one, whether a candidate score improves on the score in hand, and what the worst score of a type is. Each answer is one function under a minimised criterion and its opposite under a maximised one. This is the only method pair in the library that states the pairing, so regression_threshold, get_forward_reg_incl_excl!, get_backward_reg_incl! and the two _regression methods that seed a score vector all read it rather than restate it.

Arguments

Returns

  • polarity::NamedTuple: Three functions.

    • best: findmin under a minimised criterion, findmax under a maximised one. Returns the best entry of a score vector and its index.
    • improves: < under a minimised criterion, > under a maximised one. Answers whether the first score is better than the second.
    • worst: typemax under a minimised criterion, typemin under a maximised one. Returns the worst score of the type it is given.

Related

source
PortfolioOptimisers.regression_threshold — Function
regression_threshold(crit::MinMaxValStepwiseRegressionCriterion)

Return the starting threshold for a forward stepwise regression search.

The value is the worst score the criterion can take, so the first candidate model always improves on it. regression_polarity states which of typemax and typemin that is, and typemax(Float64) is Inf. Only ForwardSelection reads it: BackwardElimination starts from the score of the full model instead, because its first move must beat a model that already exists.

Arguments

Returns

  • t::Number: Inf for a minimisation criterion, -Inf for a maximisation criterion.

Related

source

References

[32]
J. A. Nelder and R. W. Wedderburn. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 370–384 (1972).
[33]
M. A. Efroymson. Multiple regression analysis. In: Mathematical Methods for Digital Computers, edited by A. Ralston and H. S. Wilf (John Wiley & Sons, 1960); pp. 191–203.
[34]
R. R. Hocking. The analysis and selection of variables in linear regression. Biometrics 32, 1–49 (1976).
[147]
D. McFadden. Conditional logit analysis of qualitative choice behavior. In: Frontiers in Econometrics, edited by P. Zarembka (Academic Press, New York, 1974); pp. 105–142.
[148]
D. R. Cox and E. J. Snell. Analysis of Binary Data. 2 Edition (Chapman and Hall, London, 1989).
[149]
N. J. Nagelkerke. A note on a general definition of the coefficient of determination. Biometrika 78, 691–692 (1991).
[150]
H. Akaike. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19, 716–723 (1974).
[151]
C. M. Hurvich and C.-L. Tsai. Regression and time series model selection in small samples. Biometrika 76, 297–307 (1989).
[152]
G. Schwarz. Estimating the dimension of a model. The Annals of Statistics 6, 461–464 (1978).
[153]
H. Theil. Economic Forecasts and Policy. 2 Edition (North-Holland, 1961).