Elastic net

ElasticNet is a penalised regression target. It fits a least-squares regression with a penalty on the $L_1$ norm and on the squared $L_2$ norm of the coefficients, and every estimator that takes a regression target can fit it. The strength of the penalty comes from a rule: MiddleOfPath reads it off the data at the middle of the regularisation path, and FixedStrength takes it from you. A new rule subtypes AbstractPenaltyStrength and adds a method of penalty_strength.

The fit states its effective degrees of freedom, so StepwiseRegression can score it under :aic, :aicc and :bic. StrengthDivergence adds the term of a strength that reads the data to that count.

Strength rules

PortfolioOptimisers.AbstractPenaltyStrength — Type
abstract type AbstractPenaltyStrength <: AbstractAlgorithm

Abstract supertype for the rules that set the regularisation strength of an ElasticNet.

The strength scales the penalty against the weighted squared residuals. MiddleOfPath reads it off the data, as Lai et al. [101] do. FixedStrength takes it from the caller.

Interfaces

To add a strength rule, subtype AbstractPenaltyStrength, keep its parameters in the fields of the struct, and add the methods below. A target with sdiv = nothing does not read penalty_strength_gradient, so a rule that runs under that target alone needs only penalty_strength.

penalty_strength

  • penalty_strength(s::MyPenaltyStrength, g::VecNum, theta::Real) -> Real: Return the regularisation strength $\gamma$ of the fit.

StatsAPI.fit(::ElasticNet, ::MatNum, ::VecNum) calls it once, before the coordinate sweeps. When every entry of g is zero, the fit returns its start point at every strength, and it does not read the strength. It records the strength in the fit.

Arguments

  • s: The new strength rule.
  • g: The product of each penalised column with the residual of the start point, $\mathbf{P}_k^\intercal \mathbf{V} \boldsymbol{r}_0$ for $k \in \mathcal{K}$. It is empty when the design has no penalised column.
  • theta: The share of the $L_1$ penalty, the theta field of the target.

Returns

  • gamma::Real: The regularisation strength, non-negative.

penalty_strength_gradient

  • penalty_strength_gradient(s::MyPenaltyStrength, g::VecNum, theta::Real) -> VecNum: Return the gradient of the strength that penalty_strength returns, with respect to g.

StrengthDivergence reads it, because a strength that reads the response adds a term to the divergence of the fit. A rule that does not read g returns zeros.

Arguments

  • s: The new strength rule.
  • g: The vector that penalty_strength reads.
  • theta: The share of the $L_1$ penalty.

Returns

  • dg::VecNum: The partial derivative of $\gamma$ with respect to each entry of g, one entry per entry of g.

Related

source
PortfolioOptimisers.penalty_strength — Function
penalty_strength(s::MiddleOfPath, g::AbstractVector, theta::Real) -> Real
penalty_strength(s::FixedStrength, g::AbstractVector, theta::Real) -> Real

Return the regularisation strength of an ElasticNet under its strength rule.

MiddleOfPath states the middle of the path, and FixedStrength returns its own gamma. The # Interfaces section of AbstractPenaltyStrength states the method that a rule of the caller adds.

Algorithm

The method that Julia selects is the algorithm.

  1. MiddleOfPath: divide the largest absolute entry of g by theta, which gives lmax, and return lmax * sqrt(s.ratio). An empty g gives zero.
  2. FixedStrength: return s.gamma, and read neither g nor theta.

Arguments

  • s: The strength rule.
  • g: The product of each penalised column with the residual of the start point.
  • theta: The share of the $L_1$ penalty.

Returns

  • gamma::Real: The regularisation strength.

Related

source
PortfolioOptimisers.penalty_strength_gradient — Function
penalty_strength_gradient(s::MiddleOfPath, g::AbstractVector, theta::Real) -> VecNum
penalty_strength_gradient(s::FixedStrength, g::AbstractVector, theta::Real) -> VecNum

Return the gradient of the regularisation strength of an ElasticNet with respect to the vector that penalty_strength reads.

StrengthDivergence chains it to the gradient with respect to the response. The # Interfaces section of AbstractPenaltyStrength states the method that a rule of the caller adds.

Mathematical definition

\[\begin{align} \frac{\partial \gamma}{\partial g_{j}} &= \frac{\sqrt{\rho}}{\vartheta} \operatorname{sign}(g_{k^{*}})\, \mathbb{1}[j = k^{*}]\,,\quad k^{*} = \underset{k}{\arg\max} \lvert g_{k} \rvert\,. \end{align}\]

Where:

  • $\gamma$: Regularisation strength of the elastic net.
  • $g_{k}$: Entry $k$ of g, the product of penalised column $k$ with the residual of the start point.
  • $\rho$: Floor of the path as a fraction of $\gamma_{\max}$, the ratio field of MiddleOfPath.
  • $\vartheta$: Share of the $L_1$ penalty in the elastic net, the theta field of ElasticNet.

The form is the derivative of $\gamma = \sqrt{\rho} \max_{k} \lvert g_{k} \rvert / \vartheta$, at a point where one entry holds the largest absolute value.

Algorithm

The method that Julia selects is the algorithm.

  1. MiddleOfPath: return the empty g when g is empty. Otherwise find the index k of the largest absolute entry of g, and return a vector of zeros with sign(g[k]) * sqrt(s.ratio) / theta at k.
  2. FixedStrength: return zeros, because the strength does not read g.

Arguments

  • s: The strength rule.
  • g: The product of each penalised column with the residual of the start point.
  • theta: The share of the $L_1$ penalty.

Returns

  • dg::VecNum: The gradient, one entry per entry of g.

Related

source
PortfolioOptimisers.MiddleOfPath — Type
struct MiddleOfPath{__T_ratio} <: AbstractPenaltyStrength

Sets the regularisation strength of an ElasticNet at the geometric middle of its regularisation path.

Lai et al. [101] take "the middle one in the regularization path" and state no grid. Section 2.5 of [108] runs the path over 100 strengths on a log scale, from the smallest strength at which every penalised coefficient is zero down to ratio times that strength. Such a grid has no single middle point. The rule takes the geometric middle of the path, which lies between the 50th and the 51st point of that grid at the default ratio = 1e-3.

Mathematical definition

\[\begin{align} \gamma_{\max} &= \max_{k \in \mathcal{K}} \frac{\lvert \mathbf{P}_k^\intercal \mathbf{V} \boldsymbol{r}_0 \rvert}{\vartheta}\,,\\ \gamma &= \gamma_{\max} \sqrt{\rho}\,. \end{align}\]

Where:

  • $\gamma_{\max}$: Smallest strength at which every penalised coefficient is zero.
  • $\mathcal{K}$: Penalised columns of the elastic net, every column of $\mathbf{P}$ but the intercept column that an estimator names.
  • $\mathbf{P}$: Regressors of the elastic net, one row per observation and one column per regressor. Column $k$ is $\mathbf{P}_k$.
  • $\mathbf{V}$: Diagonal matrix of the weights of the observations of the elastic net, the identity when the fit has no observation weights.
  • $\boldsymbol{r}_0$: Residual of the start point, the response less the fit of the intercept column alone, or the response when the design has no intercept column.
  • $\vartheta$: Share of the $L_1$ penalty in the elastic net, the theta field of ElasticNet.
  • $\gamma$: Regularisation strength of the elastic net.
  • $\rho$: Floor of the path as a fraction of $\gamma_{\max}$, the ratio field.

Every penalised coefficient is zero at and above $\gamma_{\max}$. The path runs from $\gamma_{\max}$ down to $\rho\, \gamma_{\max}$ on a log scale, so its geometric middle is $\gamma_{\max} \sqrt{\rho}$. A ridge alone, $\vartheta = 0$, has no $\gamma_{\max}$, so the constructor of ElasticNet refuses theta = 0 with this rule.

Fields

  • ratio: The floor of the path as a fraction of the strength above which every penalised coefficient is zero.

Constructors

MiddleOfPath(; ratio::Real = 1e-3) -> MiddleOfPath

Keywords correspond to the struct's fields.

Validation

  • 0 < ratio < 1. A DomainError is thrown otherwise.

Examples

julia> MiddleOfPath()MiddleOfPath  ratio ┴ Float64: 0.001

Related

References

  • [108] J. Friedman, T. Hastie and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1–22 (2010). Section 2.5.
  • [101] Z.-R. Lai, P.-Y. Yang, X. Wu and L. Fang. A kernel-based trend pattern tracking system for portfolio optimization. Data Mining and Knowledge Discovery 32, 1708–1734 (2018). Section 3.1.2.
source
PortfolioOptimisers.FixedStrength — Type
struct FixedStrength{__T_gamma} <: AbstractPenaltyStrength

Sets the regularisation strength of an ElasticNet to a value that the caller states.

gamma is the $\gamma$ of the objective of ElasticNet. It weighs the penalty against the sum of the weighted squared residuals, and it does not divide that sum by the number of observations. So the same gamma penalises a fit over more observations less, in relation to its residuals. Friedman et al. [108] divide the objective by twice the sum of the weights, so their strength $\lambda$ of Equation (1) is $\gamma / T_{e}$, with $T_{e}$ the sum of the scaled weights, which is the number of observations of an unweighted fit. With a variance estimator in the ve field of the target, gamma acts on the standardised columns. theta = 0 with this rule is a ridge regression.

Fields

  • gamma: The regularisation strength $\gamma$ of the objective.

Constructors

FixedStrength(; gamma::Real) -> FixedStrength

Keywords correspond to the struct's fields.

Validation

  • gamma > 0. A DomainError is thrown otherwise.

Examples

julia> FixedStrength(; gamma = 0.5)FixedStrength  gamma ┴ Float64: 0.5

Related

References

  • [108] J. Friedman, T. Hastie and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1–22 (2010). Equation (1).
source

Strength divergence

PortfolioOptimisers.AbstractStrengthDivergence — Type
abstract type AbstractStrengthDivergence <: AbstractAlgorithm

Abstract supertype for the rules that add the term of a strength that reads the response to the effective degrees of freedom of an ElasticNet.

The trace of the smoother on the active set is the effective degrees of freedom of a fit at a fixed strength, Remark (The elastic net) after Equation (28) of [109]. MiddleOfPath reads the strength off the response, so the divergence of its fit has one more term. StrengthDivergence adds it. The sdiv field of ElasticNet holds the rule, and nothing, the default, leaves the term out.

Interfaces

To add a rule, subtype AbstractStrengthDivergence, and add the method below.

strength_divergence

  • strength_divergence(sdiv::MyStrengthDivergence, tgt::ElasticNet, sys::NamedTuple) -> Real: Return the term that the rule adds to the trace of the smoother.

StatsAPI.fit(::ElasticNet, ::MatNum, ::VecNum) calls it once, after the fit, through elastic_net_counts. The term changes df of ElasticNetFit alone, not the residual count rdof.

Arguments

  • sdiv: The new rule.

  • tgt: The regression target.

  • sys::NamedTuple: The fit, (; P, Q, y, z, pen, ridge, g, lam, nrm2, w, A, Mi):

    • P, Q: The standardised design and the weighted design, as elastic_net_design returns them.
    • y: The response.
    • z: The coefficients of the fit on the columns of P.
    • pen: Whether each column is penalised.
    • ridge: The ridge term of each column.
    • g, lam: The vector that penalty_strength read, and the strength that it returned.
    • nrm2: The weighted squared norm of each column.
    • w: The frequency weights of the rows, or nothing.
    • A: The indices of the active set.
    • Mi: The pseudo-inverse of the matrix of the equations of the active set.

Returns

  • t::Real: The term.

Related

References

  • [109] R. J. Tibshirani and J. Taylor. Degrees of freedom in lasso problems. The Annals of Statistics 40, 1198–1232 (2012). Remark (The elastic net) after Equation (28).
source
PortfolioOptimisers.StrengthDivergence — Type
struct StrengthDivergence <: AbstractStrengthDivergence

Adds the term of a strength that reads the response to the effective degrees of freedom of an ElasticNet.

The degrees of freedom of a fit $\hat{\boldsymbol{\mu}}$ is the expected divergence $\sum_{i} \partial \hat{\mu}_{i} / \partial y_{i}$, Equations (10) and (11) of [109]. On a fixed active set, the fit at a fixed strength is linear in the response, and its divergence is the trace of the smoother. A strength that reads the response adds the chain term below. Under FixedStrength the gradient of the strength is zero, so the rule adds zero.

Mathematical definition

\[\begin{align} \operatorname{div} \hat{\boldsymbol{\mu}} &= \operatorname{tr}(\mathbf{H}) + \nabla_{\boldsymbol{y}} \gamma^{\intercal} \frac{\partial \hat{\boldsymbol{\mu}}}{\partial \gamma}\,,\\ \frac{\partial \hat{\boldsymbol{\mu}}}{\partial \gamma} &= -\mathbf{P}_{\mathcal{A}} \mathbf{M}^{-1} \left((1 - \vartheta) \mathbf{D}_{\mathcal{A}} \boldsymbol{z}_{\mathcal{A}} + \vartheta \boldsymbol{s}\right)\,,\\ \nabla_{\boldsymbol{y}} \gamma &= (\mathbf{I} - \mathbf{H}_{0})^{\intercal} \mathbf{V} \mathbf{P}_{\mathcal{K}} \frac{\partial \gamma}{\partial \boldsymbol{g}}\,. \end{align}\]

Where:

  • $\hat{\boldsymbol{\mu}}$: Fitted response, $\mathbf{P} \hat{\boldsymbol{z}}$.
  • $\mathbf{H}$: Smoother of the fit on its active set, which ElasticNetFit states.
  • $\mathbf{M}$: Matrix of the equations of the active set, $\mathbf{P}_{\mathcal{A}}^\intercal \mathbf{V} \mathbf{P}_{\mathcal{A}} + \gamma (1 - \vartheta) \mathbf{D}_{\mathcal{A}}$.
  • $\mathbf{P}_{\mathcal{A}}$, $\boldsymbol{z}_{\mathcal{A}}$: Columns of the regressors and coefficients of the active set.
  • $\mathbf{D}_{\mathcal{A}}$: Diagonal matrix with one for each penalised column of the active set and zero for the intercept column.
  • $\boldsymbol{s}$: Signs of the coefficients of the active set, zero for the intercept column.
  • $\mathbf{H}_{0}$: Weighted projection onto the intercept column, the zero matrix when the design has none. The start point of penalty_strength is the residual $(\mathbf{I} - \mathbf{H}_{0}) \boldsymbol{y}$.
  • $\mathbf{P}_{\mathcal{K}}$: Penalised columns of the design.
  • $\partial \gamma / \partial \boldsymbol{g}$: Gradient of the strength rule, penalty_strength_gradient.
  • $\mathbf{P}$: Regressors of the elastic net, one row per observation and one column per regressor. Column $k$ is $\mathbf{P}_k$.
  • $\mathbf{V}$: Diagonal matrix of the weights of the observations of the elastic net, the identity when the fit has no observation weights.
  • $\gamma$: Regularisation strength of the elastic net.
  • $\vartheta$: Share of the $L_1$ penalty in the elastic net, the theta field of ElasticNet.
  • $\mathcal{K}$: Penalised columns of the elastic net, every column of $\mathbf{P}$ but the intercept column that an estimator names.

The second line differentiates the equations of elastic_net_polish in $\gamma$ on the fixed active set. On the design of ElasticNet that states the gap, the count with the term matches the mean finite-difference divergence of the fit.

Examples

julia> StrengthDivergence()StrengthDivergence()

Related

References

  • [109] R. J. Tibshirani and J. Taylor. Degrees of freedom in lasso problems. The Annals of Statistics 40, 1198–1232 (2012). Equations (10) and (11), and Remark (The elastic net) after Equation (28).
source
PortfolioOptimisers.strength_divergence — Function
strength_divergence(sdiv::Nothing, tgt::ElasticNet, sys::NamedTuple) -> Real
strength_divergence(sdiv::StrengthDivergence, tgt::ElasticNet, sys::NamedTuple) -> Real

Return the term that a strength rule which reads the response adds to the effective degrees of freedom of an ElasticNet.

StrengthDivergence states the term. The # Interfaces section of AbstractStrengthDivergence states sys and the method that a rule of the caller adds.

Algorithm

The method that Julia selects is the algorithm.

  1. nothing: return zero, so the count is the trace of the smoother.
  2. StrengthDivergence: take dg, the gradient of the strength with penalty_strength_gradient. Multiply the pseudo-inverse sys.Mi by (1 - theta) z + theta sign(z) on the penalised columns of the active set, zero at the intercept column, and negate it, which gives the derivative of the active coefficients in the strength. Multiply the active columns of P by it, which gives v, the derivative of the fitted response. With an intercept column, remove from v its weighted projection onto that column. Return the product of dg with the penalised columns of Q and v.

Arguments

  • sdiv: The rule, or nothing.
  • tgt: The regression target.
  • sys: The fit.

Returns

  • t::Real: The term, $\nabla_{\boldsymbol{y}} \gamma^{\intercal} \partial \hat{\boldsymbol{\mu}} / \partial \gamma$.

Related

source

Target

PortfolioOptimisers.ElasticNet — Type
struct ElasticNet{__T_theta, __T_strength, __T_sdiv, __T_ve, __T_iters, __T_tol, __T_w, __T_ic} <: AbstractPenalisedRegressionTarget

Fits each response by the elastic net, a least-squares fit with a penalty on the $L_1$ norm and on the squared $L_2$ norm of the coefficients.

The objective is Equation (15) of [101], the elastic net of Zou and Hastie [110], under observation weights. It is Equation (3) of [110] with $\lambda_1 = 2 \vartheta \gamma$ and $\lambda_2 = (1 - \vartheta) \gamma$, so $\vartheta$ is the mixing weight of Equation (2) of [108], not the ratio $\lambda_2 / (\lambda_1 + \lambda_2)$ of [110]. Equation (10) of [108] is the coordinate update. strength sets $\gamma$: MiddleOfPath, the default, reads it off the data as Lai et al. [101] do, and FixedStrength takes it from the caller. theta = 0.99 is the value of [101]. It is almost the lasso, and it keeps the objective strictly convex.

The target adds no intercept and does not standardise the columns by default, as in the model of [101]. The glmnet package does both by default. An estimator that adds a column of ones names it with InterceptColumn through factory, and the fit then leaves that coefficient out of the penalty. A variance estimator in ve divides each penalised column by its standard deviation before the fit, and the fit scales the coefficients back. A column whose standard deviation is zero has no scale, so it keeps its units. ve reads the observation weights of the target, so the scale and the fit weigh the rows alike.

The observation weights enter as the frequency weights of glm_weights, the weights that LinearModel passes to GLM. Weights that are not StatsBase.FrequencyWeights then sum to their Kish effective sample size. Uniform weights give the unweighted fit at the same strength. The scale of the weights moves the fit under FixedStrength alone, because MiddleOfPath scales with them.

When the columns are almost collinear, for example the levels of consecutive periods in KernelTrendPattern, the coordinate sweeps converge slowly. Each sweep therefore ends with an exact solve on the sign pattern that it leaves, see elastic_net_polish. The sweeps must still find the sign pattern of the optimum. On the 4 × 5 window of the test fixture of KernelTrendPattern they find it after between 2000 and 5000 sweeps, and a cap of 1000 stops more than 0.5 from the optimum in one coefficient. When the sweeps stop at tol or at iters before that, the result is the last sweep, which is not the optimum.

The penalty depends on the units of each column, so the fit is not invariant to the basis of the design, and is_basis_invariant returns false. StepwiseRegression accepts the target under :aic, :aicc and :bic alone, because these criteria read the effective degrees of freedom that ElasticNetFit states.

The effective degrees of freedom is the trace of the smoother on the active set of the fit. That trace is exact at a fixed strength. MiddleOfPath reads the strength off the response, so the divergence of its fit has one more term, and sdiv gates it. With sdiv = nothing, the default, the count leaves the term out. The gap was measured on 2000 draws of a design of 60 observations, an intercept and 5 factors, with the coefficients [0.5, 1, -0.5, 0, 0, 0.25] and a unit noise. There the trace was above the mean finite-difference divergence of the fit by 0.06 at theta = 0.5 and by 0.03 at theta = 0.99, with a standard error of 0.014. One factor costs about one degree of freedom, so the gap did not change a comparison of two models. StrengthDivergence adds the term. Under FixedStrength the term is zero, so StrengthDivergence() adds zero.

Mathematical definition

\[\begin{align} \mathbf{P} &= \mathbf{X} \mathbf{S}^{-1}\,,\\ \hat{\boldsymbol{z}} &= \underset{\boldsymbol{z}}{\arg\min} \left(\boldsymbol{y} - \mathbf{P} \boldsymbol{z}\right)^\intercal \mathbf{V} \left(\boldsymbol{y} - \mathbf{P} \boldsymbol{z}\right) + \gamma \sum_{k \in \mathcal{K}} \left( 2 \vartheta \lvert z_k \rvert + (1 - \vartheta) z_k^2 \right)\,,\\ \hat{\boldsymbol{b}} &= \mathbf{S}^{-1} \hat{\boldsymbol{z}}\,. \end{align}\]

Where:

  • $\mathbf{X}$: Design matrix of the caller, one row per observation and one column per regressor.
  • $\mathbf{S}$: Diagonal matrix of the standard deviations of the columns of $\mathbf{X}$ under ve, with one in place of the intercept column and of a column of zero standard deviation. It is the identity when ve is nothing.
  • $\mathbf{P}$: Regressors of the elastic net, one row per observation and one column per regressor. Column $k$ is $\mathbf{P}_k$.
  • $\boldsymbol{y}$: Response of the elastic net, one entry per observation.
  • $\boldsymbol{z}$: Coefficients of the elastic net, one per column of $\mathbf{P}$. Coefficient $k$ is $z_k$.
  • $\mathbf{V}$: Diagonal matrix of the weights of the observations of the elastic net, the identity when the fit has no observation weights. Its diagonal holds the frequency weights of glm_weights.
  • $\mathcal{K}$: Penalised columns of the elastic net, every column of $\mathbf{P}$ but the intercept column that an estimator names.
  • $\gamma$: Regularisation strength of the elastic net.
  • $\vartheta$: Share of the $L_1$ penalty in the elastic net, the theta field of ElasticNet.
  • $\hat{\boldsymbol{b}}$: Coefficients of the fit on the columns of $\mathbf{X}$, which StatsAPI.coef returns.

Fields

  • theta: The share of the $L_1$ penalty, 1 for the lasso and 0 for the ridge.
  • strength: The rule that sets the regularisation strength.
  • sdiv: The rule that adds the term of a strength that reads the response to the effective degrees of freedom of the fit, or nothing to leave the term out.
  • ve: Variance estimator. It standardises each penalised column, or nothing to fit the columns as they are.
  • iters: Maximum number of coordinate-descent sweeps. Each sweep ends with the exact solve on its sign pattern.
  • tol: Tolerance on the largest change of a coefficient over one sweep. The sweeps stop below it when no exact solve has given the optimum.
  • w: Observation weights vector observations × 1.
  • ic: The index of the column of ones that the estimator names with InterceptColumn, or nothing. The fit does not penalise or standardise that column.

Constructors

ElasticNet(;    theta::Real = 0.99,    strength::AbstractPenaltyStrength = MiddleOfPath(),    sdiv::Option{<:AbstractStrengthDivergence} = nothing,    ve::Option{<:AbstractVarianceEstimator} = nothing,    iters::Integer = 10_000,    tol::Real = 1e-10,    w::Option{<:ObsWeights} = nothing,    ic::Option{<:Integer} = nothing) -> ElasticNet

Keywords correspond to the struct's fields. The constructor passes w to ve through factory.

Validation

  • 0 <= theta <= 1. A DomainError is thrown otherwise.
  • theta > 0 when strength is a MiddleOfPath. A DomainError is thrown otherwise, because a ridge alone has no strength above which every coefficient is zero.
  • iters >= 1. A DomainError is thrown otherwise.
  • tol > 0. A DomainError is thrown otherwise.
  • If w is a vector, !isempty(w). An IsEmptyError is thrown otherwise.
  • If ic is not nothing, ic >= 1. A DomainError is thrown otherwise.

Examples

julia> ElasticNet()ElasticNet     theta ┼ Float64: 0.99  strength ┼ MiddleOfPath           │   ratio ┴ Float64: 0.001      sdiv ┼ nothing        ve ┼ nothing     iters ┼ Int64: 10000       tol ┼ Float64: 1.0e-10         w ┼ nothing        ic ┴ nothingjulia> X = [1.0 0.5; 1.0 -0.2; 1.0 0.9; 1.0 0.1];julia> y = [1.2, 0.4, 1.9, 0.6];julia> tgt = factory(ElasticNet(; strength = FixedStrength(; gamma = 1e6)), InterceptColumn(1));julia> m = PortfolioOptimisers.StatsAPI.fit(tgt, X, y);julia> PortfolioOptimisers.StatsAPI.coef(m) ≈ [sum(y) / 4, 0.0]true

Related

References

  • [108] J. Friedman, T. Hastie and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1–22 (2010). Section 2.5, Equations (1), (2) and (10).
  • [110] H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67, 301–320 (2005). Equation (3).
  • [101] Z.-R. Lai, P.-Y. Yang, X. Wu and L. Fang. A kernel-based trend pattern tracking system for portfolio optimization. Data Mining and Knowledge Discovery 32, 1708–1734 (2018). Equation (15).
  • [109] R. J. Tibshirani and J. Taylor. Degrees of freedom in lasso problems. The Annals of Statistics 40, 1198–1232 (2012). Remark (The elastic net) after Equation (28).
source
PortfolioOptimisers.factory — Method
factory(tgt::ElasticNet, w::ObsWeights) -> ElasticNet

Return a new ElasticNet whose fit weighs the observations by w.

Algorithm

  1. Rebuild tgt with w in place of tgt.w and every other field unchanged, through rebuild_struct. The keyword constructor that it calls passes w to tgt.ve through factory, so the standardisation reads the same weights.

Arguments

  • tgt: The regression target.
  • w: Observation weights vector observations × 1.

Returns

  • tgt::ElasticNet: A new regression target that holds the weights.

Related

source
PortfolioOptimisers.factory — Method
factory(tgt::ElasticNet, ic::InterceptColumn) -> ElasticNet

Return a new ElasticNet whose fit leaves the column of ones that an estimator names out of the penalty.

Algorithm

  1. Rebuild tgt with ic.k in place of tgt.ic and every other field unchanged, through rebuild_struct.

Arguments

  • tgt: The regression target.
  • ic: The intercept column.

Returns

  • tgt::ElasticNet: A new regression target that does not penalise or standardise column ic.k.

Related

source
StatsAPI.fit — Method
StatsAPI.fit(tgt::ElasticNet, X::MatNum, y::VecNum) -> ElasticNetFit

Fit an ElasticNet to the design X and the response y by cyclic coordinate descent.

ElasticNet states the objective. The fit solves it at one strength, which the strength rule gives. A design with few columns does not need the rest of the path as warm starts.

Algorithm

  1. Build the standardised design P, the weighted design Q, the scale s and the weights v with elastic_net_design.
  2. Mark the penalised columns in pen: every column but tgt.ic. Compute nrm2, the weighted squared norm of each column.
  3. Start from the coefficients z at zero and the residual r equal to y. With an intercept column, fit that column alone first, and take its residual as r.
  4. Compute g, the product of each penalised column of Q with r, the strength lam with penalty_strength, and the ridge term ridge of each column, zero at the intercept column.
  5. When every entry of g is zero, the start point is the optimum. Otherwise compute the threshold a of each column, zero at the intercept column, and run elastic_net_sweeps from the start point.
  6. Compute the counts of the fit with elastic_net_counts.
  7. Divide the coefficients by s, and return them with lam and the counts in an ElasticNetFit.

Arguments

  • tgt: The regression target.
  • X: The design matrix observations × regressors.
  • y: The response vector.

Returns

  • m::ElasticNetFit: The fitted model.

Related

source
PortfolioOptimisers.obs_weights_view — Method
obs_weights_view(tgt::ElasticNet, i) -> ElasticNet

Return an ElasticNet whose observation weights, and the weights of its variance estimator, are cut to the observations i.

The target is not @propagatable, so it needs this method of obs_weights_view. A cut of the observations of a fit, such as the drop of a Shared Series gap, then cuts the weights of the regression too, see shared_series_estimator.

Algorithm

  1. Index tgt.w with obs_weights_index.
  2. Cut the variance estimator tgt.ve with obs_weights_view.
  3. Rebuild the target with the two cut fields through rebuild_struct. The other fields keep their values.

Arguments

  • tgt: The regression target.
  • i: The observations to keep: a mask of the full sample, or their positions.

Validation

Returns

  • tgt::ElasticNet: The target at the kept observations.

Related

source

References

[101]
Z.-R. Lai, P.-Y. Yang, X. Wu and L. Fang. A kernel-based trend pattern tracking system for portfolio optimization. Data Mining and Knowledge Discovery 32, 1708–1734 (2018).
[108]
J. Friedman, T. Hastie and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33, 1–22 (2010).
[109]
R. J. Tibshirani and J. Taylor. Degrees of freedom in lasso problems. The Annals of Statistics 40, 1198–1232 (2012), arXiv:1111.0653.
[110]
H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67, 301–320 (2005).