Log-wealth regret against a comparator

struct LogWealthRegretResult{__T_regret, __T_regret_per_period, __T_difference, __T_cumulative, __T_variance, __T_z, __T_p, __T_lags, __T_n_periods, __T_wealth_a, __T_wealth_b, __T_path_length} <: AbstractResult

Holds the log-wealth regret of a strategy against a comparator, and the test of its per-row difference.

log_wealth_regret returns it, over one sequence of rows. The regret is the comparator's log terminal wealth less the strategy's, so it is positive when the comparator wins. The per-row difference carries a Newey–West test of equal expected log growth, in the shape of CovarianceForecastComparisonResult. A negative regret is common and is not a defect: a causal strategy that reads the market's structure can beat a constant portfolio, and a Hindsight Comparator is best only over the class it was chosen from.

Two fields state the comparator's side of a dynamic-regret bound. path_length is $P_T$, the summed Euclidean distance between the comparator's consecutive targets: zero for a constant comparator, and larger with every switch that a HindsightSplit comparator makes. cumulative is the running regret.

Fields

• regret: Log-wealth regret, $\sum_t \log(1 + r_{b,t}) - \sum_t \log(1 + r_{a,t})$: the comparator's log terminal wealth less the strategy's, positive when the comparator wins.

• regret_per_period: The regret divided by the number of periods, which is the mean of difference.

• difference: Per-row series $\log(1 + r_{b,t}) - \log(1 + r_{a,t})$, the comparator's log growth less the strategy's at every period.

• cumulative: Running regret, the cumulative sum of difference: its last entry is regret.

• variance: Newey–West long-run variance of difference, at lags lags.

• z: Diebold–Mariano–West statistic of difference, asymptotically standard normal under equal expected log growth when the comparator did not read the rows.

• p: Two-sided p-value of the statistic against the standard normal. It is NaN when the two series coincide, because their difference has no variance, and zero when the difference is a non-zero constant.

• lags: Number of lags of the Bartlett kernel.

• n_periods: Number of periods compared.

• wealth_a: Terminal wealth of the strategy from a unit start, $\prod_t (1 + r_{a,t})$.

• wealth_b: Terminal wealth of the comparator from a unit start, $\prod_t (1 + r_{b,t})$.

• path_length: Path length of the comparator, $P_T = \sum_{t \geq 2} \lVert u_t - u_{t-1} \rVert_2$ over its consecutive targets, read off the comparator's folds and embedded by asset name on the union of their universes. NaN when the comparator is one prediction result, which holds one target and no path.

Constructors

LogWealthRegretResult(    regret, regret_per_period, difference, cumulative, variance, z, p, lags,    n_periods, wealth_a, wealth_b, path_length) -> LogWealthRegretResult

Arguments correspond to the struct's fields, in the order they are declared. The type is a Result: log_wealth_regret builds it and a caller reads it. It has no keyword constructor and validates nothing of its own.

Related

• log_wealth_regret
• CovarianceForecastComparisonResult
• BestConstantRebalancedPortfolio
• HindsightSplit
• BudgetedHindsightPath

log_wealth_regret(a::PredRes_MultiPredRes, b::PredRes_MultiPredRes; lags::Integer = 0) -> LogWealthRegretResult

The log-wealth regret of strategy a against comparator b over one and the same sequence of rows.

Regret is defined over one sequence, so the verb refuses two prediction results that do not carry the same timestamps. Each series is read as it was scored: net of the fee the fold settled, and drifted where a Weight Drift ran. The comparator's fee policy is therefore the caller's, and a fee-free comparator against a fee-paying strategy measures the fee as regret.

The comparator is any prediction result over the rows. It can be the same estimator run causally through cross_val_predict with the strategy's cv. It can also be a Hindsight Comparator, an estimator fit on the rows it is scored on and predicted over them, predict(optimise(est, rd_test), rd_test). Two Hindsight Comparators are common:

• The best constant rebalanced portfolio in hindsight: BestConstantRebalancedPortfolio with no solver, or MeanRisk under LogarithmicReturn and MaximumReturn with a solver, which is exact under any bound.
• The best stock in hindsight: a ScoreSelector under RankRule(; best = 1) and MeanReturn(; flag = true), composed with EqualWeighted in a Pipeline.

No scorer and no summary column ship for regret. A hyperparameter search ranks on MeanReturn(; flag = true), the log wealth per period, which orders candidates as the regret against any fixed comparator orders them. The performance summary reads one series and holds no comparator.

Mathematical definition

\[\begin{align} \delta_t &= \log\left(1 + r_{b,t}\right) - \log\left(1 + r_{a,t}\right)\,, \\ R &= \sum_{t=1}^{T} \delta_t\,, \quad \bar{\delta} = \frac{R}{T}\,, \\ \hat{\omega}^2 &= \hat{\gamma}_0 + 2 \sum_{k=1}^{\ell} \left(1 - \frac{k}{\ell + 1}\right) \hat{\gamma}_k\,, \\ z &= \frac{\sqrt{T}\, \bar{\delta}}{\sqrt{\hat{\omega}^2}}\,, \\ P_T &= \sum_{t=2}^{T} \lVert \boldsymbol{u}_t - \boldsymbol{u}_{t-1} \rVert_2\,. \end{align}\]

Where:

• $r_{a,t}$, $r_{b,t}$: Realised return of the strategy and of the comparator at period $t$, as scored.
• $\delta_t$: Per-row difference in log growth.
• $R$: Log-wealth regret, positive when the comparator wins.
• $\bar{\delta}$: Regret per period.
• $T$: Number of periods.
• $\hat{\omega}^2$: Long-run variance of $\delta_t$, the Bartlett-kernel estimate with $\ell$ lags.
• $\hat{\gamma}_k$: Sample autocovariance of $\delta_t$ at lag $k$, about $\bar{\delta}$ and divided by $T$.
• $\ell$: Number of lags, $0$ by default because the rows of a walk-forward at test_size = 1 do not overlap.
• $z$: Diebold–Mariano–West statistic.
• $P_T = \sum_{t=2}^{T} \lVert \boldsymbol{u}_t - \boldsymbol{u}_{t-1} \rVert_2$: Path length of a comparator sequence $\boldsymbol{u}_1, \ldots, \boldsymbol{u}_T$ in the feasible set.
• $\boldsymbol{u}_t$: Comparator's target on fold $t$, embedded by asset name on the union of the folds' universes.

Under equal expected log growth, $z$ is asymptotically standard normal, and the two-sided p reads it against that law. The test is exact only for a comparator that did not read the rows it is scored on. Against a Hindsight Comparator it is optimistic by construction: the comparator was chosen on the sequence that the difference is tested over, so p overstates the evidence that the comparator is better.

Three degenerate series have a stated answer:

• Two identical series have $\bar{\delta} = 0$ and $\hat{\omega}^2 = 0$, so $z$ and p are NaN.
• A difference that is a non-zero constant has $\hat{\omega}^2 = 0$, so $z$ is infinite and p is zero. In floating point $\hat{\omega}^2$ is round-off of about $10^{-33}$, so $z$ is a very large finite number.
• A return of $-1$ is a total loss. Its log growth is $-\infty$, so $R$ is infinite, and $z$ and p are NaN.

Dynamic regret is regret against a comparator that moves. Its bounds are stated in $P_T$, so $R$ and $P_T$ are read together: a static comparator has $P_T = 0$ and its regret is the static regret, and a moving comparator buys its lower regret with a path that the bound charges for. $P_T$ is NaN when the comparator is one prediction result, which holds one target and no path. The literature states dynamic regret against three comparators, and each is the same estimator through its own scheme. On the three-row example $\boldsymbol{x}_1 = (1.2, 0.98)$, $\boldsymbol{x}_2 = (0.9, 1.1)$, $\boldsymbol{x}_3 = (1.3, 0.7)$ of price relatives:

• Static: the best constant rebalanced portfolio over all three rows, predict(optimise(est, rd), rd), is all-in on the first asset. Its wealth is $1.2 \cdot 0.9 \cdot 1.3 = 1.404$, and $P_T = 0$. est is BestConstantRebalancedPortfolio, or MeanRisk under LogarithmicReturn and MaximumReturn.
• Be-the-leader: cross_val_predict(est, rd, HindsightSplit()) plays on row $t$ the best constant rebalanced portfolio over the rows through $t$. The targets are $\boldsymbol{u}_1 = (1, 0)$, $\boldsymbol{u}_2 = (23/44, 21/44) \approx (0.523, 0.477)$ and $\boldsymbol{u}_3 = (1, 0)$. The wealth is $1.2 \cdot 0.995 \cdot 1.3 \approx 1.553$, and $P_T = 2 \sqrt{2} \cdot 21/44 \approx 1.350$. The first fold trains on one row, so MeanRisk needs a prior that fits one row, which HindsightSplit states.
• Per-period minimiser: the top-1 ScoreSelector under MeanReturn(; flag = true), composed with EqualWeighted through HindsightSplit(; prefix = false), is one-hot on each row's best asset. Its wealth is $1.2 \cdot 1.1 \cdot 1.3 = 1.716$, and $P_T = 2 \sqrt{2}$.

The bound also reads the other way. The regret at a given budget $L$ is the gap to the best sequence whose path length is at most $L$. Neither be-the-leader nor the per-period minimiser is that sequence. BudgetedHindsightPath fits it, and its prediction result has one fold per row for this verb to read.

Algorithm

1. Read the strategy's series and timestamps through regret_series, giving ra and tsa, and the comparator's, giving rb and tsb.
2. Check that the two series share their timestamps and their length, giving T.
3. Check lags, and check that no return is below $-1$.
4. Take the log growth of each series, giving la and lb, and their difference d.
5. Sum d cumulatively, giving cumulative. Its last entry is regret, and regret / T is md.
6. Estimate the long-run variance of d through newey_west_variance, giving v.
7. Form the statistic z from md and v, and its two-sided p-value p.
8. Read the comparator's path length through comparator_path_length.

Arguments

• a: The strategy's prediction result.
• b: The comparator's prediction result.
• lags: Number of lags of the Bartlett kernel.

Validation

• a and b carry the same timestamps and the same number of rows. An ArgumentError is thrown otherwise: two runs over different rows compare nothing.
• 0 <= lags < n_periods. A DomainError is thrown otherwise.
• No return of either series is below $-1$. A DomainError is thrown otherwise, because a negative wealth has no log. A NaN return passes, and its NaN reaches the regret.

Returns

• reg::LogWealthRegretResult: The regret, its per-period form, the per-row difference, its running sum and its test, the two terminal wealths, and the comparator's path length.

Related

• LogWealthRegretResult
• HindsightSplit
• BudgetedHindsightPath
• BestConstantRebalancedPortfolio
• covariance_forecast_compare
• newey_west_variance
• performance_summary
• cross_val_predict

References

• [151] B. Li and S. C. H. Hoi. Online portfolio selection: A survey. ACM Computing Surveys 46(3), 1–36 (2014).

struct BudgetedHindsightPath{__T_L, __T_p, __T_wb, __T_sets, __T_strict, __T_slv, __T_fb} <: OptimisationEstimator

Fits the path of per-row allocations with the largest log wealth whose summed step length is at most L.

Dynamic regret at a budget is the gap to this path, the comparator of Zinkevich's (2003) Definition 7. Neither per-row comparator that a HindsightSplit builds is this path: be-the-leader is one point at its own path length, and the per-period minimiser is the limit with no budget. The fit is one concave programme over the whole panel. The Lagrangian form, which puts a penalty on the path length in place of the budget, is the same programme with one term moved, and the library does not build it.

The estimator follows the Hindsight Comparator rule of log_wealth_regret, as every other comparator does. optimise returns a BudgetedHindsightPathResult, which holds the path. predict over the same rows returns one fold per row, and it refuses any other rows. So predict(optimise(est, rd_test), rd_test) is a prediction result that the regret verb reads unchanged, and the verb reports its Euclidean path length beside the regret.

Each row's universe is the Coverage Universe of its one-row window, as coverage_mask derives it: an asset is in the row when its return is finite and the Asset Panel's active mask is true there. A static panel, or no panel, reads finiteness alone. A HindsightSplit(; prefix = false) fold fits on the same universe, so the two per-row comparators agree on what a row can hold. The result carries the masks, so predict views each row at its own mask and names no Held Gap. A row with no asset in its universe is refused, because its budget of one cannot be met.

The budget is stated in the norm of order p, but LogWealthRegretResult reports the path length in the Euclidean norm. The two numbers agree at p = 2 alone. A tight budget on a long panel puts most step cones at their apex, where an interior-point solver can stall short of its tolerance and report insufficient progress. A first-order solver solves such a programme, so on a panel of many rows the robust slv is a solver vector with a first-order solver as the fallback. The fit reads the rows it is scored on, so the test of log_wealth_regret against it is optimistic by construction.

Mathematical definition

\[\begin{align} \underset{\boldsymbol{u}_1, \ldots, \boldsymbol{u}_T}{\max} &\quad \sum_{t=1}^{T} \log \sum_{i \in \mathcal{M}_t} u_{t,i}\, x_{t,i} \\ \textrm{s.t.} &\quad \sum_{i \in \mathcal{M}_t} u_{t,i} = 1\,, \quad l_i \leq u_{t,i} \leq h_i \;\; \forall i \in \mathcal{M}_t\,, \quad u_{t,i} = 0 \;\; \forall i \notin \mathcal{M}_t\,, \\ &\quad \sum_{t=2}^{T} \lVert \boldsymbol{u}_t - \boldsymbol{u}_{t-1} \rVert_p \leq L\,. \end{align}\]

Where:

• $\boldsymbol{u}_t$: Comparator's allocation on row $t$, with entries $u_{t,i}$.
• $\boldsymbol{x}_t$: Price relative vector of period $t$, each asset's price over its price one period earlier.
• $x_{t,i}$: Entry $i$ of $\boldsymbol{x}_t$, one plus the return of asset $i$ on row $t$.
• $\mathcal{M}_t$: Coverage Universe of row $t$, the assets that the row can hold.
• $l_i$, $h_i$: Resolved lower and upper weight bounds of asset $i$, zero and one by default.
• $L$: Path-length budget.
• $p$: Norm order.
• $T$: Number of observations.

Three consequences follow from the programme:

• At $L = 0$ the path is constant. It holds only the assets that every row covers, and it is the best constant rebalanced portfolio in hindsight over them, as BestConstantRebalancedPortfolio fits it under the same bounds. When no asset is in every row's universe, the programme is infeasible.
• When $L$ is at least the path length of the per-period minimiser, the budget is slack. Under the default bounds the answer is then that minimiser, one-hot on each row's best asset, when each row has one best asset.
• An asset that delists inside the panel is outside every later row's universe. The path therefore sells it on its last row at any budget, and the forced step counts towards the path length like any other step.

Between the two ends the path spends the budget where a switch buys the most log wealth, and the regret against it is the regret at that budget.

Fields

• L: The path-length budget, 0 <= L.

• p: The norm order of the budget, 1 <= p.

• wb: Weight bounds.

• sets: Sets used to map estimator values to assets.

• strict: Whether a constraint that names an asset or a group that sets does not hold raises an error. When it is false, the constraint generator warns and skips the entry.

• slv: Solver or vector of solvers.

• fb: Fallback estimator, another BudgetedHindsightPath that optimise runs when the fit fails, or nothing.

Constructors

BudgetedHindsightPath(;    L::Number,    slv::Slv_VecSlv,    p::Number = 2,    wb::Option{<:WbE_Wb} = WeightBounds(),    sets::Option{<:UniverseSets} = nothing,    strict::Bool = false,    fb::Option{<:BudgetedHindsightPath} = nothing) -> BudgetedHindsightPath

Keywords correspond to the struct's fields. L and slv are required: a budget has no default, and the fit is a programme.

Validation

• 0 <= L and 1 <= p. A DomainError is thrown otherwise.
• A vector slv is not empty. An IsEmptyError is thrown otherwise.
• If wb is a WeightBoundsEstimator: !isnothing(sets). An IsNothingError is thrown otherwise.

View parameters

When port_opt_view is called on this type, the following @vprop-tagged fields are automatically subset to the selected indices:

• wb: Recursively viewed via port_opt_view.
• sets: Sliced to the selected indices via port_opt_view.

Examples

julia> BudgetedHindsightPath(; L = 5 * sqrt(2), slv = Solver(; solver = nothing))BudgetedHindsightPath       L ┼ Float64: 7.0710678118654755       p ┼ Int64: 2      wb ┼ WeightBounds         │   lb ┼ Float64: 0.0         │   ub ┴ Float64: 1.0    sets ┼ nothing  strict ┼ Bool: false     slv ┼ Solver         │          name ┼ String: ""         │        solver ┼ nothing         │      settings ┼ nothing         │     check_sol ┼ @NamedTuple{}: NamedTuple()         │   add_bridges ┴ Bool: true      fb ┴ nothing

Related

• BudgetedHindsightPathResult
• optimise
• predict
• log_wealth_regret
• HindsightSplit
• BestConstantRebalancedPortfolio
• port_opt_view

References

• [152] M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In: Proceedings of the 20th International Conference on Machine Learning, 928–936 (2003).

struct BudgetedHindsightPathResult{__T_w, __T_wb, __T_retcode, __T_imsk, __T_nx, __T_ts, __T_fb} <: OptimisationResult

Holds a budgeted hindsight path, one allocation per row of the panel it was solved over.

BudgetedHindsightPath fits it. It also carries the rows' Investable Masks, the resolved bounds, the rows' names and clock, and the return code. The path is bound to its rows, so predict answers one fold per row over those rows alone, and refuses a returns result whose names, clock or row count differ. A fit that no solver solved carries NaN on every investable entry, and an OptimisationFailure that names the trials.

Fields

• w: The path, rows × assets, one allocation per row on the full universe, with an exact zero at every non-investable entry and NaN on a failed fit.

• wb: The resolved weight bounds the path was solved on.

• retcode: Optimisation return code.

• imsk: The rows' Investable Masks, rows × assets, each row the Coverage Universe of its one-row window, or nothing when every row covers every asset.

• nx: The asset names of the panel the path was solved over, or nothing when it carried none.

• ts: The clock of the panel the path was solved over, or nothing when it carried none.

• fb: The fallback chain optimise walked to reach this result, or nothing when the first fit answered.

Constructors

BudgetedHindsightPathResult(;    w::MatNum,    wb::WeightBounds,    retcode::OptimisationReturnCode,    imsk::Option{<:BitMatrix} = nothing,    nx::Option{<:AbstractVector} = nothing,    ts::Option{<:AbstractVector} = nothing,    fb::Option{<:FbChain} = nothing) -> BudgetedHindsightPathResult

Keywords correspond to the struct's fields. The type is a Result: optimise builds it and a caller reads it. It validates nothing of its own.

Related

• BudgetedHindsightPath
• predict
• MultiPeriodPredictionResult
• log_wealth_regret

factory(res::BudgetedHindsightPathResult, fb::Option{<:FbChain})

Rebuilds a budgeted path result with the fallback chain fb that optimise walked, and keeps every other field.

Arguments

• res: The budgeted path result.
• fb: The fallback chain, or nothing.

Returns

• res::BudgetedHindsightPathResult: A new result with fb in place of the old chain.

Related

• BudgetedHindsightPathResult
• FbChain

_optimise(est::BudgetedHindsightPath, rd::ReturnsResult; kwargs...) -> BudgetedHindsightPathResult

Solves the budgeted path programme of BudgetedHindsightPath over the rows of rd.

optimise is the door, and it walks est.fb on a failure. The value of a variable fixed at zero is the solver's zero, so the fit writes an exact zero at every entry outside a row's mask.

Algorithm

1. Check that rd.X is not nothing.
2. Add one to the returns, giving the price relatives X, of size T × N.
3. Derive the rows' Investable Masks through path_row_masks, giving imsk, and msk, which is all true when imsk is nothing.
4. Resolve the weight bounds through weight_bounds_constraints, giving wb.
5. Make a bare model with the allocation variables u and the log-wealth variables z.
6. Write each row through path_row_constraints!, and the budget through path_budget_constraint!.
7. Set the objective, and solve the model through optimise_JuMP_model!, giving res.
8. Read the path W from u and OptimisationSuccess on a success. On a failure, fill W with NaN and make an OptimisationFailure that names the trials.
9. Set every entry of W outside msk to zero.

JuMP formulation

Variables

• u: created, an anonymous T × N matrix of allocations, one row per period.
• z: created, an anonymous vector of $T$ log-wealth variables, one per period.

Constraints

The rows of path_row_constraints!, one call per period, which give $z_t \leq \log \sum_{i \in \mathcal{M}_t} u_{t,i}\, x_{t,i}$, and the rows of path_budget_constraint!. The function registers no row itself.

Objective

• Max: $\sum_{t=1}^{T} z_t$, the log wealth of the path.

Where:

• $z_t$: Log-wealth variable of period $t$.
• $u_{t,i}$: Allocation of asset $i$ in period $t$.
• $x_{t,i}$: Price relative of asset $i$ in period $t$, one plus its return.
• $\mathcal{M}_t$: Coverage Universe of period $t$.
• $T$: Number of observations.

Arguments

• est: The estimator.
• rd: The returns result to use. Its returns matrix is the panel the path is solved over.
• kwargs: Ignored.

Validation

• !isnothing(rd.X). An IsNothingError is thrown otherwise.
• Every row has at least one asset in its Coverage Universe, through path_row_masks.

Returns

• res::BudgetedHindsightPathResult: The path, or the failure.

Related

• BudgetedHindsightPath
• BudgetedHindsightPathResult
• path_row_masks
• path_row_constraints!
• path_budget_constraint!
• optimise_JuMP_model!

predict(res::BudgetedHindsightPathResult, rd::ReturnsResult) -> MultiPeriodPredictionResult

The prediction result of a budgeted path over the rows it was solved on: one fold per row, each a PredictionResult whose NaiveOptimisationResult holds that row's allocation on the resolved bounds, with the row's Investable Mask when the row has a missing return, predicted over that row alone.

The path is bound to its rows, so rd must be the panel the path was solved over: the same names, the same clock and the same row count. predict(optimise(est, rd_test), rd_test) is then the Hindsight Comparator rule of log_wealth_regret, and the regret verb reads the folds unchanged.

Arguments

• res: The fit.
• rd: The returns result to use. The panel the path was solved over.

Validation

• rd carries the names, the clock and the row count of res. An ArgumentError is thrown otherwise: a path over other rows is another fit.

Returns

• pred::MultiPeriodPredictionResult: One fold per row.

Related

• BudgetedHindsightPath
• BudgetedHindsightPathResult
• log_wealth_regret
• MultiPeriodPredictionResult