Online Portfolio Selection
OnlinePortfolioSelection runs an online portfolio selection rule. It is a naive optimiser, so it needs a solver only when its set of allowed weights does. optimise(opt, rd) runs the rule from the start weights over every row of the returns in order, and returns the weights for the next period. partial_fit! updates the rule with a block of new rows, one row at a time. optimise(opt) with no data returns the current weights of the rule, and runs no batch fit.
The pages of this group list the rules in the order the library added them. Li and Hoi [5] sort the rules into five families: benchmarks, follow the winner, follow the loser, pattern matching and meta-learning. The capability catalogue and the roster by group of the user guide list the rules by these families. The user guide also runs the rules on two synthetic markets and prints a table of their regret.
PortfolioOptimisers.OnlinePortfolioSelection — Type
struct OnlinePortfolioSelection{__T_alg, __T_set, __T_w0, __T_fees, __T_fb, __T_x_src, __T_strict, __T_cache} <: NaiveOptimisationEstimatorUpdates its allocation from each realised price relative through the Online Selection Rule on alg.
The head is a naive optimiser. It fits no moment and solves no programme of its own. Every rule of the family is one recursion, which rebalances the allocation to its target at the start of every period, and the head does all the work that the rules share. # Mathematical definition states the recursion and the wealth it earns.
The batch verb is the Causal Pass. optimise(opt, rd) starts from the Start Allocation w0, applies the Online Update to every row of rd in order, and returns the Next-Period Allocation. After rows 1:T that is the portfolio for period T + 1, so the pass takes one more update than a backtest driver that stops at the last row it holds. The pass rebalances at every row and pays no fee. predict(res, rd) on its Result charges the head's fees, with no drift.
The online verbs run the same recursion. partial_fit!(opt, rd) folds each row into the state as the Block Step. A fold of k rows is k single-row updates, so a walk-forward with test_size = k holds the Next-Period Allocation of the end of the block for k periods. optimise(opt) with no data is the Recursion Result. It wraps the allocation of the state in a Result and runs no batch path. So optimise(opt) after a fold of rows 1:t equals optimise(opt, rd[1:t]) exactly, at every test_size. The held path of a walk-forward differs from the batch pass in two ways. The target set at the start of a block drifts over the block, and the Block Step sets the cadence of the trades.
The Start Allocation is w0, nothing by default. When it is absent, the recursion starts at 1/N over the whole pinned universe, and the unlisted names are included. If k of N assets are unlisted at the first row, the recursion puts k/N of its weight on legs that see x = 1, as cash does, until the rule moves that weight. optimise(opt) with no data renormalises over the listed legs, so the fund holds the listed legs alone. But the fund is not always the recursion that starts over the listed names alone, because the parked legs enter the gross return ⟨w, x⟩ that a rule reads. Buy-and-hold scales each leg by its own price relative, so its fund is the same. The exponentiated-gradient step divides by the gross return, so its fund is different. A ProgrammeAllocationSet does not remove the difference. Its projection gives a zero weight to a parked leg that the prior of the set cannot price, as to every asset outside the Investable Mask of the step. But the gradient of that step already read the parked leg in the gross return, so the weights of the listed legs keep the difference, also for an asset that never lists. A caller who wants the recursion to be the fund from row one gives w0 over the listed assets, with zeros elsewhere. The rule's step then decides what a zero does. The exponentiated-gradient step keeps a zero at zero, so that asset never gets weight. A Euclidean step that moves no weight onto the asset keeps the zero exact too, because project_simplex only renormalises a raw step that is on the simplex up to rounding. Buy-and-hold is such a step. A Euclidean step that moves weight onto the asset, as the passive-aggressive step does, gives it weight, and a gap at that asset is then a Held Gap.
A given w0 is over the pinned names, and the head pins and views it with them. The first step projects it once onto the Allocation Set in the rule's geometry, as it projects the uniform start, so a start outside the set becomes feasible and is never refused. A set that reads the head's rows has no constraints before the first row, so it holds the start as given until the first update. The start reaches the first Online Update as w. A rule that steps from w continues from it. A rule whose next allocation is not a step from w replaces it at that update. The constant rebalanced portfolio returns its own w, an expert mixture returns the mix of its experts, and the Newton step solves its Gram. Such a rule can still carry w0 forward. The gradient of the Newton step reads w, and an expert that steps from w starts at w0. Under every rule, the fund holds w0 for exactly one period.
The Online Update reads the recursion's own allocation, and no flag changes that. The previous weights of the loop reach the head through factory, into fees and fb, and the recursion never reads them as its allocation. A schedule in a field of a leader in the rule tree can also read them from ctx.w_prev, to select the configuration of the leader at the fold. The head then needs_previous_weights, so a walk-forward runs its folds in sequence, see schedule_reads_previous_weights. The fund's held book is the one base against which a turnover fee on this family measures a trade. So the online arm of the fold loop refuses a head whose fees carry a tn term when the Previous-Weights Source of the walk-forward is nothing. It throws an ArgumentError before any fold, and pws = DriftedWeights() is the whole configuration. A head with no tn fee is not checked.
Under x_src = ExcessSource() the head selects in the Excess Frame. The rule's step reads the price relative of the return in excess of the benchmark, x = 1 + r - b. Here b is the benchmark return of the row, one value that every asset shares for a vector benchmark, and one value per asset for a matrix benchmark. Every reader of the rows inside a rule reads the same frame through head_frame_returns: the refit and the fold of a forecaster, the scale of a ForecastReversion, the window of an AntiCorrelation and the row selector of a FollowTheLeader. The rows buffer keeps the raw returns and the benchmark, so the prior of a Risk Loss, the prior and each graph or exposure slot of a programme set, and a leader read the raw rows and state their own source. The source of the step, the readers of a Risk Loss and the readers of a programme set form one group. When one of them states ExcessSource() and another states no source and reads the raw rows, the head warns once, see warn_raw_readers. A leader and the fallback are heads with their own groups. A gap in the benchmark is a gap in the frame, which the step reads as one and the head reports as a Held Gap. predict(res, rd) reports the raw wealth of the fund, because the fund holds the assets. Returns data with no benchmark is refused with an ArgumentError.
A non-finite return is kept, and read in two ways. The rule's step reads x = 1 there, as if the leg held cash. At an asset that the panel marks inactive, or at one to which the recursion gives no weight, this is silent. At an active asset with a non-zero weight it is a Held Gap, which warns by default and refuses under strict. The rows buffer keeps the cell as NaN, with the row's active mask beside it. Every statistic over the rows reads them as the batch verb reads a Returns Result, and reduces to its own Coverage Universe. Such a statistic is the mean of a forecaster, the prior of a Risk Loss or of a programme set, or the re-solve of a leader. A plain estimator drops an asset with a gap anywhere in its window. A mask-aware estimator answers the asset from the rows it has. A kernel over price relatives reads the gap as one.
The head never forces a zero into the recursion's allocation, and a relisted asset comes back at the recursion's own weight. A programme set that fits on the rows writes a zero at a leg that its prior cannot price, as a batch head does. It admits the leg again when the prior can price it. Under a time-varying panel, the Investable Mask of the Recursion Result is the active mask of the last folded row. optimise(opt) with no data slices the full allocation to that mask and renormalises it, and the Result expands it back with a zero at every non-investable asset.
The Allocation Set on set is the one the Constrained Update projects onto. project projects the raw step of every rule onto it, in the rule's own Projection Geometry. The default BoundedAllocationSet is the simplex, and every projection onto it is closed form, so the default configuration solves nothing. A ProgrammeAllocationSet admits the full constraint vocabulary, and its projection is a programme. When a programme fails, the step is a Held Step. The projection returns the Price-Adjusted Allocation it was given, so the fund trades nothing that period, and the state of the rule still absorbs the row. The head warns once, with the row's timestamp. The retcode of the Recursion Result is an OptimisationSuccess that carries the HeldStep record of the last folded row, so a fallback chain never runs on a hold. The constructor refuses a negative lower bound under an entropic, Tsallis or log-barrier rule.
merge_states on the state and Online(head) each throw an ArgumentError. An update depends on the order of the rows, and the family never refits from a buffer.
Mathematical definition
\[\begin{align} \boldsymbol{w}_1 &= \underset{\boldsymbol{w} \in \mathcal{W}}{\arg\min} \; D_\Psi(\boldsymbol{w}, \boldsymbol{w}_0)\,, \\ \boldsymbol{w}_{t+1} &= \underset{\boldsymbol{w} \in \mathcal{W}}{\arg\min} \; D_\Psi(\boldsymbol{w}, \boldsymbol{q})\,, \\ x_{tj} &= \begin{cases} 1 + r_{tj} & \text{if } r_{tj} \text{ is finite}\,, \\ 1 & \text{otherwise}\,, \end{cases} \\ S_T &= \prod_{t=1}^{T} \langle \boldsymbol{w}_t, \boldsymbol{x}_t \rangle\,. \end{align}\]
Where:
- $\boldsymbol{w}_1$: Start Allocation, the allocation that the recursion starts from.
- $\boldsymbol{w}_0$: Given start, or $\boldsymbol{1}/N$ when none is given.
- $\boldsymbol{w}_t$: Iterate of the rule at period $t$, the allocation its update reads.
- $\boldsymbol{q}$: Raw step, the vector the rule's unconstrained step gives before the projection onto the Allocation Set. The rule makes it from $\boldsymbol{w}_t$ and $\boldsymbol{x}_t$.
- $\mathcal{W}$: Allocation Set.
- $D_\Psi$: Bregman divergence of $\Psi$.
- $\Psi$: Potential of the Projection Geometry.
- $\boldsymbol{x}_t$: Price relative vector of period $t$, each asset's price over its price one period earlier.
- $r_{tj}$: Return of asset $j$ at time $t$.
- $S_T$: Wealth of the fund after $T$ periods, from a wealth of one.
- $t$: Period index, counted from one.
- $T$: Number of observations.
- $N$: Number of assets.
The wealth $S_T$ is the wealth of Equation 1.1 of [7], earned in the protocol of Section 2 of [5]. The Next-Period Allocation after $T$ rows is $\boldsymbol{w}_{T+1}$. Both projections are one call of project, so a ProgrammeAllocationSet adds its Objective Penalty to both objectives.
Under x_src = ExcessSource(), $r_{tj}$ is the return of asset $j$ in excess of the benchmark, $r_{tj} - b_{tj}$, where $b_{tj}$ is the benchmark return of period $t$ for asset $j$. A vector benchmark gives every asset the same $b_{tj}$.
Fields
alg: The Online Selection Rule: the update, its parameters and its Projection Geometry.
set: The Allocation Set every allocation of the recursion lies in, or aTimeDependentschedule of one set per fold. Entryiis the complete set of foldi. The fold loop puts it in place before it folds the rows of foldi, so those rows step inside entryi, and the Recursion Result reports it.
w0: The Start Allocation over the pinned universe, ornothingfor1/Nover every pinned name.
fees: Fees estimator or result.
fb: Fallback result or estimator.
x_src: The source of the returns that the online step reads.nothingandDataSourceread the returns of the rows as they arrived.ExcessSourcereads the returns in excess of the benchmark,X - B, so the step reads the price relative1 + r - b. Each rule, and each reader of the rows inside a rule, reads the rows in the frame of the step, throughhead_frame_returns. The rows buffer keeps the raw returns and the benchmark, so a prior, a leader and the fallback state their own source. The step reads no prior, so it refusesPriorSource.
strict: Whether a constraint that names an asset or a group thatsetsdoes not hold raises an error. When it isfalse, the constraint generator warns and skips the entry. The flag reaches every constraint of the set, the fee andoptimise(opt)with no data, and the constraints that a programme set resolves on the rows at each step. It also governs a Held Gap, which warns when it isfalseand raises an error when it istrue.
cache: The Partial Fit State of the recursion,nothinguntilpartial_fit!writes one.factorycarries it unchanged andport_opt_viewslices it to the selected assets.
Constructors
OnlinePortfolioSelection(; alg::AbstractOnlinePortfolioSelectionAlgorithm = ExponentiatedGradient(), set::TD{<:AbstractAllocationSet} = BoundedAllocationSet(), w0::Option{<:AbstractVector} = nothing, fees::TD_Option{<:FeesE_Fees} = nothing, fb::TDO_Option{<:OptE_Opt} = nothing, x_src::Option{<:AbstractReturnsSource} = nothing, strict::Bool = false, cache::Option{<:OnlinePortfolioSelectionState} = nothing) -> OnlinePortfolioSelectionKeywords correspond to the struct's fields. set, fees and fb can hold a TimeDependent per-fold schedule. alg, w0, x_src and strict are static. A field of an optimiser that the rule tree on alg holds, such as the leader of a FollowTheLeader, can hold a schedule too. The fold loop resolves it at each fold, before the step of the fold reads the leader, see online_step_fold.
Validation
w0: non-empty and finite, when given. It need not lie in the set, because the first step projects it onto the set.- If
feesis aFeesEstimator:!isnothing(set.sets). A schedule onsetis held to it per entry. - Everything
assert_geometry_admits_setrefuses: a negative lower bound under an entropic, Tsallis or log-barrier rule. A schedule onsetis held to it per entry. - Everything
assert_rule_admits_setrefuses: a solver-free leader under a programme set, and a leader under a programme set whose fallback chain holds aNearOptimalCenteringunderUnconstrainedNearOptimalCentering, or aFactorRiskContributionbeside a semidefinite phylogeny of the set, seeassert_leader_keeps_programme_set. A schedule onsetis held to it per entry. fbschedules:bind !== :nearest.x_srcisnothing,DataSource()orExcessSource(), seeassert_returns_source. The step reads no prior, so it refusesPriorSource().- A reader that states no source and reads the raw returns data, beside a reader that states
x_src = ExcessSource(), logs one warning that names its path, seewarn_raw_readers. The head then builds as configured.
Propagated parameters
factory passes its arguments on to these fields:
fees: Recursively updated throughfactory, which threads the previous weights into a turnover fee.fb: Recursively updated throughfactory.
View parameters
port_opt_view takes the view of the fields with a per-asset axis at the selected indices: alg and set recursively, w0 sliced and renormalised, fees viewed at the indices, and cache through the state's own view.
Examples
julia> OnlinePortfolioSelection()OnlinePortfolioSelection alg ┼ MirrorDescent │ eta ┼ Float64: 0.05 │ proj ┼ EntropicProjection() │ alpha ┼ Int64: 0 │ obj ┼ LogWealth() │ grad ┼ nothing │ predictor ┴ nothing set ┼ BoundedAllocationSet │ wb ┼ WeightBounds │ │ lb ┼ Float64: 0.0 │ │ ub ┴ Float64: 1.0 │ sets ┴ nothing w0 ┼ nothing fees ┼ nothing fb ┼ nothing x_src ┼ nothing strict ┴ Bool: falseRelated
NaiveOptimisationEstimatorAbstractOnlinePortfolioSelectionAlgorithmOnlinePortfolioSelectionStateBoundedAllocationSetProgrammeAllocationSetHeldStepNaiveOptimisationResultoptimisepartial_fit!
References
PortfolioOptimisers.optimise — Method
optimise(opt::OnlinePortfolioSelection; kwargs...) -> OptimisationResultReturns the Recursion Result, the allocation of the state as a Result, and runs no batch path.
The method refuses a head that has taken no step with an ArgumentError. The Recursion Result fails in one way, when the allocation has no mass on the assets active at the last row. The fallback chain then runs as the batch verb runs it. The fallback reads out through its own optimise(fb), from the rows that step_fallback folded into it at every step of the head. So a fallback that needs rows answers from the same rows as in batch. The chain of the Result starts with the failed pair of the head, and the chain that the fallback walked follows it.
Algorithm
- Refuse a head whose
cacheisnothing. - Reset every schedule of the head to its fold-less value through
reset_time_dependent_estimator. - Read the state out through
online_selection_result, givingres. - Walk the fallback chain after
reswithread_out_fallback. It returnsreswhenressucceeded or when the head has no fallback.
Arguments
opt: The stepped head.kwargs: The keyword arguments ofoptimisewith no data on the fallback, passed on unchanged.
Validation
opt.cacheis notnothing. AnArgumentErroris thrown otherwise.
Returns
res::OptimisationResult: The Next-Period Allocation, or the Result of the fallback chain when the Recursion Result fails.
Related
PortfolioOptimisers.partial_fit! — Method
partial_fit!(opt::OnlinePortfolioSelection, rd::ReturnsResult)Folds the rows of a Returns Result into the head's recursion as the Block Step, and reads nothing out.
Each row of rd is one Online Update, in order, and the state never learns the cadence of the loop. A fold of k rows is k updates and builds no Recursion Result. After a fold of rows 1:t, optimise(opt) equals optimise(opt, rd[1:t]) exactly, because both take the same t single-row updates. The step also folds rd into the fallback through step_fallback, because the head holds no returns window that the fallback could read when optimise(opt) runs with no data.
Arguments
opt: The head.rd: The returns data, which holds one row or a block of rows,observations × assets.
Validation
- Everything
fold_online_selectionrefuses. - Everything
step_fallbackrefuses.
Returns
opt: The head, with itscacherebound to the state after the last row and its fallback stepped.
Related
PortfolioOptimisers.factory — Method
factory(opt::OnlinePortfolioSelection, w::VecNum)Thread the previous fold's weights into the head's fee and fallback, and nowhere else.
The recursion never reads them, so the method carries alg, set, w0, x_src, strict and cache unchanged.
Related
PortfolioOptimisers.port_opt_view — Method
port_opt_view(opt::OnlinePortfolioSelection, i, args...)Slices the head to the selected assets.
The rule, the set and the fee take their own views. The Start Allocation is sliced and renormalised, and the state takes the view of port_opt_view on OnlinePortfolioSelectionState.
Related
PortfolioOptimisers.Online — Method
Online(::OnlinePortfolioSelection, args...; kwargs...)Refuses to construct Online(head), and names the three routes to the two answers that the wrapper stands for.
Online declares a refit from a buffer, and for a head each of its two settings is a batch walk-forward that the library already runs. Uncapped, the refit is the expanding walk-forward. Its allocation equals the allocation of the online arm exactly, because both take the same single-row updates from w0. But it costs a pass over every row folded so far, where the online arm costs one row. Capped, the refit is the rolling walk-forward, whose allocation is the recursion restarted from w0 inside each window. So the wrapper adds no answer, and the refusal costs the caller nothing.
Related
PortfolioOptimisers._optimise — Method
_optimise(
opt::OnlinePortfolioSelection,
rd::ReturnsResult;
kwargs...
) -> NaiveOptimisationResult{Nothing, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk, Nothing} where {__T_wb, __T_fees, __T_retcode, __T_w, __T_imsk}
Runs the Causal Pass.
optimise calls this internal dispatch. It reads and writes no state that the head carries.
Algorithm
- Reset every schedule of the head to its fold-less value through
reset_time_dependent_estimator. - Fold every row of
rdfrom the Start Allocation, with no carried state, throughfold_online_selection, givingstate. - Read the Next-Period Allocation out of
statethroughonline_selection_result.
Arguments
opt: The head.rd: The returns result to use. Every row is one Online Update, in order.kwargs: Additional keyword arguments, ignored.
Returns
res::NaiveOptimisationResult: The Next-Period Allocation, withpr = nothing.
Related
PortfolioOptimisers.rows_needed — Method
rows_needed(opt::OnlinePortfolioSelection) -> Any
Returns the number of rows that the head's buffer keeps, the maximum over the rule tree and the Allocation Set.
A count of nothing is unbounded, and it wins the maximum. A schedule on set answers over every entry it holds, because the seed sizes the buffer once and every fold reads it.
Related
PortfolioOptimisers.rows_needed — Method
rows_needed(td::TimeDependent)Returns the number of rows that a per-fold schedule of Allocation Sets needs.
The count is the maximum over every entry of a vector schedule and over its explicit default. A callable schedule answers nothing, which is unbounded. The seed sizes the rows buffer once and never resizes it, so the cap must answer for every fold, and not only for the fold in which the seed runs. The entries of a callable do not exist before their fold, so the cap keeps every row folded so far, and no set of a fold can read a row that the cap dropped.
Related
PortfolioOptimisers.needs_previous_weights — Method
needs_previous_weights(
opt::OnlinePortfolioSelection
) -> Bool
Returns true if the fold loop must give the head the weights of the previous fold.
The previous weights reach the head on two paths. factory gives them to fees and to fb, so the head reads the turnover and fee terms of those two fields, and every schedule in its own fields, as a naive optimiser does. A schedule in a leader of the rule tree on alg can read them from ctx.w_prev of its context. The head reads alg through schedule_reads_previous_weights alone. A turnover or a fee of a leader measures its trade from the Price-Adjusted Allocation of the step, which factory gives the leader inside the rule, so it needs no weights from the loop.
Related
References
- [5]
- B. Li and S. C. Hoi. Online portfolio selection: A survey. ACM Computing Surveys 46, 1–36 (2014).
- [7]
- T. M. Cover. Universal portfolios. Mathematical Finance 1, 1–29 (1991).