Online Portfolio Selection: private API
PortfolioOptimisers.online_selection_seed — Function
online_selection_seed(
opt::OnlinePortfolioSelection,
rd::ReturnsResult,
set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{Int64, __T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl, Nothing, Nothing, Nothing} where {__T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl}
Seeds the head's state on its first row, before the first update.
The uniform start and a given w0 meet the set in the same way. So the allocation held during the first period lies in the set wherever the set can be formed without rows. A set that reads the head's rows, such as a programme set with a fitted risk ceiling or a tracking error, has no constraints before the first row. The seed then holds the start as given, and the first Online Update projects it. That update holds every step until the head has two rows. A programme that fails at the start holds the start as given and warns, as the Held Step of a row does.
Algorithm
- Form
start, which is1/Nat each of theNpinned assets whenw0isnothing, andw0otherwise. - Project
startonce ontosetin the rule's geometry throughproject_start, givingw0. A set that reads rows returnsstartunchanged. - Seed the state of the rule on
w0andsetthroughrule_state_seed, givingst.stis the state of the rule: what the rule keeps from one update to the next, such as a Gram matrix or a belief covariance, ornothingfor a rule that keeps nothing. Steps 2 and 3 run inside onewith_projection_stepunder the head'sstrict, which collects the holds inheld. - Warn on a hold through
report_held_steps. - Size the rows buffer
X, aReturnsBufferStatethat keeps the rows with every other field of the Returns Result, fromrows_needed. A count ofnothinggives an uncapped buffer, a count of zero gives no buffer, and any other count caps the buffer at that count. - Keep the panel of
rdaspnlwhen it is static, andnothingotherwise. Keep the benchmark names ofrdasnbwhen a reader of the head's group statesExcessSource, andnothingotherwise, throughpinned_benchmark_names. - Return the state with no row folded, the allocation
w0, the statestof the rule, the bufferX, the names,nbandpnl.
Arguments
opt: The head.rd: The returns data of the first block.set: The Allocation Set, resolved over the pinned universe.
Validation
length(opt.w0) == size(rd.X, 2), whenw0is given. ADimensionMismatchis thrown otherwise.
Returns
state::OnlinePortfolioSelectionState: The state before the first update.
Related
PortfolioOptimisers.fold_online_selection — Function
fold_online_selection(
opt::OnlinePortfolioSelection,
cache::Union{Nothing, OnlinePortfolioSelectionState},
rd::ReturnsResult
) -> OnlinePortfolioSelectionState{var"#s185", <:AbstractVector{T}} where {var"#s185"<:Integer, T}
fold_online_selection(
opt::OnlinePortfolioSelection,
cache::Union{Nothing, OnlinePortfolioSelectionState},
rd::ReturnsResult,
set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{var"#s185", <:AbstractVector{T}} where {var"#s185"<:Integer, T}
Folds every row of a Returns Result into the head's state, in order.
This is the Block Step, and a fold of the whole Returns Result from no state is the Causal Pass.
Algorithm
- Check that every
UniverseSetsin the tree of the head and insetnames the columns ofrd, in order, throughassert_universe_axis_order. A rule that holds an optimiser checks that optimiser when it runs on the rows. - Read the returns of the block in the head's frame through
benchmark_relative_returns, givingR.Risrd.X - rd.Bwhen the head'sx_srcisExcessSource, andrd.Xotherwise. - Read the active mask of
rdthroughstep_active_mask, givingamsk. - Resolve the Allocation Set over the pinned universe through
resolve_allocation_set, givingset. - Seed the state on the first block, or pin and check the carried state, through
online_selection_pin, givingstate. - Fold each row in order through
online_selection_row!, under the timestamp thatrow_timestampgives it. The row's returns are its row ofR, and the row of the raw returns data is viewed as a Returns Result of one observation throughport_opt_view. Each row updates the statestof the rule, the allocationw, the bufferXand the hold recordhold, and adds one to the row countn.stis the state of the rule: what the rule keeps from one update to the next, such as a Gram matrix or a belief covariance, ornothingfor a rule that keeps nothing. - Return the state with the new
n,w,st,Xandhold, the active mask of the last row, and the timestamps folded so far.
Arguments
opt: The head.cache: The state, ornothingbefore the first row.rd: The returns data of the block,observations × assets.set: The block's Allocation Set, before resolution over the pinned universe. It defaults to the head's own, and the fold loop's online arm passes the fold's entry of a schedule (seeonline_step_fold).
Validation
rd.Xis notnothing. AnIsNothingErroris thrown otherwise.- The rules of
assert_universe_axis_order. - The head's
x_srcis notExcessSource, orrdcarries a benchmark. AnArgumentErroris thrown otherwise, throughbenchmark_relative_returns. - Everything
step_active_maskandassert_pinned_contextrefuse. - A non-finite cell at an active asset, under
strict. AnArgumentErroris thrown.
Returns
state::OnlinePortfolioSelectionState: The state after the last row.
Related
PortfolioOptimisers.online_selection_pin — Function
online_selection_pin(
opt::OnlinePortfolioSelection,
_::Nothing,
rd::ReturnsResult,
set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{Int64, __T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl, Nothing, Nothing, Nothing} where {__T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl}
Returns the state that a block folds into.
On the first block the method seeds the state through online_selection_seed. On every later block it returns the state that the head carries, after the pin-and-check that every estimator runs at its online step. The check compares the names and the static panel with those of the first step, and the width with that of the state. When a reader of the head's group states ExcessSource, it compares the benchmark names too, because that reader reads the excess over that benchmark. The timestamps must be present at every block or at none.
Validation
length(cache.w) == size(rd.X, 2). ADimensionMismatchis thrown otherwise.- Everything
assert_pinned_contextandassert_column_presencerefuse.
Related
PortfolioOptimisers.online_selection_row! — Function
online_selection_row!(
opt::OnlinePortfolioSelection,
st,
w::AbstractVector,
X::Union{Nothing, ReturnsBufferState},
row::NamedTuple,
nx::Union{Nothing, AbstractVector{<:AbstractString}},
set::AbstractAllocationSet
) -> Tuple{Any, Any, Union{Nothing, ReturnsBufferState}, Union{Nothing, HeldStep}}
Takes one row of the recursion, inside a ProjectionStep that gives every projection of the row the rows it reads and a log for its holds.
The row's returns r are in the head's frame, in excess of the benchmark under x_src = ExcessSource(). The row's Returns Result rd is raw. The row reads a gap in two ways, and each reader gets the reading that the rest of the library gives it. The Online Update and every kernel over the price relatives read one at a gap, as if the leg held cash, which is the reading of a Held Gap. So the recursion stays over the full pinned universe and never carries a forced zero. Every statistic over the rows reads them as a batch verb reads a Returns Result, with NaN where there was no return and the active mask beside it. Such a statistic is the mean of a forecaster, the prior of a Risk Loss or of a programme set, or the re-solve of a leader. It reduces to its own Coverage Universe. So a window over an unlisted span never meets a constant column, and returns that never happened never dilute the statistic of a relisted asset.
A row with a held projection warns once, with the row's timestamp and each hold, and returns the first HeldStep record. A row whose projections all solved returns nothing.
Algorithm
- Name the Held Gaps of the returns
rat the row's indexithroughreport_row_gaps. - Fold the row's Returns Result
rdinto the rows bufferXthroughpartial_fit!, when the tree keeps one. The buffer keeps the raw returns ofrdverbatim with its active mask, and every other field ofrdbeside it. - Form the price relative
xthroughprice_relative, which is one at a gap. - Read the buffer out as
rows, aReturnsResultof the rows that the head holds, throughbuffer_returns_result. - Take the Online Update through
online_update!, inside a projection step overrows, the timestampts, the head'sstrictand the sourcex_srcof its step, whichwith_projection_stepopens. This gives the new statestof the rule, the allocationwand the log of holdsheld. - Warn on a hold through
report_held_steps, which gives the row's hold record.
Arguments
opt: The head.st: The state of the rule: what the rule keeps from one update to the next, such as a Gram matrix or a belief covariance, ornothingfor a rule that keeps nothing.w: The allocation held during the row's period.X: The rows buffer, ornothing.row: The row, as a named tuple of five fields.rholds its returns in the head's frame,amskholds its active mask ornothing,tsholds its timestamp or its index in the fold,iholds its index in the fold, andrdholds the row as aReturnsResultof one observation.nx: The asset names, for the Held Gap message.set: The Allocation Set, resolved.
Returns
(st', w', X', hold)::Tuple: The state of the rule, the allocation for the next period, the buffer after the row, and the row's hold record ornothing.
Related
PortfolioOptimisers.row_timestamp — Function
row_timestamp(ts::Nothing, t::Integer, n::Integer)
row_timestamp(ts::AbstractVector, t::Integer, n::Integer)Returns the timestamp under which a row of a block folds.
It is the timestamp of the row in the returns data. When the returns data holds no timestamps, it is the row's index in the whole fold, n + 1 after n rows.
Related
PortfolioOptimisers.report_held_steps — Function
report_held_steps(
held::AbstractVector{<:HeldStep},
ts
) -> Any
Warns once for a row with a held projection, and names the row's timestamp and every hold.
It returns the first HeldStep record, or nothing when every projection of the row solved.
Related
PortfolioOptimisers.report_row_gaps — Function
report_row_gaps(
r::AbstractVector,
w::AbstractVector,
amsk::Union{Nothing, AbstractVector{<:Bool}},
nx::Union{Nothing, AbstractVector{<:AbstractString}},
strict::Bool,
obs::Integer
)
Names the Held Gaps of one row of returns, and writes nothing.
The function passes over a non-finite cell that the row's active mask marks inactive, because an inactive asset has no return by definition. It also passes over a non-finite cell at which the recursion's weight is zero. Every other non-finite cell is a Held Gap, and strict_diagnostic reports them all in one held_gap_msg. The report is a warning by default and a refusal under strict. The function does not fill the row. The buffer keeps the cell as it is, and the Online Update reads it as one through price_relative.
Arguments
r: The row.w: The allocation held during the row's period.amsk: The row's active mask, ornothing.nx: The asset names, for the message, ornothing.strict: Whether a Held Gap is an error.obs: The row's index in the fold, which the message names as the observation of each Held Gap.
Validation
- Under
strict, the row holds no Held Gap. AnArgumentErroris thrown otherwise.
Returns
nothing.
Related
PortfolioOptimisers.buffer_returns_result — Function
buffer_returns_result(X::Nothing)
buffer_returns_result(X::ReturnsBufferState)Reads the rows buffer out as the Returns Result that the Online Update gets.
The result is a ReturnsResult of the buffer's rows verbatim, under the pinned names, with every other field that the folded rows carried: the factor returns, the benchmark, the Exogenous Series, the timestamps and the implied volatilities of the rows, and the pinned names and ivpa. So a leader whose prior states x_src = ExcessSource() subtracts the benchmark that the rows carry, as it does in batch. When the buffer records active masks, buffer_panel makes them a time-varying Asset Panel on that Returns Result. Otherwise the result carries the static panel that the buffer pinned. The method returns nothing when the tree keeps no rows.
Every statistic over the rows reads this Returns Result, and the batch verbs read the same type. So prior(pe, rd), mean(me, rd.X, rd.pnl) and optimise(opt, rd) reduce to the Coverage Universe of the window exactly as they do on a Returns Result that the ingestion layer built. The method takes no copy, because every array of the Returns Result views the valid region of its buffer. A Returns Result that carries rows carries their names by its own contract, so the refusal of a buffer with no pinned names never meets the head.
Validation
- The buffer pinned asset names. An
IsNothingErroris thrown otherwise.
Related
PortfolioOptimisers.buffer_column_view — Function
buffer_column_view(buffer::Nothing)
buffer_column_view(buffer::SampleBufferState)
buffer_column_view(buffer::AbstractVector)Views one column of a ReturnsBufferState for the Returns Result of the rows that the head holds. A buffer gives a view of its valid rows, a plain vector gives itself, and nothing gives nothing. It takes no copy, as column_matrix does, because the Online Update reads the rows within the step.
Related
PortfolioOptimisers.buffer_panel — Function
buffer_panel(X::SampleBufferState)Returns the active masks of a rows buffer as the AssetPanel of a Returns Result, or nothing when the buffer records none, which is the static panel.
The panel uses the active mask as its estimation mask too, because step_active_mask admits only that panel.
Related
PortfolioOptimisers.online_selection_result — Function
online_selection_result(
opt::OnlinePortfolioSelection,
state::OnlinePortfolioSelectionState
) -> NaiveOptimisationResult{Nothing, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk, Nothing} where {__T_wb, __T_fees, __T_retcode, __T_w, __T_imsk}
Returns the Recursion Result, the allocation of the state as a NaiveOptimisationResult over the assets active at the last folded row.
This function reads the state and writes nothing, so every call on one state returns the same Result. Its retcode is an OptimisationSuccess. Its res is the HeldStep record of the last folded row when that row was held, and nothing otherwise. A reduced allocation with no mass is the one failure from which a rule cannot recover. It happens when every held asset is inactive at the last row. This function then returns an OptimisationFailure with NaN weights, so that a fallback chain continues.
Mathematical definition
\[\begin{align} \bar{w}_i &= \frac{w_{T+1,i}}{\sum_{j \in \mathcal{I}} w_{T+1,j}}\,, \quad i \in \mathcal{I}\,. \end{align}\]
Where:
- $\bar{w}_i$: Weight of asset $i$ in the Result.
- $w_{T+1,i}$: Weight of asset $i$ in the allocation of the state, the Next-Period Allocation after $T$ rows.
- $\mathcal{I}$: Investable Mask of the Result, the assets active at the last folded row, or every asset under a static panel.
- $T$: Number of observations.
The Result expands $\bar{\boldsymbol{w}}$ back over the pinned universe with a zero at every asset outside $\mathcal{I}$. The weights are defined when the denominator is positive and every $w_{T+1,i}$ is finite.
Algorithm
- Read the active mask
imskof the last folded row, and formidx, the indices of the active assets, or every index whenimskisnothing. - View the head's set at
idx, givingset, and slice the allocation tow. - Resolve the head's fee against the full set, and view it at the Investable Mask through
investable_fees_view, givingfees. - Sum
wtos. When no asset is active,sis not positive, or an entry ofwis not finite, return anOptimisationFailurewithNaNweights and no bounds. - Resolve the weight bounds of
setover the active assets, givingwb. The resolution reads the rows that the head holds, viewed at the active assets, so a bound that reads the returns, such as aScoreWeightBounds, scores the rows of the last step. - Return
w ./ swithwb,feesand anOptimisationSuccesswhoseresis the hold record of the state.
Arguments
opt: The head.state: The state to read out.
Returns
res::NaiveOptimisationResult: The Next-Period Allocation,pr = nothing.
Related
PortfolioOptimisers.batch_from_state — Method
batch_from_state(opt::OnlinePortfolioSelection)Refuses the head with an ArgumentError, because the head has no batch estimator to return.
The head gives a Recursion Result. optimise(opt) rebuilds no returns data, runs no batch path and reads the state directly.
Related
PortfolioOptimisers.held_timestamps — Method
held_timestamps(opt::OnlinePortfolioSelection)Returns every timestamp that the head's state holds, or nothing before the first step.
Related
PortfolioOptimisers.update_online_estimator — Method
update_online_estimator(opt::Union{OnlinePortfolioSelection, PreviousWeights})Resolves the Online wrappers in the fallback of the head, at the warm-up.
The two heads step their fallback through step_fallback, so a wrapper in the prior of a fallback estimator declares the window that the fallback folds. The method resolves the fallback as the warm-up resolves an optimiser. A PreviousWeights that is the fallback of the head resolves its own fallback in turn. Any other fallback holds no wrapper to resolve, so the head passes unchanged. The recursion of the head holds no wrapper.
Related
PortfolioOptimisers.fees_carry_turnover — Function
fees_carry_turnover(fees::FeesE_Fees)
fees_carry_turnover(v::AbstractVector)
fees_carry_turnover(td::TimeDependent)
fees_carry_turnover(::Any)Returns whether a fee, or any fee that a per-fold schedule visibly holds, carries a turnover term.
A static fee answers for its own tn. A TimeDependent schedule answers for every entry of a vector value, including the entries of a nested vector, and for its explicit default. The output of a callable does not exist before its fold, so a callable answers false. Every other value, nothing among them, answers false too, as it does for needs_previous_weights.
Related
PortfolioOptimisers.assert_online_fee_source — Method
assert_online_fee_source(opt::OnlinePortfolioSelection, pws)Refuse, at the entry of the fold loop's online arm, an OnlinePortfolioSelection head whose fees carry a turnover term while the walk-forward threads no Previous-Weights Source.
The recursion reads its own allocation, so the fund's held book is the one base against which a turnover fee on this family can measure a trade. Without a Previous-Weights Source, the fee prices the distance between two consecutive targets. That distance counts buy-and-hold as trading and constant rebalancing as free. The head and the walk-forward first meet at this check, so the check runs as soon as the pair exists. A head with no tn fee passes, and so does every other estimator. fees_carry_turnover reads a per-fold schedule on fees. It checks every entry that the schedule visibly holds, and it does not check the output of a callable, which does not exist before its fold.
Arguments
opt: The estimator handed to the loop.pws: The scheme's Previous-Weights Source asresolve_previous_weights_sourcegives it, wherenothingthreads the target weights. A scheme that setswdand nopwspasses aDriftedWeights, and an explicitTargetWeightspassesnothing.
Validation
pwsis notnothingwhenopt.feescarries atnterm, in any entry that a schedule visibly holds. AnArgumentErroris thrown otherwise.
Related
PortfolioOptimisers.online_portfolio_selection_td_defaults — Function
online_portfolio_selection_td_defaults(
) -> @NamedTuple{set::BoundedAllocationSet{WeightBounds{Float64, Float64}, Nothing}}
Return the static defaults of the OnlinePortfolioSelection fields that may hold a TimeDependent.
The constructor's test-substitution pass and time_dependent_field_defaults share it, so the fold-less value of each field is declared once. fees and fb default to nothing and are left out. set has a default, so a fold-less solve runs the default Allocation Set and not an absent set.
Related
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(opt::OnlinePortfolioSelection)
assert_time_dependent_fold_count(opt::OnlinePortfolioSelection, n::Integer, all_binds::Bool = true)
update_time_dependent_estimator(opt::OnlinePortfolioSelection, ctx::TimeDependentContext, all_binds::Bool = true)Reads the schedules of the head: those in its own fields, in its fallback fb, and in the optimisers of its rule tree on alg.
The methods of a naive optimiser read the fields and the fallback. The head adds alg, because a leader in the rule tree solves at each step, and a schedule in a field of the leader must resolve at the fold, see rule_tree_fields. The update rebuilds the head through its keyword constructor, so the constructor checks every resolved value. A rule that holds no schedule comes back as the same object.
Algorithm
The update of the head:
- Return
optunchanged whenis_time_dependentisfalsefor it. - Replace each scheduled field of
optwith its value at the fold, throughupdate_time_dependent_fields, giving the newopt. - Resolve the fallback
opt.fband the ruleopt.algof the newoptat the samectx. - Rebuild
optwith the two resolved values throughrebuild_struct.
Arguments
opt: The head.n: The number of folds.ctx: The context of the fold.all_binds: Asupdate_time_dependent_estimatortakes it.
Validation
- Each vector schedule of the head, of its fallback and of its rule tree has
nentries, else aDimensionMismatchthat names the field is thrown.
Related
PortfolioOptimisers.static_allocation_set — Function
static_allocation_set(opt::OnlinePortfolioSelection) -> Any
Returns the head's Allocation Set outside every fold loop, which is its own set or the fold-less value of a schedule on set.
A schedule is defined over the folds of a cross-validation scheme. So a partial_fit! that a caller takes by hand steps inside the default of the schedule, or inside BoundedAllocationSet when the schedule has no default. The Causal Pass steps inside the same set, because its reset of the schedules has already run.
Related
PortfolioOptimisers.online_step_fold — Method
online_step_fold(opt::OnlinePortfolioSelection, ctx::TimeDependentContext, rd::ReturnsResult)Folds the fold's rows into the head inside the fold's own Allocation Set and the fold's own rule tree.
The head is the one family whose online step is the optimisation. The Constrained Update projects the raw step of every row onto set, so the step reads a schedule on set, and not only optimise(opt) with no data. A leader in the rule tree on alg solves at each row, so the step also reads each schedule in a field of the leader, see rule_tree_fields. So the method resolves the entry of set and the schedules of alg here, in the same fold and before optimise(opt) with no data resolves the other schedules. The rows fold through a head that holds the resolved rule. A rule that holds no schedule comes back as the same object, and the head is then not rebuilt. The head that the loop threads on keeps every schedule, so fold i + 1 resolves from the schedule and not from entry i. The step writes no field but the state and the fallback, so the method returns the head with its cache rebound and its fallback stepped through step_fallback.
Arguments
opt: The head the loop threads, with its schedules unresolved.ctx: The fold's context.rd: The returns data of the rows that the fold has gained.
Returns
opt: The head, with itscacherebound and its schedules unresolved.
Related
PortfolioOptimisers.pinned_benchmark_names — Function
pinned_benchmark_names(
opt::OnlinePortfolioSelection,
rd::ReturnsResult
) -> Any
Returns the benchmark names that the state of the head pins: the names of rd when a reader of the head's group states ExcessSource, and nothing otherwise.
A reader of the Excess Frame reads the excess over the benchmark of the rows, so the names of that benchmark must stay the same from one block to the next. The reader can be the step, the prior of a RiskLoss, or a reader of a ProgrammeAllocationSet. A group that reads no benchmark pins no names, so a change of benchmark between two blocks passes.
Related
PortfolioOptimisers.group_frames — Method
group_frames(opt::OnlinePortfolioSelection) -> ReturnsFrames
reader_frames(alg::AbstractOnlinePortfolioSelectionAlgorithm, path::AbstractString,
refit::Bool) -> ReturnsFramesWalk the group of an OnlinePortfolioSelection head, for the warning of a forgotten raw reader and the benchmark names that the state pins.
The group is the source of the step, the readers of each RiskLoss in the rule tree, and the readers of a ProgrammeAllocationSet on set. The step is the outermost reader of the rows, so at x_src = nothing it reads the raw returns data, and the walk names it by the type of the head. field_frames walks every field of the head and of each rule. A rule over rules walks its sub-rules, and a vector of rules walks each one. The walk stops at a leader and at the fallback, because each is a head with its own group. It stops at a forecaster and at a sample selector too: these read the frame of the step through head_frame_returns, so they are inner readers and state no source. A Risk Loss and a programme set walk their own fields under the refit rule of their own prior, see refits_regression.
Arguments
opt: The head.alg: An Online Selection Rule.path: The path of the value from the head, for example"pe","cle"or"r[2].pe". A warning names a reader by its path.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.
Returns
fr::ReturnsFrames: The frames of the group ofopt, or the frames thatalgadds.
Related