The online portfolio selection head: private API

PortfolioOptimisers.online_selection_seed — Function

online_selection_seed(
    opt::OnlinePortfolioSelection,
    rd::ReturnsResult,
    set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{Int64, __T_w, __T_st, __T_X, __T_nx, __T_pnl, Nothing, Nothing, Nothing} where {__T_w, __T_st, __T_X, __T_nx, __T_pnl}

Seeds the head's state on its first row, before the first update.

The uniform start and a given w0 meet the set in the same way. So the allocation held during the first period lies in the set wherever the set can be formed without rows. A set that reads the head's rows, such as a programme set with a fitted risk ceiling or a tracking error, has no constraints before the first row. The seed then holds the start as given, and the first Online Update projects it. That update holds every step until the head has two rows. A programme that fails at the start holds the start as given and warns, as the Held Step of a row does.

Algorithm

Form start, which is 1/N at each of the N pinned assets when w0 is nothing, and w0 otherwise.
Project start once onto set in the rule's geometry through project_start, giving w0. A set that reads rows returns start unchanged.
Seed the rule's carrier on w0 and set through rule_state_seed, giving st. Steps 2 and 3 run inside one with_projection_step under the head's strict, which collects the holds in held.
Warn on a hold through report_held_steps.
Size the rows buffer X from rows_needed. A count of nothing gives an uncapped buffer, a count of zero gives no buffer, and any other count caps the buffer at that count.
Keep the carrier's panel as pnl when it is static, and nothing otherwise.
Return the state with no row folded, the allocation w0, the carrier st, the buffer X, the names and pnl.

Arguments

opt: The head.
rd: The carrier of the first block.
set: The Allocation Set, resolved over the pinned universe.

Validation

length(opt.w0) == size(rd.X, 2), when w0 is given. A DimensionMismatch is thrown otherwise.

Returns

state::OnlinePortfolioSelectionState: The state before the first update.

Related

PortfolioOptimisers.fold_online_selection — Function

fold_online_selection(
    opt::OnlinePortfolioSelection,
    cache::Union{Nothing, OnlinePortfolioSelectionState},
    rd::ReturnsResult
) -> OnlinePortfolioSelectionState{__T_n, __T_w, _A, __T_X, __T_nx, __T_pnl, __T_amsk, __T_ts, __T_hold} where {__T_n<:Integer, __T_w<:(AbstractVector), _A, __T_X<:Union{Nothing, SampleBufferState}, __T_nx<:Union{Nothing, AbstractVector{<:AbstractString}}, __T_pnl<:Union{Nothing, AssetPanel}, __T_amsk<:Union{Nothing, AbstractVector{<:Bool}}, __T_ts<:Union{Nothing, AbstractVector}, __T_hold<:Union{Nothing, HeldStep}}
fold_online_selection(
    opt::OnlinePortfolioSelection,
    cache::Union{Nothing, OnlinePortfolioSelectionState},
    rd::ReturnsResult,
    set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{__T_n, __T_w, _A, __T_X, __T_nx, __T_pnl, __T_amsk, __T_ts, __T_hold} where {__T_n<:Integer, __T_w<:(AbstractVector), _A, __T_X<:Union{Nothing, SampleBufferState}, __T_nx<:Union{Nothing, AbstractVector{<:AbstractString}}, __T_pnl<:Union{Nothing, AssetPanel}, __T_amsk<:Union{Nothing, AbstractVector{<:Bool}}, __T_ts<:Union{Nothing, AbstractVector}, __T_hold<:Union{Nothing, HeldStep}}

Folds every row of a carrier into the head's state, in order.

This is the Block Step, and a fold of the whole carrier from no state is the Causal Pass.

Algorithm

Read the active mask of the carrier through step_active_mask, giving amsk.
Resolve the Allocation Set over the pinned universe through resolve_allocation_set, giving set.
Seed the state on the first block, or pin and check the carried state, through online_selection_pin, giving state.
Fold each row in order through online_selection_row!, under the timestamp that row_timestamp gives it. Each row updates the carrier st, the allocation w, the buffer X and the hold record hold, and adds one to the row count n.
Return the state with the new n, w, st, X and hold, the active mask of the last row, and the timestamps folded so far.

Arguments

opt: The head.
cache: The state, or nothing before the first row.
rd: The carrier of the block, observations × assets.
set: The block's Allocation Set, before resolution over the pinned universe. It defaults to the head's own, and the fold loop's online arm passes the fold's entry of a schedule (see online_step_fold).

Validation

rd.X is not nothing. An IsNothingError is thrown otherwise.
Everything step_active_mask and assert_pinned_context refuse.
A non-finite cell at an active asset, under strict. An ArgumentError is thrown.

Returns

state::OnlinePortfolioSelectionState: The state after the last row.

Related

PortfolioOptimisers.online_selection_pin — Function

online_selection_pin(
    opt::OnlinePortfolioSelection,
    _::Nothing,
    rd::ReturnsResult,
    set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{Int64, __T_w, __T_st, __T_X, __T_nx, __T_pnl, Nothing, Nothing, Nothing} where {__T_w, __T_st, __T_X, __T_nx, __T_pnl}

Returns the state that a block folds into.

On the first block the method seeds the state through online_selection_seed. On every later block it returns the state that the head carries, after the pin-and-check that every host runs. The check compares the names and the static panel with those of the first step, and the width with that of the state. The timestamps must be present at every block or at none.

Validation

length(cache.w) == size(rd.X, 2). A DimensionMismatch is thrown otherwise.
Everything assert_pinned_context and assert_column_presence refuse.

Related

fold_online_selection
online_selection_seed

PortfolioOptimisers.online_selection_row! — Function

online_selection_row!(
    opt::OnlinePortfolioSelection,
    st,
    w::AbstractVector,
    X::Union{Nothing, SampleBufferState},
    row::NamedTuple,
    nx::Union{Nothing, AbstractVector{<:AbstractString}},
    set::AbstractAllocationSet
) -> Tuple{Any, Any, Any, Union{Nothing, HeldStep}}

Takes one row of the recursion, inside a ProjectionStep that gives every projection of the row the rows it reads and a log for its holds.

The row reads a gap in two ways, and each reader gets the reading that the rest of the library gives it. The Online Update and every kernel over the price relatives read one at a gap, as if the leg held cash, which is the reading of a Held Gap. So the recursion stays over the full pinned universe and never carries a forced zero. Every statistic over the rows reads the carrier as a batch verb reads one, with NaN where there was no return and the active mask beside it. Such a statistic is the mean of a forecaster, the prior of a Risk Loss or of a programme set, or the re-solve of a leader. It reduces to its own Coverage Universe. So a window over an unlisted span never meets a constant column, and returns that never happened never dilute the statistic of a relisted asset.

A row with a held projection warns once, with the row's timestamp and each hold, and returns the first HeldStep record. A row whose projections all solved returns nothing.

Algorithm

Name the Held Gaps of the returns r at the row's index i through report_row_gaps.
Push r verbatim, with its active mask amsk, into the rows buffer X, when the tree keeps one.
Form the price relative x through price_relative, which is one at a gap.
Read the buffer out as the carrier rows through rows_carrier.
Take the Online Update through online_update!, inside a projection step over rows and the timestamp ts that with_projection_step opens. This gives the carrier st, the allocation w and the log of holds held.
Warn on a hold through report_held_steps, which gives the row's hold record.

Arguments

opt: The head.
st: The rule's carrier.
w: The allocation held during the row's period.
X: The rows buffer, or nothing.
row: The row, as a named tuple of four fields. r holds its returns verbatim, amsk holds its active mask or nothing, ts holds its timestamp or its index in the fold, and i holds its index in the fold.
nx: The asset names, for the Held Gap message.
set: The Allocation Set, resolved.

Returns

(st', w', X', hold)::Tuple: The carrier, the allocation for the next period, the buffer after the row, and the row's hold record or nothing.

Related

PortfolioOptimisers.row_timestamp — Function

row_timestamp(ts::Nothing, t::Integer, n::Integer)
row_timestamp(ts::AbstractVector, t::Integer, n::Integer)

Returns the timestamp under which a row of a block folds.

It is the carrier's timestamp. When the carrier holds none, it is the row's index in the whole fold, n + 1 after n rows.

Related

fold_online_selection
HeldStep

PortfolioOptimisers.report_held_steps — Function

report_held_steps(
    held::AbstractVector{<:HeldStep},
    ts
) -> Any

Warns once for a row with a held projection, and names the row's timestamp and every hold.

It returns the first HeldStep record, or nothing when every projection of the row solved.

Related

online_selection_row!
HeldStep

PortfolioOptimisers.report_row_gaps — Function

report_row_gaps(
    r::AbstractVector,
    w::AbstractVector,
    amsk::Union{Nothing, AbstractVector{<:Bool}},
    nx::Union{Nothing, AbstractVector{<:AbstractString}},
    strict::Bool,
    obs::Integer
)

Names the Held Gaps of one row of returns, and writes nothing.

The function passes over a non-finite cell that the row's active mask marks inactive, because an inactive asset has no return by definition. It also passes over a non-finite cell at which the recursion's weight is zero. Every other non-finite cell is a Held Gap, and strict_diagnostic reports them all in one held_gap_msg. The report is a warning by default and a refusal under strict. The function does not fill the row. The buffer keeps the cell as it is, and the Online Update reads it as one through price_relative.

Arguments

r: The row.
w: The allocation held during the row's period.
amsk: The row's active mask, or nothing.
nx: The asset names, for the message, or nothing.
strict: Whether a Held Gap is an error.
obs: The row's index in the fold, which the message names as the observation of each Held Gap.

Validation

Under strict, the row holds no Held Gap. An ArgumentError is thrown otherwise.

Returns

nothing.

Related

PortfolioOptimisers.rows_carrier — Function

rows_carrier(X::Nothing, nx)
rows_carrier(X::SampleBufferState, nx::VecStr)
rows_carrier(X::SampleBufferState, nx::Nothing)

Reads the rows buffer out as the carrier that the Online Update gets.

The carrier is a ReturnsResult of the buffer's rows verbatim, under the pinned names. When the buffer records active masks, buffer_panel makes them a time-varying Asset Panel on the carrier. The method returns nothing when the tree keeps no rows.

Every statistic over the rows reads this carrier, and it is the carrier that the batch verbs read. So prior(pe, rd), mean(me, rd.X, rd.pnl) and optimise(opt, rd) reduce to the Coverage Universe of the window exactly as they do on a carrier that the ingestion layer built. The method takes no copy, because the carrier views the valid region of the buffer. A buffer with no pinned names is refused by name. A Returns Result that carries rows carries their names by its own contract, so the head never meets that pair.

Validation

nx is not nothing when the head keeps rows. An IsNothingError is thrown otherwise.

Related

PortfolioOptimisers.buffer_panel — Function

buffer_panel(X::SampleBufferState)

Returns the active masks of a rows buffer as the AssetPanel of a carrier, or nothing when the buffer records none, which is the static panel.

The panel uses the active mask as its estimation mask too, because step_active_mask admits only that panel.

Related

PortfolioOptimisers.online_selection_readout — Function

online_selection_readout(
    opt::OnlinePortfolioSelection,
    state::OnlinePortfolioSelectionState
) -> NaiveOptimisationResult{Nothing, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk, Nothing} where {__T_wb, __T_fees, __T_retcode, __T_w, __T_imsk}

Returns the Recursion Read-out, the allocation of the state as a NaiveOptimisationResult over the assets active at the last folded row.

The read-out reads the state and writes nothing, so every call on one state returns the same Result. Its retcode is an OptimisationSuccess. Its res is the HeldStep record of the last folded row when that row was held, and nothing otherwise. A reduced allocation with no mass is the one failure from which a rule cannot recover. It happens when every held asset is inactive at the last row. The read-out then returns an OptimisationFailure with NaN weights, so that a fallback chain continues.

Mathematical definition

\[\begin{align} \bar{w}_i &= \frac{w_{T+1,i}}{\sum_{j \in \mathcal{I}} w_{T+1,j}}\,, \quad i \in \mathcal{I}\,. \end{align}\]

Where:

$\bar{w}_i$: Weight of asset $i$ in the Result.
$w_{T+1,i}$: Weight of asset $i$ in the allocation of the state, the Next-Period Allocation after $T$ rows.
$\mathcal{I}$: Investable Mask of the read-out, the assets active at the last folded row, or every asset under a static panel.
$T$: Number of observations.

The Result expands $\bar{\boldsymbol{w}}$ back over the pinned universe with a zero at every asset outside $\mathcal{I}$. The weights are defined when the denominator is positive and every $w_{T+1,i}$ is finite.

Algorithm

Read the active mask imsk of the last folded row, and form idx, the indices of the active assets, or every index when imsk is nothing.
View the head's set at idx, giving set, and slice the allocation to w.
Resolve the head's fee against the full set, and view it at the Investable Mask through investable_fees_view, giving fees.
Sum w to s. When no asset is active, s is not positive, or an entry of w is not finite, return an OptimisationFailure with NaN weights and no bounds.
Resolve the weight bounds of set over the active assets, giving wb.
Return w ./ s with wb, fees and an OptimisationSuccess whose res is the hold record of the state.

Arguments

opt: The head.
state: The state to read out.

Returns

res::NaiveOptimisationResult: The Next-Period Allocation, pr = nothing.

Related

PortfolioOptimisers.online_readout — Method

online_readout(opt::OnlinePortfolioSelection)

Refuses by name, because the head has no batch estimator to return.

A Recursion Read-out rebuilds no carrier and runs no batch path. optimise(opt) reads the state directly.

Related

PortfolioOptimisers.held_timestamps — Method

held_timestamps(opt::OnlinePortfolioSelection)

Returns every timestamp that the head's state holds, or nothing before the first step.

Related

PortfolioOptimisers.fees_carry_turnover — Function

fees_carry_turnover(fees::Nothing)
fees_carry_turnover(fees::FeesE_Fees)
fees_carry_turnover(v::AbstractVector)
fees_carry_turnover(td::TimeDependent)
fees_carry_turnover(::Any)

Returns whether a fee, or any fee that a per-fold schedule visibly holds, carries a turnover term.

A static fee answers for its own tn. A TimeDependent schedule answers for every entry of a vector value, including the entries of a nested vector, and for its explicit default. The output of a callable does not exist before its fold, so a callable answers false. Every other value answers false too, as it does for needs_previous_weights.

Related

PortfolioOptimisers.assert_online_fee_source — Method

assert_online_fee_source(opt::OnlinePortfolioSelection, pws)

Refuse, at the entry of the fold loop's online arm, an OnlinePortfolioSelection head whose fees carry a turnover term while the walk-forward threads no Previous-Weights Source.

The recursion reads its own allocation, so the fund's held book is the one base against which a turnover fee on this family can measure a trade. Without a Previous-Weights Source, the fee prices the distance between two consecutive targets. That distance counts buy-and-hold as trading and constant rebalancing as free. The head and the walk-forward first meet at this check, so the check runs as soon as the pair exists. A head with no tn fee passes, and so does every other estimator. fees_carry_turnover reads a per-fold schedule on fees. It checks every entry that the schedule visibly holds, and it does not check the output of a callable, which does not exist before its fold.

Arguments

opt: The estimator handed to the loop.
pws: The scheme's Previous-Weights Source, or nothing.

Validation

pws is not nothing when opt.fees carries a tn term, in any entry that a schedule visibly holds. An ArgumentError is thrown otherwise.

Related

PortfolioOptimisers.online_portfolio_selection_td_defaults — Function

online_portfolio_selection_td_defaults(

) -> @NamedTuple{set::BoundedAllocationSet{WeightBounds{Float64, Float64}, Nothing}}

Return the static defaults of the OnlinePortfolioSelection fields that may hold a TimeDependent.

The constructor's test-substitution pass and time_dependent_field_defaults share it, so the fold-less value of each field is declared once. fees and fb default to nothing and are left out. set has a default, so a fold-less solve runs the default Allocation Set and not an absent set.

Related

PortfolioOptimisers.static_allocation_set — Function

static_allocation_set(opt::OnlinePortfolioSelection) -> Any

Returns the head's Allocation Set outside every fold loop, which is its own set or the fold-less value of a schedule on set.

A schedule is defined over the folds of a cross-validation scheme. So a partial_fit! that a caller takes by hand steps inside the default of the schedule, or inside BoundedAllocationSet when the schedule has no default. The Causal Pass steps inside the same set, because its reset of the schedules has already run.

Related

PortfolioOptimisers.online_step_fold — Method

online_step_fold(opt::OnlinePortfolioSelection, ctx::TimeDependentContext, rd::ReturnsResult)

Folds the fold's rows into the head inside the fold's own Allocation Set.

The head is the one family whose online step is the optimisation. The Constrained Update projects the raw step of every row onto set, so the step reads a schedule on set, and not only the read-out. So the method resolves the entry here, in the same fold and before the read-out resolves the other schedules. The head that the loop threads on keeps the schedule, so fold i + 1 resolves from the schedule and not from entry i. The step writes no field but the state, so the method returns the head with its cache rebound.

Arguments

opt: The head the loop threads, with its schedules unresolved.
ctx: The fold's context.
rd: The carrier of the rows the fold has gained.

Returns

opt: The head, with its cache rebound and its schedules unresolved.

Related

online_step_fold(opt::OnlinePortfolioSelection, ctx::TimeDependentContext, rd::ReturnsResult)

Folds the fold's rows into the head inside the fold's own Allocation Set.

abstract type AbstractAllocationSet <: AbstractAlgorithm

Abstract supertype for the Allocation Set of an online portfolio selection head, the set that contains each allocation of the recursion.

struct AssetPanel{__T_pf, __T_amsk, __T_emsk} <: AbstractResult

The Asset Panel: the Panel Fields of one universe, and the two point-in-time universe masks.

struct ReturnsResult{__T_nx, __T_X, __T_nf, __T_F, __T_nb, __T_B, __T_ts, __T_iv, __T_ivpa, __T_pnl} <: AbstractReturnsResult

Returns data of a universe: the asset returns, and optionally the factor returns, the benchmark returns, the timestamps, the implied volatilities and an Asset Panel.

struct WeightBounds{__T_lb, __T_ub} <: AbstractConstraintResult

Bounds every portfolio weight between a lower and an upper limit.

struct TimeDependent{T1, T2} <: AbstractEstimator

Varies one optimiser input across the folds of a cross-validation scheme.

struct TimeDependentContext{T1, T2, T3, T4, T5, T6, T7} <: AbstractResult

Describes one fold to the time-dependent constraints that resolve against it.

struct NaiveOptimisationResult{__T_pr, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk, __T_fb} <: NonJuMPOptimisationResult

Holds the weights, the bounds, the fee and the return code of a naive optimiser's fit.

struct BoundedAllocationSet{__T_wb, __T_sets} <: AbstractAllocationSet

Limits the allocation to the budget and to per-asset weight bounds, with closed-form projections and no solver.

struct OnlinePortfolioSelection{__T_alg, __T_set, __T_w0, __T_fees, __T_fb, __T_strict, __T_cache} <: NaiveOptimisationEstimator

Updates its allocation from each realised price relative through the Online Selection Rule on alg.

const VecStr = AbstractVector{<:AbstractString}

Alias for an abstract vector of strings.

struct SampleBufferState{__T_n, __T_off, __T_X, __T_A, __T_E, __T_F, __T_max_history} <: AbstractPartialFitState

Carries the observations an estimator keeps when its estimate has no exact incremental fold.

const FeesE_Fees = Union{<:Fees, <:FeesEstimator}

Union type for fee constraint objects and estimators.

online_step_fold(est, ctx::Option{<:TimeDependentContext}, rd::Prices_RR)

Folds the rows that a fold gained into the estimator that the online arm threads.

assert_online_fee_source(est, pws)

Refuses, at the entry of the online arm of the fold loop, an estimator whose fee needs a Previous-Weights Source that the scheme does not give.

struct HeldStep{T1, T2<:AbstractString, T3}

Records a Held Step, a projection of one row that did not give an allocation on the budget, so the step traded nothing.

struct OnlinePortfolioSelectionState{__T_n, __T_w, __T_st, __T_X, __T_nx, __T_pnl, __T_amsk, __T_ts, __T_hold} <: AbstractPartialFitState

Holds the Partial Fit State of an online portfolio selection head: the Rule State, the rows, the Fold Context and every folded timestamp.