Online Portfolio Selection: private API

PortfolioOptimisers.online_selection_seed — Function
online_selection_seed(
    opt::OnlinePortfolioSelection,
    rd::ReturnsResult,
    set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{Int64, __T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl, Nothing, Nothing, Nothing} where {__T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl}

Seeds the head's state on its first row, before the first update.

The uniform start and a given w0 meet the set in the same way. So the allocation held during the first period lies in the set wherever the set can be formed without rows. A set that reads the head's rows, such as a programme set with a fitted risk ceiling or a tracking error, has no constraints before the first row. The seed then holds the start as given, and the first Online Update projects it. That update holds every step until the head has two rows. A programme that fails at the start holds the start as given and warns, as the Held Step of a row does.

Algorithm

  1. Form start, which is 1/N at each of the N pinned assets when w0 is nothing, and w0 otherwise.
  2. Project start once onto set in the rule's geometry through project_start, giving w0. A set that reads rows returns start unchanged.
  3. Seed the state of the rule on w0 and set through rule_state_seed, giving st. st is the state of the rule: what the rule keeps from one update to the next, such as a Gram matrix or a belief covariance, or nothing for a rule that keeps nothing. Steps 2 and 3 run inside one with_projection_step under the head's strict, which collects the holds in held.
  4. Warn on a hold through report_held_steps.
  5. Size the rows buffer X, a ReturnsBufferState that keeps the rows with every other field of the Returns Result, from rows_needed. A count of nothing gives an uncapped buffer, a count of zero gives no buffer, and any other count caps the buffer at that count.
  6. Keep the panel of rd as pnl when it is static, and nothing otherwise. Keep the benchmark names of rd as nb when a reader of the head's group states ExcessSource, and nothing otherwise, through pinned_benchmark_names.
  7. Return the state with no row folded, the allocation w0, the state st of the rule, the buffer X, the names, nb and pnl.

Arguments

  • opt: The head.
  • rd: The returns data of the first block.
  • set: The Allocation Set, resolved over the pinned universe.

Validation

  • length(opt.w0) == size(rd.X, 2), when w0 is given. A DimensionMismatch is thrown otherwise.

Returns

  • state::OnlinePortfolioSelectionState: The state before the first update.

Related

source
PortfolioOptimisers.fold_online_selection — Function
fold_online_selection(
    opt::OnlinePortfolioSelection,
    cache::Union{Nothing, OnlinePortfolioSelectionState},
    rd::ReturnsResult
) -> OnlinePortfolioSelectionState{var"#s185", <:AbstractVector{T}} where {var"#s185"<:Integer, T}
fold_online_selection(
    opt::OnlinePortfolioSelection,
    cache::Union{Nothing, OnlinePortfolioSelectionState},
    rd::ReturnsResult,
    set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{var"#s185", <:AbstractVector{T}} where {var"#s185"<:Integer, T}

Folds every row of a Returns Result into the head's state, in order.

This is the Block Step, and a fold of the whole Returns Result from no state is the Causal Pass.

Algorithm

  1. Check that every UniverseSets in the tree of the head and in set names the columns of rd, in order, through assert_universe_axis_order. A rule that holds an optimiser checks that optimiser when it runs on the rows.
  2. Read the returns of the block in the head's frame through benchmark_relative_returns, giving R. R is rd.X - rd.B when the head's x_src is ExcessSource, and rd.X otherwise.
  3. Read the active mask of rd through step_active_mask, giving amsk.
  4. Resolve the Allocation Set over the pinned universe through resolve_allocation_set, giving set.
  5. Seed the state on the first block, or pin and check the carried state, through online_selection_pin, giving state.
  6. Fold each row in order through online_selection_row!, under the timestamp that row_timestamp gives it. The row's returns are its row of R, and the row of the raw returns data is viewed as a Returns Result of one observation through port_opt_view. Each row updates the state st of the rule, the allocation w, the buffer X and the hold record hold, and adds one to the row count n. st is the state of the rule: what the rule keeps from one update to the next, such as a Gram matrix or a belief covariance, or nothing for a rule that keeps nothing.
  7. Return the state with the new n, w, st, X and hold, the active mask of the last row, and the timestamps folded so far.

Arguments

  • opt: The head.
  • cache: The state, or nothing before the first row.
  • rd: The returns data of the block, observations × assets.
  • set: The block's Allocation Set, before resolution over the pinned universe. It defaults to the head's own, and the fold loop's online arm passes the fold's entry of a schedule (see online_step_fold).

Validation

Returns

  • state::OnlinePortfolioSelectionState: The state after the last row.

Related

source
PortfolioOptimisers.online_selection_pin — Function
online_selection_pin(
    opt::OnlinePortfolioSelection,
    _::Nothing,
    rd::ReturnsResult,
    set::AbstractAllocationSet
) -> OnlinePortfolioSelectionState{Int64, __T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl, Nothing, Nothing, Nothing} where {__T_w, __T_st, __T_X, __T_nx, __T_nb, __T_pnl}

Returns the state that a block folds into.

On the first block the method seeds the state through online_selection_seed. On every later block it returns the state that the head carries, after the pin-and-check that every estimator runs at its online step. The check compares the names and the static panel with those of the first step, and the width with that of the state. When a reader of the head's group states ExcessSource, it compares the benchmark names too, because that reader reads the excess over that benchmark. The timestamps must be present at every block or at none.

Validation

Related

source
PortfolioOptimisers.online_selection_row! — Function
online_selection_row!(
    opt::OnlinePortfolioSelection,
    st,
    w::AbstractVector,
    X::Union{Nothing, ReturnsBufferState},
    row::NamedTuple,
    nx::Union{Nothing, AbstractVector{<:AbstractString}},
    set::AbstractAllocationSet
) -> Tuple{Any, Any, Union{Nothing, ReturnsBufferState}, Union{Nothing, HeldStep}}

Takes one row of the recursion, inside a ProjectionStep that gives every projection of the row the rows it reads and a log for its holds.

The row's returns r are in the head's frame, in excess of the benchmark under x_src = ExcessSource(). The row's Returns Result rd is raw. The row reads a gap in two ways, and each reader gets the reading that the rest of the library gives it. The Online Update and every kernel over the price relatives read one at a gap, as if the leg held cash, which is the reading of a Held Gap. So the recursion stays over the full pinned universe and never carries a forced zero. Every statistic over the rows reads them as a batch verb reads a Returns Result, with NaN where there was no return and the active mask beside it. Such a statistic is the mean of a forecaster, the prior of a Risk Loss or of a programme set, or the re-solve of a leader. It reduces to its own Coverage Universe. So a window over an unlisted span never meets a constant column, and returns that never happened never dilute the statistic of a relisted asset.

A row with a held projection warns once, with the row's timestamp and each hold, and returns the first HeldStep record. A row whose projections all solved returns nothing.

Algorithm

  1. Name the Held Gaps of the returns r at the row's index i through report_row_gaps.
  2. Fold the row's Returns Result rd into the rows buffer X through partial_fit!, when the tree keeps one. The buffer keeps the raw returns of rd verbatim with its active mask, and every other field of rd beside it.
  3. Form the price relative x through price_relative, which is one at a gap.
  4. Read the buffer out as rows, a ReturnsResult of the rows that the head holds, through buffer_returns_result.
  5. Take the Online Update through online_update!, inside a projection step over rows, the timestamp ts, the head's strict and the source x_src of its step, which with_projection_step opens. This gives the new state st of the rule, the allocation w and the log of holds held.
  6. Warn on a hold through report_held_steps, which gives the row's hold record.

Arguments

  • opt: The head.
  • st: The state of the rule: what the rule keeps from one update to the next, such as a Gram matrix or a belief covariance, or nothing for a rule that keeps nothing.
  • w: The allocation held during the row's period.
  • X: The rows buffer, or nothing.
  • row: The row, as a named tuple of five fields. r holds its returns in the head's frame, amsk holds its active mask or nothing, ts holds its timestamp or its index in the fold, i holds its index in the fold, and rd holds the row as a ReturnsResult of one observation.
  • nx: The asset names, for the Held Gap message.
  • set: The Allocation Set, resolved.

Returns

  • (st', w', X', hold)::Tuple: The state of the rule, the allocation for the next period, the buffer after the row, and the row's hold record or nothing.

Related

source
PortfolioOptimisers.row_timestamp — Function
row_timestamp(ts::Nothing, t::Integer, n::Integer)
row_timestamp(ts::AbstractVector, t::Integer, n::Integer)

Returns the timestamp under which a row of a block folds.

It is the timestamp of the row in the returns data. When the returns data holds no timestamps, it is the row's index in the whole fold, n + 1 after n rows.

Related

source
PortfolioOptimisers.report_row_gaps — Function
report_row_gaps(
    r::AbstractVector,
    w::AbstractVector,
    amsk::Union{Nothing, AbstractVector{<:Bool}},
    nx::Union{Nothing, AbstractVector{<:AbstractString}},
    strict::Bool,
    obs::Integer
)

Names the Held Gaps of one row of returns, and writes nothing.

The function passes over a non-finite cell that the row's active mask marks inactive, because an inactive asset has no return by definition. It also passes over a non-finite cell at which the recursion's weight is zero. Every other non-finite cell is a Held Gap, and strict_diagnostic reports them all in one held_gap_msg. The report is a warning by default and a refusal under strict. The function does not fill the row. The buffer keeps the cell as it is, and the Online Update reads it as one through price_relative.

Arguments

  • r: The row.
  • w: The allocation held during the row's period.
  • amsk: The row's active mask, or nothing.
  • nx: The asset names, for the message, or nothing.
  • strict: Whether a Held Gap is an error.
  • obs: The row's index in the fold, which the message names as the observation of each Held Gap.

Validation

  • Under strict, the row holds no Held Gap. An ArgumentError is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.buffer_returns_result — Function
buffer_returns_result(X::Nothing)
buffer_returns_result(X::ReturnsBufferState)

Reads the rows buffer out as the Returns Result that the Online Update gets.

The result is a ReturnsResult of the buffer's rows verbatim, under the pinned names, with every other field that the folded rows carried: the factor returns, the benchmark, the Exogenous Series, the timestamps and the implied volatilities of the rows, and the pinned names and ivpa. So a leader whose prior states x_src = ExcessSource() subtracts the benchmark that the rows carry, as it does in batch. When the buffer records active masks, buffer_panel makes them a time-varying Asset Panel on that Returns Result. Otherwise the result carries the static panel that the buffer pinned. The method returns nothing when the tree keeps no rows.

Every statistic over the rows reads this Returns Result, and the batch verbs read the same type. So prior(pe, rd), mean(me, rd.X, rd.pnl) and optimise(opt, rd) reduce to the Coverage Universe of the window exactly as they do on a Returns Result that the ingestion layer built. The method takes no copy, because every array of the Returns Result views the valid region of its buffer. A Returns Result that carries rows carries their names by its own contract, so the refusal of a buffer with no pinned names never meets the head.

Validation

  • The buffer pinned asset names. An IsNothingError is thrown otherwise.

Related

source
PortfolioOptimisers.buffer_column_view — Function
buffer_column_view(buffer::Nothing)
buffer_column_view(buffer::SampleBufferState)
buffer_column_view(buffer::AbstractVector)

Views one column of a ReturnsBufferState for the Returns Result of the rows that the head holds. A buffer gives a view of its valid rows, a plain vector gives itself, and nothing gives nothing. It takes no copy, as column_matrix does, because the Online Update reads the rows within the step.

Related

source
PortfolioOptimisers.online_selection_result — Function
online_selection_result(
    opt::OnlinePortfolioSelection,
    state::OnlinePortfolioSelectionState
) -> NaiveOptimisationResult{Nothing, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk, Nothing} where {__T_wb, __T_fees, __T_retcode, __T_w, __T_imsk}

Returns the Recursion Result, the allocation of the state as a NaiveOptimisationResult over the assets active at the last folded row.

This function reads the state and writes nothing, so every call on one state returns the same Result. Its retcode is an OptimisationSuccess. Its res is the HeldStep record of the last folded row when that row was held, and nothing otherwise. A reduced allocation with no mass is the one failure from which a rule cannot recover. It happens when every held asset is inactive at the last row. This function then returns an OptimisationFailure with NaN weights, so that a fallback chain continues.

Mathematical definition

\[\begin{align} \bar{w}_i &= \frac{w_{T+1,i}}{\sum_{j \in \mathcal{I}} w_{T+1,j}}\,, \quad i \in \mathcal{I}\,. \end{align}\]

Where:

  • $\bar{w}_i$: Weight of asset $i$ in the Result.
  • $w_{T+1,i}$: Weight of asset $i$ in the allocation of the state, the Next-Period Allocation after $T$ rows.
  • $\mathcal{I}$: Investable Mask of the Result, the assets active at the last folded row, or every asset under a static panel.
  • $T$: Number of observations.

The Result expands $\bar{\boldsymbol{w}}$ back over the pinned universe with a zero at every asset outside $\mathcal{I}$. The weights are defined when the denominator is positive and every $w_{T+1,i}$ is finite.

Algorithm

  1. Read the active mask imsk of the last folded row, and form idx, the indices of the active assets, or every index when imsk is nothing.
  2. View the head's set at idx, giving set, and slice the allocation to w.
  3. Resolve the head's fee against the full set, and view it at the Investable Mask through investable_fees_view, giving fees.
  4. Sum w to s. When no asset is active, s is not positive, or an entry of w is not finite, return an OptimisationFailure with NaN weights and no bounds.
  5. Resolve the weight bounds of set over the active assets, giving wb. The resolution reads the rows that the head holds, viewed at the active assets, so a bound that reads the returns, such as a ScoreWeightBounds, scores the rows of the last step.
  6. Return w ./ s with wb, fees and an OptimisationSuccess whose res is the hold record of the state.

Arguments

  • opt: The head.
  • state: The state to read out.

Returns

  • res::NaiveOptimisationResult: The Next-Period Allocation, pr = nothing.

Related

source
PortfolioOptimisers.update_online_estimator — Method
update_online_estimator(opt::Union{OnlinePortfolioSelection, PreviousWeights})

Resolves the Online wrappers in the fallback of the head, at the warm-up.

The two heads step their fallback through step_fallback, so a wrapper in the prior of a fallback estimator declares the window that the fallback folds. The method resolves the fallback as the warm-up resolves an optimiser. A PreviousWeights that is the fallback of the head resolves its own fallback in turn. Any other fallback holds no wrapper to resolve, so the head passes unchanged. The recursion of the head holds no wrapper.

Related

source
PortfolioOptimisers.fees_carry_turnover — Function
fees_carry_turnover(fees::FeesE_Fees)
fees_carry_turnover(v::AbstractVector)
fees_carry_turnover(td::TimeDependent)
fees_carry_turnover(::Any)

Returns whether a fee, or any fee that a per-fold schedule visibly holds, carries a turnover term.

A static fee answers for its own tn. A TimeDependent schedule answers for every entry of a vector value, including the entries of a nested vector, and for its explicit default. The output of a callable does not exist before its fold, so a callable answers false. Every other value, nothing among them, answers false too, as it does for needs_previous_weights.

Related

source
PortfolioOptimisers.assert_online_fee_source — Method
assert_online_fee_source(opt::OnlinePortfolioSelection, pws)

Refuse, at the entry of the fold loop's online arm, an OnlinePortfolioSelection head whose fees carry a turnover term while the walk-forward threads no Previous-Weights Source.

The recursion reads its own allocation, so the fund's held book is the one base against which a turnover fee on this family can measure a trade. Without a Previous-Weights Source, the fee prices the distance between two consecutive targets. That distance counts buy-and-hold as trading and constant rebalancing as free. The head and the walk-forward first meet at this check, so the check runs as soon as the pair exists. A head with no tn fee passes, and so does every other estimator. fees_carry_turnover reads a per-fold schedule on fees. It checks every entry that the schedule visibly holds, and it does not check the output of a callable, which does not exist before its fold.

Arguments

Validation

  • pws is not nothing when opt.fees carries a tn term, in any entry that a schedule visibly holds. An ArgumentError is thrown otherwise.

Related

source
PortfolioOptimisers.online_portfolio_selection_td_defaults — Function
online_portfolio_selection_td_defaults(

) -> @NamedTuple{set::BoundedAllocationSet{WeightBounds{Float64, Float64}, Nothing}}

Return the static defaults of the OnlinePortfolioSelection fields that may hold a TimeDependent.

The constructor's test-substitution pass and time_dependent_field_defaults share it, so the fold-less value of each field is declared once. fees and fb default to nothing and are left out. set has a default, so a fold-less solve runs the default Allocation Set and not an absent set.

Related

source
PortfolioOptimisers.is_time_dependent — Method
is_time_dependent(opt::OnlinePortfolioSelection)
assert_time_dependent_fold_count(opt::OnlinePortfolioSelection, n::Integer, all_binds::Bool = true)
update_time_dependent_estimator(opt::OnlinePortfolioSelection, ctx::TimeDependentContext, all_binds::Bool = true)

Reads the schedules of the head: those in its own fields, in its fallback fb, and in the optimisers of its rule tree on alg.

The methods of a naive optimiser read the fields and the fallback. The head adds alg, because a leader in the rule tree solves at each step, and a schedule in a field of the leader must resolve at the fold, see rule_tree_fields. The update rebuilds the head through its keyword constructor, so the constructor checks every resolved value. A rule that holds no schedule comes back as the same object.

Algorithm

The update of the head:

  1. Return opt unchanged when is_time_dependent is false for it.
  2. Replace each scheduled field of opt with its value at the fold, through update_time_dependent_fields, giving the new opt.
  3. Resolve the fallback opt.fb and the rule opt.alg of the new opt at the same ctx.
  4. Rebuild opt with the two resolved values through rebuild_struct.

Arguments

Validation

  • Each vector schedule of the head, of its fallback and of its rule tree has n entries, else a DimensionMismatch that names the field is thrown.

Related

source
PortfolioOptimisers.static_allocation_set — Function
static_allocation_set(opt::OnlinePortfolioSelection) -> Any

Returns the head's Allocation Set outside every fold loop, which is its own set or the fold-less value of a schedule on set.

A schedule is defined over the folds of a cross-validation scheme. So a partial_fit! that a caller takes by hand steps inside the default of the schedule, or inside BoundedAllocationSet when the schedule has no default. The Causal Pass steps inside the same set, because its reset of the schedules has already run.

Related

source
PortfolioOptimisers.online_step_fold — Method
online_step_fold(opt::OnlinePortfolioSelection, ctx::TimeDependentContext, rd::ReturnsResult)

Folds the fold's rows into the head inside the fold's own Allocation Set and the fold's own rule tree.

The head is the one family whose online step is the optimisation. The Constrained Update projects the raw step of every row onto set, so the step reads a schedule on set, and not only optimise(opt) with no data. A leader in the rule tree on alg solves at each row, so the step also reads each schedule in a field of the leader, see rule_tree_fields. So the method resolves the entry of set and the schedules of alg here, in the same fold and before optimise(opt) with no data resolves the other schedules. The rows fold through a head that holds the resolved rule. A rule that holds no schedule comes back as the same object, and the head is then not rebuilt. The head that the loop threads on keeps every schedule, so fold i + 1 resolves from the schedule and not from entry i. The step writes no field but the state and the fallback, so the method returns the head with its cache rebound and its fallback stepped through step_fallback.

Arguments

  • opt: The head the loop threads, with its schedules unresolved.
  • ctx: The fold's context.
  • rd: The returns data of the rows that the fold has gained.

Returns

  • opt: The head, with its cache rebound and its schedules unresolved.

Related

source
PortfolioOptimisers.pinned_benchmark_names — Function
pinned_benchmark_names(
    opt::OnlinePortfolioSelection,
    rd::ReturnsResult
) -> Any

Returns the benchmark names that the state of the head pins: the names of rd when a reader of the head's group states ExcessSource, and nothing otherwise.

A reader of the Excess Frame reads the excess over the benchmark of the rows, so the names of that benchmark must stay the same from one block to the next. The reader can be the step, the prior of a RiskLoss, or a reader of a ProgrammeAllocationSet. A group that reads no benchmark pins no names, so a change of benchmark between two blocks passes.

Related

source
PortfolioOptimisers.group_frames — Method
group_frames(opt::OnlinePortfolioSelection) -> ReturnsFrames
reader_frames(alg::AbstractOnlinePortfolioSelectionAlgorithm, path::AbstractString,
              refit::Bool) -> ReturnsFrames

Walk the group of an OnlinePortfolioSelection head, for the warning of a forgotten raw reader and the benchmark names that the state pins.

The group is the source of the step, the readers of each RiskLoss in the rule tree, and the readers of a ProgrammeAllocationSet on set. The step is the outermost reader of the rows, so at x_src = nothing it reads the raw returns data, and the walk names it by the type of the head. field_frames walks every field of the head and of each rule. A rule over rules walks its sub-rules, and a vector of rules walks each one. The walk stops at a leader and at the fallback, because each is a head with its own group. It stops at a forecaster and at a sample selector too: these read the frame of the step through head_frame_returns, so they are inner readers and state no source. A Risk Loss and a programme set walk their own fields under the refit rule of their own prior, see refits_regression.

Arguments

  • opt: The head.
  • alg: An Online Selection Rule.
  • path: The path of the value from the head, for example "pe", "cle" or "r[2].pe". A warning names a reader by its path.
  • refit: true when a regression reader at x_src = nothing in the group fits its regression on the raw returns data. refits_regression finds the value.

Returns

  • fr::ReturnsFrames: The frames of the group of opt, or the frames that alg adds.

Related

source