GitHub

The online step: private API

PortfolioOptimisers.Online_Option — Type
const Online_Option{X} = Union{Nothing, <:Online, X}

Alias for a field that accepts nothing, a static estimator of type X, or an Online declaration.

The set of fields whose constructor signatures use this alias is the single source of truth for which estimators may take the online step from a sample buffer, exactly as TD_Option is for the fields that may vary over folds.

Related

source
PortfolioOptimisers.SampleBufferState — Type
struct SampleBufferState{__T_n, __T_off, __T_X, __T_A, __T_E, __T_F, __T_max_history} <: AbstractPartialFitState

Carries the observations an estimator keeps when its estimate has no exact incremental fold.

The buffer of the partial_fit! seam, and a state like any other: it lives in the estimator's cache field, it copies, it slices by asset, and obs_weights_view drops it. It holds the observations verbatim, NaN included, so a read-out over the buffer answers exactly what a batch fit over the same rows answers, and the Coverage Universe of the two agrees by construction rather than by test.

It holds the per-observation masks verbatim on the same terms. A CoveragePolicy reads two facts out of an active mask that the rows alone do not carry — a cell that is finite but inactive is excluded, and an asset active at one observation and inactive at the next is a delisting rather than a holiday — so a buffer that kept the rows and dropped the mask would answer a different question from the batch fit, silently. A is that mask, E is the estimation mask the two regime-adjusted families read, and each is a backing matrix of the shape of X or nothing. A wrapped estimator folded under a policy therefore matches a batch fit over the same window exactly, as it already does without one.

It holds the factor observations on the same terms again. A prior whose batch verb is prior(pe, X, F) regresses the asset returns on the factor returns, so its refit reads two matrices whose t-th rows are contemporaneous; F is the second, a backing matrix of capacity × factors or nothing, indexed by the same off and n as X so that the contemporaneity is structural rather than asserted. Whether a fit reads it is a fact of the estimator tree, which needs_factor_returns answers, and not of the buffer: the buffer records what the fold gives it. There is no second buffer type, and a cap windows the factor rows with the rest.

Whether the buffer records a mask, and whether it records factor rows, is fixed by its first append, exactly as the width and the element type are, and a later fold that disagrees is refused by name in both directions. A buffer holding no observations records nothing about them, so it empties and seeds afresh.

Two kinds of member hold one. A carry folds its estimate exactly and keeps the observations because a consumer downstream reads them — LowOrderPrior carries X for the scenario risk measures — so the buffer is memory rather than arithmetic. A refit has no recursion at all, and its read-out runs the batch verb over the buffer. Neither is a windowed estimator: Online is the configuration that seeds a buffer, and max_history caps what the buffer keeps and nothing else.

The rows are held in a backing matrix with spare capacity, so an append costs amortised O(1): off is the number of rows before the valid region and n its length, and the region is moved to the front of the backing matrix only when the append would run past its end. sample_buffer reads the valid region out.

Fields

  • n: Number of observations folded into the state.
  • off: Number of rows of X that sit before the valid region. It rises when a capped buffer drops its oldest observation, and returns to zero when the valid region is moved to the front of X.
  • X: Backing matrix of the buffer, capacity × assets. Rows off + 1 to off + n are the observations, in the order they were folded; the rest is spare capacity and holds no meaning. The width and the element type are fixed by the first append.
  • A: Backing matrix of the active mask, of the shape of X, or nothing when the buffer records no activity. Rows off + 1 to off + n are the mask of the observations, cell for cell with them. It is fixed by the first append, and it is what lets a read-out tell a delisting from a holiday.
  • E: Backing matrix of the estimation mask, of the shape of X, or nothing when the buffer records none. It is the second per-observation mask the batch verbs take, it is carried on the same terms as A, and only the two regime-adjusted families read it.
  • F: Backing matrix of the factor observations, capacity × factors, indexed by the same off and n as X, or nothing when the fold carries none. Its width and element type are fixed by the first append that carries it.
  • max_history: Optional cap on the number of observations the carry buffer keeps. nothing keeps every observation folded so far. A cap bounds memory and nothing else: it does not make the estimator windowed, because an estimate that folds exactly is fitted over every observation whatever the buffer holds.

Constructors

SampleBufferState(;    n::Integer = 0,    off::Integer = 0,    X::MatNum = Matrix{Float64}(undef, 0, 0),    A::Option{<:AbstractMatrix{<:Bool}} = nothing,    E::Option{<:AbstractMatrix{<:Bool}} = nothing,    F::Option{<:MatNum} = nothing,    max_history::Option{<:Integer} = nothing) -> SampleBufferState

Keywords correspond to the struct's fields. The default is the empty seed Online builds: it carries the cap and no observations, and the first append fixes the width, the element type, the masks and the factor rows from the observations it is given.

Validation

  • n >= 0. A DomainError is thrown otherwise.
  • off >= 0. A DomainError is thrown otherwise.
  • off + n <= size(X, 1). A DimensionMismatch is thrown otherwise.
  • max_history > 0 when it is not nothing. A DomainError is thrown otherwise.
  • n <= max_history when max_history is not nothing. A DimensionMismatch is thrown otherwise.
  • A and E, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.
  • F, when it is not nothing, has as many rows as X. A DimensionMismatch is thrown otherwise.

View parameters

When port_opt_view is called on this type, its fields are subset to the selected assets:

  • X: Sliced to the selected indices via port_opt_view.
  • A: Sliced to the selected indices via port_opt_view.
  • E: Sliced to the selected indices via port_opt_view.
  • F: Copied unchanged, because the selection indexes assets and this backing describes factors.

Examples

julia> PortfolioOptimisers.SampleBufferState(; max_history = 2)PortfolioOptimisers.SampleBufferState            n ┼ Int64: 0          off ┼ Int64: 0            X ┼ 0×0 Matrix{Float64}            A ┼ nothing            E ┼ nothing            F ┼ nothing  max_history ┴ Int64: 2

Related

source
PortfolioOptimisers.assert_sample_buffer_state — Function
assert_sample_buffer_state(
    n::Integer,
    off::Integer,
    X::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    max_history::Union{Nothing, Integer}
)

Refuses a sample buffer whose valid region does not sit inside its backing matrix.

Algorithm

  1. Refuse a negative n and a negative off.
  2. Refuse a valid region that runs past the last row of X.
  3. Return when max_history is nothing.
  4. Refuse a non-positive max_history, and refuse a valid region longer than it.

Arguments

  • n: Number of observations folded into the state.
  • off: Number of rows of X before the valid region.
  • X: Backing matrix of the buffer.
  • max_history: Optional cap on the number of observations the carry buffer keeps. nothing keeps every observation folded so far. A cap bounds memory and nothing else: it does not make the estimator windowed, because an estimate that folds exactly is fitted over every observation whatever the buffer holds.

Validation

  • n >= 0. A DomainError is thrown otherwise.
  • off >= 0. A DomainError is thrown otherwise.
  • off + n <= size(X, 1). A DimensionMismatch is thrown otherwise.
  • max_history > 0 when it is not nothing. A DomainError is thrown otherwise.
  • n <= max_history when max_history is not nothing. A DimensionMismatch is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_buffer_mask_shape — Function
assert_buffer_mask_shape(
    X::AbstractMatrix,
    A::Union{Nothing, AbstractMatrix{<:Bool}},
    E::Union{Nothing, AbstractMatrix{<:Bool}}
)

Refuses a per-observation mask whose shape is not the shape of the observations it belongs to.

A mask a buffer holds is cell for cell with the observations it holds, and a mask a fold is given is cell for cell with the block it is given, so one rule covers both: a mask is either absent or of the shape of the matrix beside it. It is the check the batch verbs run on their own arguments, moved to where the buffer stores them.

Algorithm

  1. Return for a mask that is nothing, which is the whole of the no-policy case.
  2. Refuse a mask whose size is not size(X), naming the mask.

Arguments

  • X: The observations the masks belong to.
  • A: The active mask, or nothing.
  • E: The estimation mask, or nothing.

Validation

  • A and E, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_buffer_factor_shape — Function
assert_buffer_factor_shape(
    X::AbstractMatrix,
    F::Union{Nothing, AbstractMatrix}
)

Refuses a factor block whose rows are not the rows of the observations it belongs to.

A factor observation is contemporaneous with the asset observation of the same row, so a factor block a fold is given has one row per row of the block beside it, and the backing a buffer holds has one row per row of X. The two widths are deliberately not compared: an asset count and a factor count are unrelated.

Algorithm

  1. Return for a factor block that is nothing, which is the whole of the no-factor case.
  2. Refuse a factor block whose row count is not size(X, 1).

Arguments

  • X: The observations the factor rows belong to.
  • F: The factor rows, or nothing.

Validation

  • F, when it is not nothing, has as many rows as X. A DimensionMismatch is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.buffer_rows_view — Function
buffer_rows_view(M::Nothing, rows) -> Nothing
buffer_rows_view(M::AbstractMatrix, rows) -> SubArray

Reads a row range out of a per-observation backing, and passes a backing that is not there through.

The one place the buffer slices a mask or the factor rows on the observation axis, so the nothing case is written once rather than at each of its callers. sample_buffer_kwargs and factor_buffer read the valid region of a stored backing with it, and partial_fit! truncates an incoming block with it.

Arguments

  • M: The backing to slice, or nothing.
  • rows: The rows to keep.

Returns

  • M: The rows of the backing, or nothing.

Related

source
PortfolioOptimisers.sample_buffer — Function
sample_buffer(state::SampleBufferState)
sample_buffer(est)

Reads the observations a sample buffer holds, observations × assets.

The read-out of SampleBufferState. It is a view of the valid region of the backing matrix, in the order the observations were folded, so a refit runs over it without copying it. The estimator form reads the state out of the cache field, and refuses an estimator that carries no buffer.

Arguments

  • state: The buffer to read.
  • est: Estimator whose cache field carries the buffer.

Validation

Returns

  • X::SubArray: The observations the buffer holds, observations × assets.

Related

source
sample_buffer(state::PriorCarryState) -> Any

Reads the observations a PriorCarryState carries, observations × assets.

Arguments

  • state: The carry state to read.

Returns

  • X::SubArray: The observations the state carries, in the order they were folded.

Related

source
PortfolioOptimisers.sample_buffer_kwargs — Function
sample_buffer_kwargs(state::SampleBufferState) -> NamedTuple

Reads the per-observation masks a sample buffer holds, as the keywords a batch verb takes.

The second half of the buffer's read-out. sample_buffer gives the rows and this gives the masks that explain them, so a read-out arm is the batch verb over the one, splatted with the other, and the wrapped answer is the unwrapped answer over the same window. A buffer that records no mask gives an empty set of keywords, which is why the no-policy case costs nothing: the arm reduces to the call it has always made, and the branch is on a field whose type is concrete, so it is resolved at compile time.

Algorithm

  1. Slice each stored mask to the valid region with buffer_rows_view.
  2. Name the masks that are there, and omit the ones that are not.

Arguments

  • state: The buffer to read.

Returns

  • kwargs::NamedTuple: The active_mask and estimation_mask keywords the buffer holds, of which either or both may be absent.

Related

source
PortfolioOptimisers.factor_buffer — Function
factor_buffer(state::SampleBufferState) -> Any

Reads the factor observations a sample buffer holds, observations × factors, or nothing when it holds none.

The third part of the buffer's read-out, beside sample_buffer and sample_buffer_kwargs. A refit whose batch verb reads a factor matrix takes it from here, as the second positional argument of that verb, and the t-th row is contemporaneous with the t-th row of sample_buffer by construction, because the two backings are indexed by the same off and n. A buffer that records no factor rows answers nothing, which is what the batch verb of a prior that reads none is given, so the branch is on a field whose type is concrete and costs the no-factor case nothing.

Arguments

  • state: The buffer to read.

Returns

  • F: The factor observations the buffer holds, as a view of the valid region, or nothing.

Related

source
PortfolioOptimisers.assert_sample_buffer — Method
assert_sample_buffer(
    est::Union{AbstractEstimator, CovarianceEstimator}
)

Returns the sample buffer an estimator carries, and refuses an estimator that carries none.

The buffering arm of partial_fit! reads its state through this function, so an estimator that was never wrapped in Online meets a message naming the wrapper rather than a MethodError. An estimator with no cache field at all meets the same message. An unresolved Online has a method of its own, because the thing it is missing is not a buffer but the warm-up that would have seeded one.

Arguments

  • est: Estimator whose cache field carries the buffer.

Validation

  • est has a cache field holding a SampleBufferState. An ArgumentError is thrown otherwise.

Returns

  • state::SampleBufferState: The buffer the estimator carries.

Related

source
PortfolioOptimisers.assert_sample_buffer — Method
assert_sample_buffer(_::Online)

Refuses an Online that reached a fold without being resolved, naming the warm-up that resolves it.

update_online_estimator replaces every wrapper it can reach at warm-up, so a wrapper that reaches a fold is one the warm-up never saw. The case that produces it is a callable TimeDependent: a schedule's value is computed per fold, after the warm-up has run, so a wrapper the callable returns is never seeded. The vector and default forms of a schedule are refused at construction; a callable's return cannot be, because it does not exist until the fold does.

Returning the estimator the wrapper would produce is not the fix either, and #870 owns what is: a schedule replaces the field's value every fold, while the buffer is threaded through it, so a schedule that hands back a captured estimator hands back the buffer as it stood when the closure was built and discards every step since.

Arguments

  • ::Online: The unresolved wrapper.

Validation

  • Always throws an ArgumentError.

Returns

  • Never returns.

Related

source
PortfolioOptimisers.sample_buffer_seed — Function
sample_buffer_seed(
    cache::Union{Nothing, SampleBufferState}
) -> SampleBufferState

Returns the SampleBufferState a carry folds into, seeding an empty one when the estimator carries none.

The seed is written here rather than inside partial_fit!, so the fold reads as one line and the branch that reads the cache field has one home. A family that carries its observations calls fold_buffer, which is this seed and the fold in one step; the seed it builds is uncapped, because a cap is configuration and Online is what carries it.

Arguments

  • cache: The buffer the estimator carries, or nothing.

Returns

  • state::SampleBufferState: The buffer cache holds, or an empty uncapped buffer.

Related

source
PortfolioOptimisers.fold_buffer — Function
fold_buffer(cache::Option{<:SampleBufferState}, x::VecNum, f::Option{<:VecNum} = nothing; kwargs...)
fold_buffer(cache::Option{<:SampleBufferState}, X::MatNum, F::Option{<:MatNum} = nothing; dims::Int = 1, kwargs...)

Folds observations into a sample buffer, seeding an empty one when the estimator carries none.

The one line a carry writes inside its own partial_fit! method: it folds its estimate exactly through its members, and hands its observations here. It is sample_buffer_seed followed by partial_fit!. The masks of the fold ride with the observations, so a carry hands over the mask it was given and the buffer records it beside the rows it explains; the factor rows ride on the same terms.

Arguments

  • cache: The buffer the estimator carries, or nothing.
  • x: One observation, whose entries are the assets.
  • f: The contemporaneous factor observation, whose entries are the factors, or nothing.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • F: Factor observations to fold, oriented as X is, or nothing.
  • dims: Dimension along which to perform the computation.
  • kwargs...: The active_mask and estimation_mask of the fold, forwarded to partial_fit!.

Returns

  • state::SampleBufferState: The buffer after the last observation.

Related

source
PortfolioOptimisers.assert_buffer_factor_width — Function
assert_buffer_factor_width(B::Nothing, Fo) -> Nothing
assert_buffer_factor_width(B::AbstractMatrix, Fo::AbstractMatrix) -> Nothing

Refuses a factor block whose width is not the width the first factor append fixed.

The factor twin of the width check on X, written for the backing that may be absent: a buffer that records no factor rows has no width to hold a block to, and assert_buffer_presence_agreement has already refused a block that gives rows where the buffer records none, so the nothing arm is reached only with Fo absent too.

Arguments

  • B: The factor backing the buffer holds, or nothing.
  • Fo: The oriented factor block, or nothing.

Validation

  • size(B, 2) == size(Fo, 2) when the buffer records factor rows. A DimensionMismatch is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.assert_buffer_presence_agreement — Function
assert_buffer_presence_agreement(
    M::Union{Nothing, AbstractMatrix},
    Mo::Union{Nothing, AbstractMatrix},
    name::AbstractString
)

Refuses a fold whose masks, or whose factor rows, disagree with what the buffer already records.

The mixture rule of the buffer, and it refuses rather than resolves because neither resolution is answerable. A buffer that dropped an incoming mask would fit a held gap the policy excludes and would read every delisting as a holiday, which is the silent wrong answer the mask exists to prevent. A buffer that invented a mask for the rows folded before it would claim an activity it never saw. The factor rows meet the same rule for the same reason: a buffer whose factor rows exist for some observations and not for others describes no regression, and inventing them is not on offer. So what a buffer records is fixed by its first append, and a caller that changes its mind starts a fresh buffer.

Arguments

  • M: The backing the buffer records, or nothing.
  • Mo: The same backing of the incoming block, or nothing.
  • name: Name of the backing, for the message.

Validation

  • M and Mo are both nothing, or neither is. An ArgumentError is thrown otherwise.

Returns

  • nothing.

Related

source
PortfolioOptimisers.reset_empty_buffer — Function
reset_empty_buffer(
    state::SampleBufferState,
    Ao::Union{Nothing, AbstractMatrix{<:Bool}},
    Eo::Union{Nothing, AbstractMatrix{<:Bool}},
    Fo::Union{Nothing, AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}}
) -> Any

Empties a buffer that holds no observations and does not record the masks, or the factor rows, the fold is about to give it.

The one case the mixture rule does not bite. A buffer with no rows records nothing about them, so it has no answer to lose and no claim to invent, and emptying it lets seed_sample_buffer fix the masks and the factor rows as it fixes the width and the element type. The element type of the backing matrix survives, because similar reads it off the matrix it empties.

Arguments

  • state: The buffer to empty.
  • Ao: The active mask of the incoming block, or nothing.
  • Eo: The estimation mask of the incoming block, or nothing.
  • Fo: The factor rows of the incoming block, or nothing.

Returns

  • state::SampleBufferState: The buffer unchanged, or the empty seed of the same element type and cap.

Related

source
PortfolioOptimisers.seed_sample_buffer — Function
seed_sample_buffer(
    state::SampleBufferState,
    Xo::AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}},
    Ao::Union{Nothing, AbstractMatrix{<:Bool}},
    Eo::Union{Nothing, AbstractMatrix{<:Bool}},
    Fo::Union{Nothing, AbstractMatrix{<:Union{var"#s136", var"#s53"} where {var"#s136"<:Number, var"#s53"<:AbstractJuMPScalar}}}
) -> Any

Fills the empty seed of a buffer from its first block, fixing the width, the element type, the masks and the factor rows.

The first append of a buffer, split out of partial_fit! so that the fold reads as the orientation, the truncation and the copy. The backing matrices are an exact fit for the block, so a buffer that is folded once and read once allocates nothing it does not use, and every array is built with similar from the argument it holds, so no element type is chosen here.

Arguments

  • state: The empty seed to fill.
  • Xo: The oriented block.
  • Ao: The oriented active mask of the block, or nothing.
  • Eo: The oriented estimation mask of the block, or nothing.
  • Fo: The oriented factor rows of the block, or nothing.

Returns

  • state::SampleBufferState: The buffer holding the block, its masks and its factor rows.

Related

source
PortfolioOptimisers.seed_buffer_array — Function
seed_buffer_array(Mo::Nothing) -> Nothing
seed_buffer_array(Mo::AbstractMatrix) -> AbstractMatrix

Copies an incoming block into a backing matrix of its own, and passes a backing that is not there through.

A block a fold is given is the caller's array, and the buffer outlives the call, so the buffer takes a copy for the same reason Base.copy and port_opt_view do: a later fold writes into the backing matrix, and writing through into a caller's array would be a defect the caller cannot see. The element type is read off the block with similar, so an observation, a mask and a factor row each keep their own.

Arguments

  • Mo: The oriented block, mask or factor rows, or nothing.

Returns

  • M: A backing matrix holding the block, or nothing.

Related

source
PortfolioOptimisers.copy_buffer_rows! — Function
copy_buffer_rows!(M::Nothing, rows, Mo) -> Nothing
copy_buffer_rows!(M::AbstractMatrix, rows, Mo::AbstractMatrix) -> AbstractMatrix

Copies an incoming backing into the rows of the buffer's backing matrix that were just made room for.

The optional half of the append, written once so that the two masks, the factor rows and the nothing case cost the fold one line each. It is only ever reached with the two arguments agreeing, because assert_buffer_presence_agreement has run.

Arguments

  • M: The buffer's backing matrix for this mask or the factor rows, or nothing.
  • rows: The rows of M to write.
  • Mo: The oriented mask or factor rows of the block, or nothing.

Returns

  • M: The rows written, or nothing.

Related

source
PortfolioOptimisers.reserve_sample_buffer — Function
reserve_sample_buffer(
    state::SampleBufferState,
    t::Integer
) -> Any

Makes room for t further observations in a SampleBufferState, and returns the buffer that has it.

The growth half of the buffer's fold, split out of partial_fit! so the fold reads as the orientation, the width check and the copy. It writes no observation: it drops what the cap pushes out, moves the valid region to the front of the backing matrices, and allocates larger ones when the front is not enough.

The two steps together are what makes an append amortised O(1). Dropping moves off rather than rows, so a capped buffer pays nothing for the observation it forgets, and it forgets the masks and the factor rows of that observation on the same terms because they are indexed by the same off and n. Compaction copies the valid region once, and the growth rule leaves the region filling at most half of the backing matrix, so the next compaction is at least n appends away. A capped buffer never allocates more than twice its cap, so a cap of w costs at most 2w rows.

Algorithm

  1. Drop the oldest observations the cap pushes out, by moving off forward and n back.
  2. Return when the valid region and the block already fit inside the backing matrix.
  3. Take a capacity of twice the current one, or of the room needed if that is larger, when the valid region and the block would fill more than half of the current one. Clamp it to twice the cap when a cap is set.
  4. Compact the observations, each mask and the factor rows the buffer records to that capacity with compact_buffer_array, which allocates only when the capacity changes, and set off to zero.

Arguments

  • state: The buffer to make room in.
  • t: Number of observations about to be appended.

Returns

  • state::SampleBufferState: The buffer whose backing matrices have room for t rows after their valid region.

Related

source
PortfolioOptimisers.compact_buffer_array — Function
compact_buffer_array(B::Nothing, off, n, newcap) -> Nothing
compact_buffer_array(B::AbstractMatrix, off::Integer, n::Integer, newcap::Integer) -> AbstractMatrix

Moves the valid region of one backing matrix to the front, allocating a matrix of newcap rows when the capacity changes.

The array half of reserve_sample_buffer, written once because the buffer carries four backing matrices that are indexed alike and compacted alike. Copying forward with an increasing row index is safe in place, because the destination row is never past the source row.

Arguments

  • B: The backing matrix to compact, or nothing.
  • off: Number of rows before the valid region.
  • n: Length of the valid region.
  • newcap: Number of rows the compacted matrix has.

Returns

  • D: The compacted matrix, or nothing.

Related

source
PortfolioOptimisers.observation_row — Function
observation_row(m::Nothing) -> Nothing
observation_row(m::AbstractVector) -> AbstractMatrix

Reshapes one observation, its mask or its factor observation into a one-row block, and passes one that is not there through.

The vector half of the single-observation arm of partial_fit!. It costs no copy, so one observation folded under a policy or beside its factors allocates no more than one folded without.

Arguments

  • m: One observation, one mask or one factor observation, or nothing.

Returns

  • M: The vector as a one-row block, or nothing.

Related

source
PortfolioOptimisers.merge_buffer_array — Function
merge_buffer_array(Ma::Nothing, Mb::Nothing) -> Nothing
merge_buffer_array(Ma::AbstractMatrix, Mb::AbstractMatrix) -> AbstractMatrix

Concatenates the same mask, or the factor rows, of two buffers, and passes a backing neither records through.

The optional half of merge_states. It is only ever reached with the two arguments agreeing, because assert_buffer_presence_agreement has run.

Arguments

  • Ma: The backing of the first buffer, or nothing.
  • Mb: The backing of the second buffer, or nothing.

Returns

  • M: The two backings stacked, or nothing.

Related

source
PortfolioOptimisers.trim_merged_array — Function
trim_merged_array(M::Nothing, rows) -> Nothing
trim_merged_array(M::AbstractMatrix, rows) -> AbstractMatrix

Keeps the rows a cap admits of a merged mask or of the merged factor rows, by index copy, and passes a backing that is not there through.

The cap half of merge_states. It copies rather than views for the reason port_opt_view does: the merged buffer is appended to afterwards, and a view would write through into the matrix vcat built.

Arguments

  • M: The merged backing, or nothing.
  • rows: The rows the cap admits.

Returns

  • M: The admitted rows, or nothing.

Related

source
Base.copy — Method
copy(x::SampleBufferState) -> SampleBufferState

Copies a SampleBufferState, so the copy shares no array with the original.

The copy method of the AbstractPartialFitState interface, which partial_fit calls before it folds. Each backing matrix is copied whole, spare capacity included, so the copy appends on the same terms as the original, and the masks and the factor rows stay indexed by the same off and n as the observations.

Arguments

  • x: The buffer to copy.

Returns

  • state::SampleBufferState: A fresh buffer, equal to x, whose backing matrices are fresh matrices.

Related

source
PortfolioOptimisers.copy_buffer_array — Function
copy_buffer_array(M::Nothing) -> Nothing
copy_buffer_array(M::AbstractMatrix) -> AbstractMatrix

Copies one backing matrix of a buffer, and passes a matrix that is not there through.

The nothing arm of copy for the two masks and the factor rows, written once so that Base.copy and port_opt_view read as one line per field.

Arguments

  • M: The backing matrix to copy, or nothing.

Returns

  • M: A fresh matrix equal to it, or nothing.

Related

source
PortfolioOptimisers.slice_buffer_mask — Function
slice_buffer_mask(M::Nothing, i) -> Nothing
slice_buffer_mask(M::AbstractMatrix{<:Bool}, i) -> AbstractMatrix{<:Bool}

Slices one backing mask of a buffer to the selected assets, and passes a mask that is not there through.

The mask arm of port_opt_view for this state, split out so that the slice reads as one line per field. It copies by index for the reason that method states.

Arguments

  • M: The backing mask to slice, or nothing.
  • i: Index or indices of the assets to keep.

Returns

  • M: The mask over the selected assets, or nothing.

Related

source
PortfolioOptimisers.supports_partial_fit — Function
supports_partial_fit(est) -> Bool

Answers whether partial_fit! on this estimator folds rather than refuses.

A host that carries the observations asks this of each of its members, and this is the whole of the mixed-host rule: a host folds every member that folds, and runs the batch verb over its own rows for every member that does not. So a caller writes the estimator they would write in batch, needs no wrapper at the call site, and no member carries a second copy of the sample.

The default is the buffering route: an estimator folds when it carries a SampleBufferState, which Online is what seeds. A family with an exact fold of its own adds a method returning true, beside that fold and under the same type bound, minus the cache parameter — because an estimator of that family folds either way, exactly when its bound matches and by buffering when a wrapper seeded one. A configuration a family refuses — the SemiMoment arms, whose clip moves when the mean moves — therefore adds no method and falls back to the default, which is true exactly when a wrapper gave it somewhere to buffer.

The predicate reads a type and a field, never a method table. Reading dispatch would classify a refusal as a fold, because a refusal is a method too, and those are precisely the members that must buffer instead.

Arguments

  • est: The estimator to ask about.

Returns

Related

source
supports_partial_fit(
    ce::PortfolioOptimisersCovariance
) -> Any

PortfolioOptimisersCovariance method of supports_partial_fit.

The composite folds by composition, so it folds exactly when both of its halves allow it: the inner estimator must fold, and mp must carry no sample-reading alg. A composite a wrapper has given a buffer folds whatever those two say, which is the default and is what makes an mp.alg reachable online at all.

Arguments

  • ce: Covariance estimator.

Returns

Related

source
PortfolioOptimisers.online_candidate_fields — Function
online_candidate_fields(x)

Field names of x whose type admits an Online — the candidate set online_fields narrows by value.

Whether a field holds a wrapper is decidable from fieldtype alone, because a @concrete host records the value's type in the field's type parameter, so a field holding a static estimator cannot have a type intersecting Online. The tuple is therefore computed once per host type by a generated function, and a host that carries no wrapper folds to an empty tuple at compile time rather than walking every field at warm-up.

Related

source
PortfolioOptimisers.online_fields — Function
online_fields(
    est::Union{AbstractEstimator, CovarianceEstimator}
) -> Any

Returns the tuple of field names of est whose values are Online.

The scan is generic over the host's fields, so the widened constructor signatures (see Online_Option) remain the single source of truth for which estimators may take the step from a buffer — there is no hand-maintained list. Only the fields whose type admits a wrapper are visited (see online_candidate_fields); the rest are ruled out at compile time.

Arguments

  • est: Estimator whose fields are scanned.

Returns

  • fns::Tuple: The field names holding an Online.

Related

source
PortfolioOptimisers.online_state_seed — Method
online_state_seed(
    _::Union{AbstractEstimator, CovarianceEstimator},
    max_history::Union{Nothing, Integer}
) -> Union{ReturnsBufferState{Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, ReturnsBufferState{Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, __T_max_history} where __T_max_history<:Integer}

Builds the empty state an Online seeds into the estimator it wraps.

One sample buffer answers every estimator that refits, so this is that buffer and the seeding needs no per-type knowledge: a prior whose batch verb reads a factor matrix beside the returns records the factor rows inside the same buffer, and whether its fold is given them is decided by needs_factor_returns rather than by the seed. A host whose state is not a sample buffer at all — a prior-less optimiser head, whose state is a fold context — writes a method of its own returning the state its read-out reads, and nothing else about the wrapper changes.

The hook takes the estimator rather than its type, so a family that sizes its seed from a field can read it.

Arguments

  • est: The estimator the wrapper wraps, read for its type.
  • max_history: The wrapper's cap, carried into the state.

Returns

  • state::AbstractPartialFitState: The empty state to seed, carrying the cap and no observations.

Related

source
PortfolioOptimisers.update_online_estimator — Function
update_online_estimator(est)

Resolves the Online declarations of an estimator, seeding the sample buffer each one asks for.

Called once, at warm-up, before the first fold. Each wrapper is replaced by the estimator it wraps, rebuilt through its keyword constructor with cache holding an empty SampleBufferState carrying the wrapper's cap; the host is then rebuilt through its own keyword constructor, so every construction invariant re-runs. The result holds no Online, and the estimators that were wrapped carry the buffer their refit or their carry reads.

A wrapper is recognised in the fields of the estimator handed over, and in the fields of an estimator a wrapper wraps. A host that hands estimators across a boundary of its own — a meta-optimiser, a pipeline — writes a method of its own that recurses, exactly as it does for update_time_dependent_estimator.

The verb takes no fold context, and that is the difference from update_time_dependent_estimator restated: seeding reads nothing from a fold, because the wrapper resolves once rather than per fold.

Arguments

  • est: Estimator, wrapper, or nothing.

Returns

  • Estimator carrying a seeded buffer wherever a wrapper stood, and holding no Online.

Related

source
update_online_estimator(pe::Union{<:HighOrderPriorEstimator, <:BlackLittermanPrior})

Resolves the Online declarations under the prior a wrapping prior embeds, at warm-up.

A wrapping prior hands its embedded prior across a boundary of its own, so it writes the recursion the generic method does not: HighOrderPriorEstimator(; pe = BlackLittermanPrior(; pe = Online(EmpiricalPrior()), …)) is resolved through this method, where the generic scan of the host's own fields would find no wrapper — pe holds a plain prior — and leave the one two levels down unseeded. A wrapper stored in the host's own pe, HighOrderPriorEstimator(; pe = Online(EmpiricalPrior())), is seeded the same way, because the recursion meets it at the first level. The co-moments of a HighOrderPriorEstimator are scanned by the generic method as they always were.

Arguments

  • pe: The wrapping prior.

Returns

  • pe: The prior, with every wrapper under its embedded prior resolved.

Related

source
update_online_estimator(opt::JuMPOptimisationEstimator)
update_online_estimator(opt::Union{<:HierarchicalRiskParity, <:HierarchicalEqualRiskContribution, <:SchurComplementHierarchicalRiskParity})
update_online_estimator(opt::Union{<:JuMPOptimiser, <:HierarchicalOptimiser, <:InverseVolatility, <:NestedClustered, <:Stacking, <:SubsetResampling})

Resolves the Online declarations an optimiser carries in its prior, at warm-up.

An optimiser hands its prior across a boundary of its own, so it writes the recursion the generic method does not: the JuMP and hierarchical heads recurse into the bundle they hold, and the bundle, the meta-optimisers and InverseVolatility recurse into pe. The forward reaches the prior alone, as the step does — a wrapper inside an inner optimiser of a meta-optimiser, or inside a fallback, is a batch configuration of that optimiser and is not resolved here.

Arguments

  • opt: The optimiser.

Returns

  • opt: The optimiser, with every wrapper under its prior resolved to an estimator carrying a seeded buffer.

Related

source
update_online_estimator(p::Pipeline)

Resolves the Online declarations a Pipeline carries in its steps, at warm-up.

On the host route every step is resolved through update_online_step, so an Online(EmpiricalPrior(); max_history = w) step becomes the prior carrying its buffer, and an optimisation step resolves the wrappers under its own prior. On the refit route — the pipeline carries a PipelineBufferState, which Online(pipe) seeded — no member is seeded, because no member folds under a refit; assert_online_entry has refused any wrapper below the root by name already.

Related

source
PortfolioOptimisers.estimator_fields — Function
estimator_fields(x)

Field names of x whose type is an estimator — the candidate set online_entry_state walks.

Whether a field holds an estimator is decidable from fieldtype alone, because a @concrete host records the value's type in the field's type parameter, so the tuple is computed once per host type by a generated function, exactly as online_candidate_fields is. A field holding a vector of estimators, a result or a schedule is not a candidate: a vector is a batch configuration the step never folds into, a result carries no state field, and a schedule's entries are resolved per fold.

Related

source
PortfolioOptimisers.online_entry_state — Function
online_entry_state(est)
online_entry_state(::TimeDependent)

Name the first field of an estimator tree that carries a partial-fit state, or answer nothing.

The predicate behind the fold loop's cold start: the online arm of fold_loop reads its argument as the configuration alone, so an estimator that enters it holding a state is refused by name, and this is the walk that finds the state. It answers "cache" when est holds one, and otherwise descends into every estimator-valued field (estimator_fields) and prefixes the field's name to what it finds there, so the answer is the path from the root — "opt.pe.me.cache" for a mean estimator's state under a JuMP head. A TimeDependent schedule answers nothing, because its entries are batch configuration resolved per fold and the loop threads no state through them, and so does anything that is not an estimator.

Arguments

  • est: The estimator, or any value a field holds.

Returns

  • path::Option{<:String}: The dotted path of the first state found, or nothing.

Related

source
online_entry_state(o::Online)

Name the state a wrapped estimator carries at the entry of the fold loop's online arm, or answer nothing.

The Online arm of online_entry_state, for the wrapper at the root of covariance_forecast_evaluation: a wrapper holds no cache of its own, so the walk descends into the estimator it wraps and prefixes est to what it finds there, est.cache for a covariance estimator handed over already folded. The generic walk would reach the same answer through estimator_fields; this arm states it without asking the wrapper for a field it does not have.

Related

source
assert_online_entry(est::TimeDependent)
assert_online_entry(est)

Refuse an estimator that is not the configuration alone at the entry of the fold loop's online arm.

Three refusals, all before any solve. A TimeDependent schedule of optimisers at the root, because the loop threads one estimator and a schedule is a different one per fold. A schedule on a host's pe, the one stateful field whose bound admits one, through assert_stateless_schedule. And a partial-fit state anywhere in the tree, through online_entry_state: the loop starts cold. The batch loop already reads its argument as configuration alone — factory carries a state and prior(pe, X) never reads it — and so does this one. A reset was costed at one "empty, keep the cap" verb per state type, and folding the warm-up on top of what the estimator holds double-counts rows in silence; both were rejected. A resume, a state that leaves a result and re-enters a loop, is its own ticket with an explicit entry.

Arguments

  • est: The estimator handed to the loop.

Validation

  • est is not a TimeDependent. An ArgumentError is thrown otherwise.
  • No pe on the route the step takes holds a TimeDependent. An ArgumentError is thrown otherwise.
  • No cache in the tree of est holds a state. An ArgumentError naming the field is thrown otherwise.

Related

source
online_entry_state(p::Pipeline)

Names the first state a Pipeline carries at the entry of the fold loop's online arm — its own cache, or a state anywhere under a step, prefixed by the step's name — or answers nothing.

Related

source
PortfolioOptimisers.online_wrapper_path — Function
online_wrapper_path(est)
online_wrapper_path(::Online)
online_wrapper_path(::TimeDependent)

Name the first field of an estimator tree that holds an Online declaration, or answer nothing.

The predicate behind assert_batch_entry: a wrapper is resolved once, at the warm-up of the fold loop's online arm (update_online_estimator), and nothing else resolves it, so a batch fit that reaches one meets the method table at prior(pe, X) rather than a refusal. The walk is online_entry_state's: it descends into every estimator-valued field (estimator_fields) and prefixes the field's name to what it finds there, so the answer is the path from the root — "pe" for a wrapped prior on a naive optimiser, "opt.pe" for one under a JuMP head. A field holding a wrapper answers its own name and is not entered, because a wrapper below it is the warm-up's to resolve under it; a wrapper at the root answers "", the empty path. A TimeDependent schedule answers nothing, because a wrapper among its entries is refused at its construction, and so does anything that is not an estimator.

Arguments

  • est: The estimator, or any value a field holds.

Returns

  • path::Option{<:String}: The dotted path of the first wrapper found, "" when est is itself one, or nothing.

Related

source
PortfolioOptimisers.assert_batch_entry — Function
assert_batch_entry(est, door::AbstractString)

Refuse an estimator holding an Online at the door of a batch fit, by name.

A wrapper is a declaration the online arm of the fold loop resolves at its warm-up, and a batch fit — a plain optimise, or a fold of a scheme that declares no Fold Fit — runs no warm-up, so the wrapper would reach prior(pe, X) unresolved and meet a MethodError naming the whole type. The refusal names the dotted path online_wrapper_path finds and the two exits: an OnlineStep on a walk-forward, or the estimator unwrapped. A root that is itself a wrapper is refused by the door that admits one — covariance_forecast_evaluation, and the Pipeline's — so this reads the path alone.

Arguments

  • est: The estimator handed to the door.
  • door: The door's name, as the message reads it.

Validation

  • No field in the tree of est holds an Online. An ArgumentError naming the field is thrown otherwise.

Related

source