Online (a): private API

struct SampleBufferState{__T_n, __T_off, __T_X, __T_A, __T_M, __T_F, __T_ne, __T_E, __T_iv, __T_ivpa, __T_P, __T_max_history} <: AbstractPartialFitState

Carries the observations an estimator keeps when its estimate has no exact incremental fold.

This is the buffer of the partial_fit! interface. It is a state like any other. It lives in the cache field of the estimator, it copies, it slices by asset, and obs_weights_view drops it. It holds the observations verbatim, NaN included. So the call with no data over the buffer gives the answer of a batch fit over the same rows, and the two have the same Coverage Universe by construction.

The buffer also holds the per-observation masks verbatim. A CoveragePolicy reads two facts from an active mask that the rows alone do not carry. A cell that is finite but inactive is excluded, and an asset that is active at one observation and inactive at the next is a delisting, not a holiday. A buffer that kept the rows and dropped the mask would give a different answer from the batch fit, with no warning. A is the active mask, and M is the estimation mask that the two regime-adjusted families read. Each is a backing matrix of the shape of X, or nothing. So a wrapped estimator folded under a policy gives the batch fit over the same window, as it does without a policy.

The buffer holds the factor observations on the same terms. A prior whose batch verb is prior(pe, X, F) regresses the asset returns on the factor returns, so its refit reads two matrices whose t-th rows are contemporaneous. F is the second matrix, a backing matrix of capacity × factors, or nothing. The same off and n index F and X, so the rows stay contemporaneous by construction. needs_factor_returns answers whether a fit reads F. That is a fact of the estimator tree, and the buffer records what the fold gives it. There is one buffer type, and a cap drops the factor rows with the other rows.

The buffer holds the Panel Fields of a time-varying AssetPanel on the same terms, for a prior whose estimator tree reads them. reads_panel_fields answers whether a tree reads them. Such a prior computes its factors from the Panel Fields at each row, so for it a Panel Field is sample and not fold context. P is the vector of the Panel Fields over the valid region alone. The same n and the same cap index P, and off does not. An append joins the rows with vcat of two Asset Panels, so it copies the rows that the buffer holds and costs O(n). A refit reads every row at each step, so this cost is smaller than the cost of its call with no data. A panel is a Result and nothing writes into its arrays, so the buffer holds the Panel Fields by reference, and a copy of the buffer shares them. A time-varying panel carries both masks, so a buffer that records P records A and M too.

The buffer holds the Exogenous Series on the same terms, for a prior whose estimator tree reads it. reads_exogenous_series answers whether a tree reads it. An observed factor reads its return from the series by name, so for such a prior the series is sample and not fold context. E is a backing matrix of capacity × series, and ne holds the name of each column. The buffer keeps every column, not only the columns that the tree names, so the Fold Context keeps no copy. The same off and n index E and X. So E means the Exogenous Series on the whole online step, as it does on a ReturnsResult.

The buffer holds the implied volatilities on the same terms as the masks, for every estimator. A row of iv belongs to the asset observation of the same row, so iv is a backing matrix of the shape of X, and ImpliedVolatility reads it beside the returns. Its regression reads windows that end at the last observation, so a new row moves every window, and no statistic of the regression folds. The call with no data refits it over the rows of the buffer. ivpa is one adjustment for each asset or one for all, so it has no observation axis, and the first append pins it.

The first append fixes whether the buffer records a mask, whether it records factor rows, whether it records the Exogenous Series, whether it records implied volatilities and whether it records Panel Fields, as it fixes the width and the element type. It also fixes the names of the series and the adjustment ivpa. A later fold that disagrees is refused with an error that names the disagreement, in both directions. A buffer that holds no observations records nothing about them, so the next fold seeds it again from its block.

The buffer marks the observations of a Shared Series gap in G, and keeps them. A Shared Series is a vector benchmark of a group that reads the Excess Frame, or the factor returns of a prior that reads them, and a batch fit drops an observation with a gap in one of them, see shared_series_view. An online step cannot drop the row, because the buffer of a prior and the Fold Context of its optimiser hold one row each for every observation, and the call with no data rebuilds the returns data from both. So the step keeps the row and marks it, and every fit through the buffer reads the rows of shared_series_buffer, which drops the marked rows. A cap counts every row, as the training window of a batch walk-forward counts the observations before the drop. A fold that marks no gap leaves G at nothing, so a buffer that never meets a gap records nothing.

An estimator whose cache holds this buffer is a refit. It has no recursion of its own, and its call with no data runs the batch verb over the rows of the buffer. Online seeds the buffer, and a wrapped estimator takes this route also when its statistic has an exact update. A fold-and-carry prior keeps its observations in one of these buffers too, inside a PriorCarryState. That prior folds its moments exactly. It keeps the rows because LowOrderPrior carries X for the scenario risk measures, and its buffer has no cap. max_history is the window of a refit. A capped buffer holds the last max_history observations, and the call with no data gives the batch fit over them.

The backing matrix has spare capacity, so an append costs amortised O(1). off is the number of rows before the valid region, and n is the length of the region. reserve_sample_buffer moves the region to the front of the backing matrix only when an append would go past its end. sample_buffer reads the valid region.

Fields

• n: Number of observations folded into the state.

• off: Number of rows of X before the valid region. It rises when a capped buffer drops its oldest observation, and it returns to zero when the valid region moves to the front of X.

• X: Backing matrix of the buffer, capacity × assets. Rows off + 1 to off + n are the observations, in the order of the folds. The other rows are spare capacity, and their values have no meaning. The first append fixes the width and the element type.

• A: Backing matrix of the active mask, of the shape of X, or nothing when the buffer records no activity. Rows off + 1 to off + n are the mask of the observations, cell for cell with them. It is fixed by the first append, and it is what lets the call with no data tell a delisting from a holiday.

• M: Backing matrix of the estimation mask, of the shape of X, or nothing when the buffer records none. It is the second per-observation mask the batch verbs take, it is carried on the same terms as A, and only the two regime-adjusted families read it.

• F: Backing matrix of the factor observations, capacity × factors, or nothing when the fold carries none. The same off and n index it and X. The first append that carries it fixes its width and its element type.

• ne: Names of the columns of E, or nothing when the fold carries no Exogenous Series. The first append that carries the series fixes them.

• E: Backing matrix of the Exogenous Series, capacity × series, or nothing when the fold carries none. The same off and n index it and X. It holds every column of the series that the fold receives.

• iv: Backing matrix of the implied volatilities, capacity × assets, or nothing when the fold carries none. The same off and n index it and X.

• ivpa: The implied volatility risk premium adjustment, one number or one entry per asset, or nothing. The first append pins it.

• P: The Panel Fields of the observations, or nothing when the fold carries none. Each is a time-varying Panel Field over the valid region alone, so n and the cap index it and off does not. The first append that carries them fixes their names and their kinds.

• G: Backing vector of the Shared Series gaps, capacity long, or nothing when no fold marked a gap. Entry off + t is true when observation t has a gap in a Shared Series that the fit reads, so every fit through the buffer drops it, see shared_series_buffer. The same off and n index it and X. The field has a declared type and is not a type parameter, so the type of the buffer, and of every estimator that holds it, does not depend on whether a step met a gap. A step with no gap then compiles to the code it had before the marks existed.

• max_history: Optional cap on the number of observations the buffer keeps. nothing keeps every observation folded so far. A capped buffer drops its oldest observations as new ones arrive and holds the last max_history of them. The call with no data reads those rows only. The cap is the window of the fit. An estimator wrapped in Online returns the batch fit over the last max_history observations, also when its statistic has an exact update.

Constructors

SampleBufferState(;    n::Integer = 0,    off::Integer = 0,    X::MatNum = Matrix{Float64}(undef, 0, 0),    A::Option{<:AbstractMatrix{<:Bool}} = nothing,    M::Option{<:AbstractMatrix{<:Bool}} = nothing,    F::Option{<:MatNum} = nothing,    ne::Option{<:VecStr} = nothing,    E::Option{<:MatNum} = nothing,    iv::Option{<:MatNum} = nothing,    ivpa::Option{<:Num_VecNum} = nothing,    P::Option{<:AbstractVector} = nothing,    G::Option{<:AbstractVector{Bool}} = nothing,    max_history::Option{<:Integer} = nothing) -> SampleBufferState

Keywords correspond to the struct's fields. The default is the empty seed that Online builds. It carries the cap and no observations. The first append replaces every backing matrix, so the width, the element type, the masks, the factor rows, the Exogenous Series, the implied volatilities and the Panel Fields come from the observations of that append.

Validation

• n >= 0. A DomainError is thrown otherwise.
• off >= 0. A DomainError is thrown otherwise.
• off + n <= size(X, 1). A DimensionMismatch is thrown otherwise.
• max_history > 0 when it is not nothing. A DomainError is thrown otherwise.
• n <= max_history when max_history is not nothing. A DimensionMismatch is thrown otherwise.
• A, M and iv, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.
• F, when it is not nothing, has as many rows as X. A DimensionMismatch is thrown otherwise.
• The rules of assert_buffer_series_shape hold for ne and E.
• P, when it is not nothing, holds time-varying Panel Fields over n observations and the assets of X. The rules of assert_buffer_panel_shape apply.
• G, when it is not nothing, has one entry for each row of X. A DimensionMismatch is thrown otherwise.

View parameters

When port_opt_view is called on this type, its fields are subset to the selected assets:

• X: Sliced to the selected indices via port_opt_view.
• A: Sliced to the selected indices via port_opt_view.
• M: Sliced to the selected indices via port_opt_view.
• F: Copied unchanged, because the selection indexes assets and this backing describes factors.
• ne, E: Copied unchanged, because the selection indexes assets and the Exogenous Series belongs to no asset.
• iv: Sliced to the selected indices via port_opt_view.
• ivpa: Sliced to the selected indices when it is a vector, and kept when it is one number or nothing.
• P: Each Panel Field viewed over the selected assets with panel_fields_view.
• G: Copied unchanged, because the selection indexes assets and a Shared Series gap belongs to the observation.

A buffer that holds no observations views to the empty seed with the same cap.

Examples

julia> PortfolioOptimisers.SampleBufferState(; max_history = 2)PortfolioOptimisers.SampleBufferState            n ┼ Int64: 0          off ┼ Int64: 0            X ┼ 0×0 Matrix{Float64}            A ┼ nothing            M ┼ nothing            F ┼ nothing           ne ┼ nothing            E ┼ nothing           iv ┼ nothing         ivpa ┼ nothing            P ┼ nothing            G ┼ nothing  max_history ┴ Int64: 2

Related

• AbstractPartialFitState
• Online
• PriorCarryState
• sample_buffer
• sample_buffer_kwargs
• factor_buffer
• fold_buffer
• partial_fit!
• CoveragePolicy
• needs_factor_returns
• reads_panel_fields
• reads_exogenous_series
• sample_buffer_panel
• exogenous_buffer_kwargs
• implied_volatility_buffer_kwargs
• shared_series_buffer

assert_sample_buffer_state(
    n::Integer,
    off::Integer,
    X::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    max_history::Union{Nothing, Integer}
)

Refuses a sample buffer whose valid region does not sit inside its backing matrix.

Algorithm

1. Refuse a negative n and a negative off.
2. Refuse a valid region that runs past the last row of X.
3. Return when max_history is nothing.
4. Refuse a non-positive max_history, and refuse a valid region longer than it.

Arguments

• n: Number of observations folded into the state.
• off: Number of rows of X before the valid region.
• X: Backing matrix of the buffer.
• max_history: Optional cap on the number of observations the buffer keeps. nothing keeps every observation folded so far. A capped buffer drops its oldest observations as new ones arrive and holds the last max_history of them. The call with no data reads those rows only. The cap is the window of the fit. An estimator wrapped in Online returns the batch fit over the last max_history observations, also when its statistic has an exact update.

Validation

• n >= 0. A DomainError is thrown otherwise.
• off >= 0. A DomainError is thrown otherwise.
• off + n <= size(X, 1). A DimensionMismatch is thrown otherwise.
• max_history > 0 when it is not nothing. A DomainError is thrown otherwise.
• n <= max_history when max_history is not nothing. A DimensionMismatch is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState

assert_buffer_mask_shape(
    X::AbstractMatrix,
    A::Union{Nothing, AbstractMatrix{<:Bool}},
    M::Union{Nothing, AbstractMatrix{<:Bool}}
)
assert_buffer_mask_shape(
    X::AbstractMatrix,
    A::Union{Nothing, AbstractMatrix{<:Bool}},
    M::Union{Nothing, AbstractMatrix{<:Bool}},
    iv::Union{Nothing, AbstractMatrix}
)

Refuses a per-observation mask, or a matrix of implied volatilities, whose shape is not the shape of the observations it belongs to.

A mask that a buffer holds matches its observations cell for cell, and a mask that a fold receives matches its block cell for cell. So one rule covers both. A mask is absent, or it has the shape of the matrix beside it. The implied volatilities follow the same rule, because each cell belongs to the asset observation of the same cell. The batch verbs run the same check on their own arguments.

Algorithm

1. Skip a mask that is nothing. With no policy, both masks are nothing.
2. Refuse a mask whose size is not size(X), and name the mask. Apply the same rule to iv.

Arguments

• X: The observations the masks belong to.
• A: The active mask, or nothing.
• M: The estimation mask, or nothing.
• iv: The implied volatilities, or nothing.

Validation

• A, M and iv, when they are not nothing, have the shape of X. A DimensionMismatch is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• partial_fit!
• CoveragePolicy

assert_buffer_factor_shape(
    X::AbstractMatrix,
    F::Union{Nothing, AbstractMatrix}
)

Refuses a factor block whose rows are not the rows of the observations it belongs to.

A factor observation is contemporaneous with the asset observation of the same row. So a factor block that a fold receives has one row for each row of the block beside it, and the factor backing of a buffer has one row for each row of X. The check does not compare the two widths, because an asset count and a factor count are unrelated.

Algorithm

1. Skip a factor block that is nothing. With no factors, the block is nothing.
2. Refuse a factor block whose row count is not size(X, 1).

Arguments

• X: The observations the factor rows belong to.
• F: The factor rows, or nothing.

Validation

• F, when it is not nothing, has as many rows as X. A DimensionMismatch is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• partial_fit!
• needs_factor_returns

assert_buffer_gap_shape(
    X::AbstractMatrix,
    G::Union{Nothing, AbstractVector{Bool}}
)

Refuses marks of the Shared Series gaps that are not one mark for each row of the observations.

A mark belongs to the observation of the same row, so the marks that a fold receives have one entry for each row of its block, and the backing vector of a buffer has one entry for each row of X.

Algorithm

1. Skip marks that are nothing.
2. Refuse marks whose length is not size(X, 1).

Arguments

• X: The observations that the marks belong to.
• G: The marks, true at an observation with a Shared Series gap, or nothing.

Validation

• G, when it is not nothing, has one entry for each row of X. A DimensionMismatch is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• shared_series_buffer

assert_buffer_series_shape(
    X::AbstractMatrix,
    ne::Union{Nothing, AbstractVector{<:AbstractString}},
    E::Union{Nothing, AbstractMatrix}
)

Refuses an Exogenous Series whose rows are not the rows of the observations it belongs to, or whose names do not name its columns.

A row of the series is contemporaneous with the asset observation of the same row, as a factor row is. The names travel with the series, so the two are both absent or both present, and there is one name for each column.

Algorithm

1. Refuse names without a series, and a series without names.
2. Skip a series that is nothing.
3. Refuse a series whose row count is not size(X, 1), and names whose count is not the column count of the series.

Arguments

• X: The observations the series belongs to.
• ne: The names of the series, or nothing.
• E: The series, observations × series, or nothing.

Validation

• ne and E are both nothing, or neither is. An ArgumentError is thrown otherwise.
• E, when it is not nothing, has as many rows as X, and length(ne) == size(E, 2). A DimensionMismatch is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• partial_fit!
• reads_exogenous_series

buffer_rows_view(M::Nothing, rows) -> Nothing
buffer_rows_view(M::AbstractMatrix, rows) -> SubArray
buffer_rows_view(G::AbstractVector, rows) -> SubArray

Reads a row range out of a per-observation backing, and passes a backing that is not there through.

The buffer slices a mask or the factor rows on the observation axis only through this function, so the nothing case has one method. sample_buffer_kwargs, factor_buffer and merge_states read the valid region of a stored backing with it, and partial_fit! truncates an incoming block with it.

Arguments

• M: The backing to slice, or nothing. A vector is the backing of the marks of the Shared Series gaps, one entry per row.
• rows: The rows to keep.

Returns

• M: The rows of the backing, or nothing.

Related

• SampleBufferState
• sample_buffer_kwargs
• factor_buffer
• partial_fit!

sample_buffer(state::SampleBufferState)
sample_buffer(est)

Reads the observations a sample buffer holds, observations × assets.

This function reads the rows out of a SampleBufferState. It is a view of the valid region of the backing matrix, in the order of the folds, so a refit reads it with no copy. The estimator form reads the state from the cache field, and it refuses an estimator that carries no buffer.

Arguments

• state: The buffer to read.
• est: Estimator whose cache field carries the buffer.

Validation

• est.cache holds a SampleBufferState. An ArgumentError is thrown otherwise.

Returns

• X::SubArray: The observations the buffer holds, observations × assets.

Related

• SampleBufferState
• assert_sample_buffer
• Online

sample_buffer(state::PriorCarryState) -> Any

Reads the observations that a PriorCarryState carries, observations × assets.

Arguments

• state: The carry state to read.

Returns

• X::SubArray: The observations that the state carries, in the order of the fold.

Related

• PriorCarryState
• sample_buffer

sample_buffer_kwargs(state::SampleBufferState) -> NamedTuple

Reads the per-observation masks a sample buffer holds, as the keywords a batch verb takes.

sample_buffer gives the rows, and this function gives the masks of those rows. The call with no data runs the batch verb on the rows and splats these keywords into the call, so the wrapped answer is the unwrapped answer over the same window. A buffer that records no mask gives an empty set of keywords, and the arm then makes the same call as with no policy. The branch reads fields of concrete type, so the compiler resolves it.

Algorithm

1. Slice each stored mask to the valid region with buffer_rows_view.
2. Name the masks that are there, and omit the ones that are not.

Arguments

• state: The buffer to read.

Returns

• kwargs::NamedTuple: The active_mask and estimation_mask keywords the buffer holds, of which either or both may be absent.

Related

• SampleBufferState
• sample_buffer
• buffer_rows_view
• CoveragePolicy

factor_buffer(state::SampleBufferState) -> Any

Reads the factor observations a sample buffer holds, observations × factors, or nothing when it holds none.

This is the third function that reads the buffer out, with sample_buffer and sample_buffer_kwargs. A refit whose batch verb reads a factor matrix takes it from here, as the second positional argument of that verb. Its t-th row is contemporaneous with the t-th row of sample_buffer, because the same off and n index the two backings. A buffer that records no factor rows gives nothing, which is the value that the batch verb of a prior with no factors receives. The branch reads a field of concrete type, so the compiler resolves it.

Arguments

• state: The buffer to read.

Returns

• F: The factor observations the buffer holds, as a view of the valid region, or nothing.

Related

• SampleBufferState
• sample_buffer
• sample_buffer_kwargs
• buffer_rows_view
• needs_factor_returns

exogenous_buffer_kwargs(
    state::SampleBufferState
) -> Union{@NamedTuple{}, NamedTuple{(:ne, :E), <:Tuple{Any, Any}}}

Reads the Exogenous Series a sample buffer holds, as the keywords a batch verb takes.

This is the fourth function that reads the buffer out. The call with no data of a refit splats these keywords into the batch verb, as prior(pe, rd) passes rd.ne and rd.E. The Fold Context reads them too, when the prior owns the series. Each row of E is contemporaneous with the row of sample_buffer of the same index, because the same off and n index the two backings. A buffer that records no series gives an empty set of keywords. The branch reads a field of concrete type, so the compiler resolves it.

Arguments

• state: The buffer to read.

Returns

• kwargs::NamedTuple: ne and E, the names and a view of the valid region of the series, or an empty NamedTuple.

Related

• SampleBufferState
• sample_buffer
• factor_buffer
• reads_exogenous_series

implied_volatility_buffer_kwargs(
    state::SampleBufferState
) -> Union{@NamedTuple{}, NamedTuple{(:iv, :ivpa), <:Tuple{Any, Any}}}

Reads the implied volatilities a sample buffer holds, and the pinned adjustment, as the keywords a batch verb takes.

This is the fifth function that reads the buffer out. The call with no data of a refit splats these keywords into the batch verb, as prior(pe, rd) passes rd.iv and rd.ivpa, and the Fold Context reads them back into the ReturnsResult that it rebuilds. Each row of iv belongs to the row of sample_buffer of the same index, because the same off and n index the two backings. A buffer that records neither gives an empty set of keywords.

Arguments

• state: The buffer to read.

Returns

• kwargs::NamedTuple: iv, a view of the valid region of the implied volatilities or nothing, and ivpa, or an empty NamedTuple.

Related

• SampleBufferState
• sample_buffer
• exogenous_buffer_kwargs
• ImpliedVolatility

buffer_gap_marks(state::SampleBufferState) -> Any

Reads the marks of the Shared Series gaps of the valid region of a SampleBufferState.

A buffer with no marks has no gap, so it gives false for each of its observations. That answer allocates a vector, so a reader on the path of every step tests isnothing(state.G) first, as buffer_fit_count, buffer_fits_none and shared_series_buffer do. merge_buffer_gaps reads it only when one of the two buffers has marks, and the merge copies every row anyway.

Arguments

• state: The buffer to read.

Returns

• marks::AbstractVector{Bool}: One mark for each observation of the buffer, true at an observation with a Shared Series gap.

Related

• SampleBufferState
• shared_series_buffer
• merge_buffer_gaps

buffer_fit_count(state::SampleBufferState) -> Any

Count the observations of a SampleBufferState that a fit keeps, the rows that shared_series_buffer returns.

A reader that holds a history of the fitted observations beside the buffer, as the carry of a CrossSectionalFactorPrior does, indexes that history by this count and not by n, because the history never holds an observation with a Shared Series gap.

Arguments

• state: The buffer to read.

Returns

• n::Int: The number of observations of the buffer with no Shared Series gap.

Related

• shared_series_buffer
• buffer_gap_marks

buffer_kept_rows(
    state::SampleBufferState
) -> Union{Nothing, Vector{Int64}}

Return the observations of a SampleBufferState that a fit keeps, or nothing when it keeps every observation.

A fit through the buffer drops each observation with a Shared Series gap, see shared_series_buffer, so it counts the position of a row among the kept observations. Kept row k is observation rows[k] of the buffer. A reader that records the position of a fitted row maps it through rows, as shared_series_record does. A buffer with no gap answers nothing and allocates nothing.

Arguments

• state: The buffer to read.

Returns

• rows::Option{Vector{Int}}: The observations of the buffer with no Shared Series gap, or nothing when no observation has a gap.

Related

• buffer_fit_count
• buffer_gap_marks
• shared_series_record

buffer_fits_none(state::SampleBufferState) -> Bool

Answer true when every observation of a SampleBufferState has a Shared Series gap, so a fit through the buffer keeps no observation.

A reader that fits on the buffer after each step, as the Choice Rule of a refit does, asks this first, because shared_series_buffer refuses such a buffer and the first steps of a run can hold only gaps. A buffer with no marks answers false at once and allocates nothing, so the step of a buffer that never meets a gap costs what it cost before the marks existed. A buffer that holds no observation and no mark answers false too, as it did before.

Arguments

• state: The buffer to read.

Returns

• ans::Bool: Whether the buffer has marks and keeps no observation.

Related

• buffer_fit_count
• shared_series_buffer

shared_series_buffer(
    state::SampleBufferState
) -> SampleBufferState

Return the buffer that a fit reads, with the observations of a Shared Series gap dropped.

This is the buffer form of shared_series_view. An online step keeps an observation with a Shared Series gap in the buffer and marks it in G, because the buffer of a prior and the Fold Context of its optimiser hold one row each for every observation, see SampleBufferState. A fit drops the observation, as the batch fit over the same rows drops it at its entry. So every reader that fits on the rows of the buffer of a prior reads this buffer, and the call with no data gives the batch fit over the same observations. A reader that rebuilds the returns data, returns_result, reads the buffer itself, because the batch fit that it hands the data to drops the same observations again.

The rows are copied. The masks, the factor rows, the Exogenous Series, the implied volatilities and the Panel Fields keep the same rows. The result has no marks, so the function is idempotent.

Algorithm

1. Return state when it has no marks, which allocates nothing, or when no observation of its valid region is marked.
2. Find the observations that are not marked, and refuse a buffer that has none.
3. Copy the rows of the observations into a buffer of their own, with the same cap.

Arguments

• state: The buffer to read.

Validation

• At least one observation of the buffer has no Shared Series gap. An IsEmptyError is thrown otherwise.

Returns

• state::SampleBufferState: state itself when it has no gap, and otherwise a buffer of its observations with no gap.

Related

• SampleBufferState
• shared_series_view
• buffer_gap_marks

assert_sample_buffer(
    est::Union{AbstractEstimator, CovarianceEstimator}
)

Returns the sample buffer an estimator carries, and refuses an estimator that carries none.

The buffering arm of partial_fit! reads its state through this function. So an estimator that was never wrapped in Online gets a message that names the wrapper, not a MethodError. An estimator with no cache field gets the same message. An unresolved Online has a method of its own, because it does not lack a buffer. It lacks the warm-up that seeds one.

Arguments

• est: Estimator whose cache field carries the buffer.

Validation

• est has a cache field holding a SampleBufferState. An ArgumentError is thrown otherwise.

Returns

• state::SampleBufferState: The buffer the estimator carries.

Related

• SampleBufferState
• Online
• partial_fit_cache

sample_buffer_seed(
    cache::Union{Nothing, SampleBufferState}
) -> SampleBufferState

Returns the SampleBufferState a carry folds into, seeding an empty one when the estimator carries none.

The seed has its own function, so the fold is one line and the branch on the cache field has one method. A family that carries its observations calls fold_buffer, which is this seed followed by the fold. The seed has no cap, because a cap is configuration, and Online carries it.

Arguments

• cache: The buffer the estimator carries, or nothing.

Returns

• state::SampleBufferState: The buffer cache holds, or an empty uncapped buffer.

Related

• SampleBufferState
• fold_buffer
• Online

fold_buffer(cache::Option{<:SampleBufferState}, x::VecNum, f::Option{<:VecNum} = nothing; kwargs...)
fold_buffer(cache::Option{<:SampleBufferState}, X::MatNum, F::Option{<:MatNum} = nothing; dims::Int = 1, kwargs...)

Folds observations into a sample buffer, seeding an empty one when the estimator carries none.

A carry calls this function inside its own partial_fit! method. The carry folds its estimate exactly through its members, and it gives its observations to this function. The function is sample_buffer_seed followed by partial_fit!. The carry gives the masks and the factor rows of the fold with the observations, and the buffer records them beside their rows.

Arguments

• cache: The buffer the estimator carries, or nothing.
• x: One observation, whose entries are the assets.
• f: The contemporaneous factor observation, whose entries are the factors, or nothing.
• X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
• F: Factor observations to fold, oriented as X is, or nothing.
• dims: Dimension along which to perform the computation.
• kwargs...: The active_mask, the estimation_mask, the Exogenous Series ne and E, and the implied volatilities iv and ivpa of the fold, forwarded to partial_fit!.

Returns

• state::SampleBufferState: The buffer after the last observation.

Related

• SampleBufferState
• sample_buffer_seed
• partial_fit!

assert_buffer_factor_width(B::Nothing, Fo) -> Nothing
assert_buffer_factor_width(B::AbstractMatrix, Fo::AbstractMatrix) -> Nothing

Refuses a factor block whose width is not the width the first factor append fixed.

This is the width check of X, for the factor backing, which can be absent. A buffer that records no factor rows has no width to compare with. assert_buffer_presence_agreement runs first and refuses a block that gives factor rows to such a buffer, so the nothing method runs only when Fo is absent too.

Arguments

• B: The factor backing the buffer holds, or nothing.
• Fo: The oriented factor block, or nothing.

Validation

• size(B, 2) == size(Fo, 2) when the buffer records factor rows. A DimensionMismatch is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• partial_fit!
• assert_buffer_presence_agreement

assert_buffer_series_names(ne::Nothing, neo) -> Nothing
assert_buffer_series_names(ne::VecStr, neo::VecStr) -> Nothing

Refuses an Exogenous Series whose names are not the names the first append fixed.

A consumer reads a column of the series by its name, so a buffer whose columns change their names between two rows gives a column with two meanings. The first append that carries the series fixes the names, as ReturnsBufferState pins them. assert_buffer_presence_agreement runs first, so the nothing method runs only when the block carries no series either.

Arguments

• ne: The names the buffer records, or nothing.
• neo: The names of the incoming block, or nothing.

Validation

• neo == ne when the buffer records a series. An ArgumentError is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• partial_fit!
• merge_states

assert_buffer_presence_agreement(
    M::Union{Nothing, AbstractArray},
    Mo::Union{Nothing, AbstractArray},
    name::AbstractString
)

Refuses a fold whose masks, factor rows, Exogenous Series or Panel Fields disagree with what the buffer records.

This is the mixture rule of the buffer. It refuses the fold, because neither way to resolve the mixture gives a correct answer. A buffer that dropped an incoming mask would fit a held gap that the policy excludes, and it would read each delisting as a holiday. The mask exists to prevent that wrong answer. A buffer that made up a mask for its earlier rows would state an activity that it never saw. The same rule holds for the factor rows. A buffer with factor rows for some observations and not for others describes no regression, and the buffer cannot make up the missing rows. A prior that reads the Panel Fields computes its factors from them at every row, so a buffer that holds them for some rows and not for others gives it no complete panel. The same holds for the Exogenous Series that an observed factor reads at every fitted row. So the first append fixes what a buffer records, and a caller that wants a change starts a new buffer.

Arguments

• M: The backing the buffer records, or nothing.
• Mo: The same backing of the incoming block, or nothing.
• name: Name of the backing, for the message.

Validation

• M and Mo are both nothing, or neither is. An ArgumentError is thrown otherwise.

Returns

• nothing.

Related

• SampleBufferState
• partial_fit!
• seed_sample_buffer
• CoveragePolicy
• needs_factor_returns

seed_sample_buffer(
    state::SampleBufferState,
    Xo::AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}},
    Ao::Union{Nothing, AbstractMatrix{<:Bool}},
    Mo::Union{Nothing, AbstractMatrix{<:Bool}},
    Fo::Union{Nothing, AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}}},
    ne::Union{Nothing, AbstractVector{<:AbstractString}},
    Eo::Union{Nothing, AbstractMatrix{<:Union{var"#s152", var"#s69"} where {var"#s152"<:Number, var"#s69"<:AbstractJuMPScalar}}},
    Po::Union{Nothing, AbstractVector}
) -> Any

Fills a buffer that holds no observations from a block, which fixes the width, the element type, the masks, the factor rows, the Exogenous Series and the Panel Fields.

partial_fit! calls it for the first append, and for each append to a buffer that holds no observations. Each backing matrix is an exact fit for the block, so a buffer that is folded once and read once allocates no spare row. Each array comes from similar on the argument that it holds, so the function chooses no element type. The Panel Fields are not copied, because nothing writes into the arrays of a panel.

Algorithm

1. Replace X, A, M, F and E with copies of the block, its masks, its factor rows and its series, made by seed_buffer_array.
2. Replace ne with the names of the series, and P with the Panel Fields of the block.
3. Set off to zero and n to the row count of the block.

Arguments

• state: The buffer to fill. It holds no observations.
• Xo: The oriented block.
• Ao: The oriented active mask of the block, or nothing.
• Mo: The oriented estimation mask of the block, or nothing.
• Fo: The oriented factor rows of the block, or nothing.
• ne: The names of the series of the block, or nothing.
• Eo: The oriented Exogenous Series of the block, or nothing.
• Po: The Panel Fields of the block, or nothing.

Returns

• state::SampleBufferState: The buffer holding the block, its masks, its factor rows, its series and its Panel Fields.

Related

• SampleBufferState
• partial_fit!
• seed_buffer_array

seed_buffer_array(Mo::Nothing) -> Nothing
seed_buffer_array(Mo::AbstractMatrix) -> AbstractMatrix

Copies an incoming block into a backing matrix of its own, and passes a backing that is not there through.

The block of a fold is an array of the caller, and the buffer lives after the call. A later fold writes into the backing matrix, so the buffer takes a copy, as Base.copy and port_opt_view do. Without the copy, the fold would change the array of the caller. similar reads the element type from the block, so an observation, a mask and a factor row each keep their own element type.

Arguments

• Mo: The oriented block, mask or factor rows, or nothing.

Returns

• M: A backing matrix holding the block, or nothing.

Related

• SampleBufferState
• seed_sample_buffer

copy_buffer_rows!(M::Nothing, rows, Mo) -> Nothing
copy_buffer_rows!(M::AbstractMatrix, rows, Mo::AbstractMatrix) -> AbstractMatrix

Copies an incoming backing into the rows of the buffer's backing matrix that were just made room for.

This is the optional part of the append. The two masks and the factor rows each take one line of the fold, and the nothing case has one method. assert_buffer_presence_agreement runs first, so the two arguments are both nothing or both matrices.

Arguments

• M: The buffer's backing matrix for this mask or the factor rows, or nothing.
• rows: The rows of M to write.
• Mo: The oriented mask or factor rows of the block, or nothing.

Returns

• M: The rows written, or nothing.

Related

• SampleBufferState
• partial_fit!
• assert_buffer_presence_agreement

seed_buffer_gaps(Go::Nothing) -> Nothing
seed_buffer_gaps(Go::AbstractVector{Bool}) -> Option{Vector{Bool}}

Seeds the marks of the Shared Series gaps of a SampleBufferState from the first block that it holds.

A block with no gap seeds no marks, so a buffer that never meets a gap keeps G at nothing. A block with a gap seeds a copy of its marks, because a later fold writes into the backing vector.

Arguments

• Go: The marks of the block, true at an observation with a Shared Series gap, or nothing.

Returns

• G::Option{Vector{Bool}}: The backing vector of the marks, or nothing when the block has no gap.

Related

• SampleBufferState
• append_buffer_gaps

append_buffer_gaps(state::SampleBufferState, rows, Go::Nothing) -> SampleBufferState
append_buffer_gaps(state::SampleBufferState, rows, Go::AbstractVector{Bool}) -> SampleBufferState

Writes the marks of the Shared Series gaps of a block into the rows of a SampleBufferState that were just made room for.

The marks have no presence rule, because a buffer with no marks has no gap. A buffer with no marks and a block with no gap keep no marks. The first block with a gap gives the buffer a backing vector of false marks for the rows before it, and a block with no marks writes false into a buffer that has them.

Arguments

• state: The buffer, after reserve_sample_buffer made room for the block.
• rows: The rows of the backing to write.
• Go: The marks of the block, or nothing.

Returns

• state::SampleBufferState: The buffer with the marks of the block written.

Related

• SampleBufferState
• seed_buffer_gaps
• partial_fit!

reserve_sample_buffer(
    state::SampleBufferState,
    t::Integer
) -> Any

Makes room for t further observations in a SampleBufferState, and returns the buffer that has it.

This is the growth part of the fold of the buffer. It writes no observation. It drops the rows that the cap pushes out, moves the valid region to the front of the backing matrices, and allocates larger matrices when the front does not have enough room.

These steps make an append cost amortised O(1). A drop moves off and copies no row, so a capped buffer pays nothing for the observation that it drops. The masks, the factor rows and the series of that observation drop with it, because the same off and n index them. A compaction in place happens only when the valid region and the block fill at most half of the backing matrix. So it copies at most half of the matrix, and at least half of the matrix is free after the append. A growth at least doubles the capacity until the clamp at twice the cap stops it, and it copies fewer rows than the old capacity. So all the growths of a buffer copy fewer rows than its last capacity. A block longer than twice the capacity grows the matrix to fit the valid region and the block, and the next growth doubles it. A capped buffer never holds more than twice its cap, so a cap of w costs at most 2w rows.

Algorithm

1. Drop the oldest observations the cap pushes out, by moving off forward and n back.
2. Return when the valid region and the block already fit inside the backing matrix.
3. Take a capacity of twice the current one, or of the room needed if that is larger, when the valid region and the block would fill more than half of the current one. Clamp it to twice the cap when a cap is set.
4. Compact the observations, each mask, the factor rows, the series and the marks of the Shared Series gaps that the buffer records to that capacity with compact_buffer_array, which allocates only when the capacity changes, and set off to zero.

Arguments

• state: The buffer to make room in.
• t: Number of observations about to be appended.

Returns

• state::SampleBufferState: The buffer whose backing matrices have room for t rows after their valid region.

Related

• SampleBufferState
• compact_buffer_array
• partial_fit!

compact_buffer_array(B::Nothing, off, n, newcap) -> Nothing
compact_buffer_array(B::AbstractMatrix, off::Integer, n::Integer, newcap::Integer) -> AbstractMatrix
compact_buffer_array(G::AbstractVector, off::Integer, n::Integer, newcap::Integer) -> AbstractVector

Moves the valid region of one backing matrix to the front, allocating a matrix of newcap rows when the capacity changes. The backing vector of the marks of the Shared Series gaps has one entry per row, and compacts in the same way.

This is the array part of reserve_sample_buffer. The buffer carries five backing matrices with one index, and this function compacts each of them. A copy in place with an increasing row index is safe, because the destination row i is never after the source row off + i. So the loop reads each source row before a write can reach it.

Algorithm

1. Take D as B when B has newcap rows, and as a new matrix of newcap rows otherwise.
2. Copy row off + i of B to row i of D, for i from 1 to n, one column at a time.

Arguments

• B: The backing matrix to compact, or nothing.
• off: Number of rows before the valid region.
• n: Length of the valid region.
• newcap: Number of rows the compacted matrix has.

Returns

• D: The compacted matrix, or nothing.

Related

• SampleBufferState
• reserve_sample_buffer

observation_row(m::Nothing) -> Nothing
observation_row(m::AbstractVector) -> AbstractMatrix

Reshapes one observation, its mask or its factor observation into a one-row block, and passes one that is not there through.

This is the vector part of the single-observation arm of partial_fit!. reshape makes no copy, so one observation with a policy or with factors allocates no more than one observation without them.

Arguments

• m: One observation, one mask or one factor observation, or nothing.

Returns

• M: The vector as a one-row block, or nothing.

Related

• SampleBufferState
• partial_fit!

merge_buffer_array(Ma::Nothing, Mb::Nothing) -> Nothing
merge_buffer_array(Ma::AbstractMatrix, Mb::AbstractMatrix) -> AbstractMatrix

Concatenates the same mask, or the factor rows, of two buffers, and passes a backing neither records through.

This is the optional part of merge_states. assert_buffer_presence_agreement runs first, so the two arguments are both nothing or both matrices.

Arguments

• Ma: The backing of the first buffer, or nothing.
• Mb: The backing of the second buffer, or nothing.

Returns

• M: The two backings stacked, or nothing.

Related

• SampleBufferState
• merge_states

merge_buffer_gaps(
    a::SampleBufferState,
    b::SampleBufferState
) -> Any

Concatenates the marks of the Shared Series gaps of two buffers over their valid regions.

The marks have no presence rule, because a buffer with no marks has no gap. So two buffers with no marks give no marks, and a buffer with no marks gives false for each of its observations beside a buffer that has them.

Arguments

• a: The buffer of the first block of observations.
• b: The buffer of the second block of observations.

Returns

• G::Option{Vector{Bool}}: The marks of the rows of a followed by the marks of the rows of b, or nothing when neither buffer has marks.

Related

• merge_states
• SampleBufferState

trim_merged_array(M::Nothing, rows) -> Nothing
trim_merged_array(M::AbstractMatrix, rows) -> AbstractMatrix
trim_merged_array(G::AbstractVector, rows) -> AbstractVector

Keeps the rows a cap admits of a merged mask or of the merged factor rows, by index copy, and passes a backing that is not there through.

This is the cap part of merge_states. It copies the rows and does not take a view. A view would keep the whole concatenation in memory, with the rows that the cap drops, and the merged buffer would carry a backing of a different type from the backing that a fold makes.

Arguments

• M: The merged backing, or nothing. A vector is the backing of the marks of the Shared Series gaps.
• rows: The rows the cap admits.

Returns

• M: The admitted rows, or nothing.

Related

• SampleBufferState
• merge_states

copy(x::SampleBufferState) -> SampleBufferState

Copies a SampleBufferState, so the copy shares no array with the original.

This is the copy method of the AbstractPartialFitState interface, which partial_fit calls before it folds. It copies each backing matrix whole, spare capacity included. So the copy appends on the same terms as the original, and the same off and n index the masks, the factor rows, the series and the observations. The copy shares the Panel Fields and the names of the series with the original. Nothing writes into the arrays of a panel or into the names, and an append makes new arrays.

Arguments

• x: The buffer to copy.

Returns

• state::SampleBufferState: A fresh buffer, equal to x, whose backing matrices are fresh matrices and whose Panel Fields are the Panel Fields of x.

Related

• SampleBufferState
• partial_fit
• AbstractPartialFitState

copy_buffer_array(M::Nothing) -> Nothing
copy_buffer_array(M::AbstractArray) -> AbstractArray

Copies one backing matrix of a buffer, and passes a matrix that is not there through.

This is the nothing case of copy for the two masks, the factor rows, the series and the backing vector of the marks of the Shared Series gaps. Base.copy and port_opt_view use it to copy each optional field in one line.

Arguments

• M: The backing matrix to copy, or nothing.

Returns

• M: A fresh matrix equal to it, or nothing.

Related

• SampleBufferState
• Base.copy

slice_buffer_mask(M::Nothing, i) -> Nothing
slice_buffer_mask(M::AbstractMatrix, i) -> AbstractMatrix

Slices one backing matrix of a buffer that has an asset axis to the selected assets, and passes a matrix that is not there through.

This is the per-asset part of port_opt_view for this state, so the slice is one line for each field. It slices the two masks and the implied volatilities. It copies by index, for the reason that the method states.

Arguments

• M: The backing mask or implied volatilities to slice, or nothing.
• i: Index or indices of the assets to keep.

Returns

• M: The matrix over the selected assets, or nothing.

Related

• SampleBufferState
• port_opt_view