Returns buffer state: private API
Types
PortfolioOptimisers.ReturnsBufferState — Type
struct ReturnsBufferState{__T_nx, __T_X, __T_nf, __T_F, __T_nb, __T_B, __T_ne, __T_E, __T_ts, __T_pnl, __T_max_history} <: AbstractPartialFitStateThe Fold Context of an optimiser: the columns of a ReturnsResult that the prior beneath the optimiser does not keep, and the context that the first step pins.
An optimiser's online step sends each observation to its prior and to nothing else. The call optimise(opt) with no data rebuilds a ReturnsResult from the prior's buffer and from this state, and runs the batch path over it. The batch path needs the whole ReturnsResult and not a bare matrix, because a meta-optimiser hands its inner optimisers a view of the ReturnsResult, and a UniverseSets constraint finds its assets by name.
Each column has one owner. The prior at the bottom of the chain keeps the returns in its own state. This state keeps them only for a head that holds no prior, which is EqualWeighted, RandomWeighted or BestConstantRebalancedPortfolio. The prior's buffer keeps the factor column when the prior's estimator tree reads it, and this state keeps it when needs_factor_returns answers false or when there is no prior. This state always keeps the benchmark column, the Exogenous Series and the timestamps.
The first step pins the asset, factor, benchmark and Exogenous Series names and a static AssetPanel, and every later step must carry the same values. A time-varying panel is not pinned. Its active mask goes into the returns buffer with the rows it describes, and optimise(opt) with no data rebuilds the panel from it.
Each matrix column is a SampleBufferState, so each column takes its orientation, cap, merge, copy and asset slice from that type, and max_history caps every column at once. The timestamps and a single-column benchmark are plain vectors, because no asset indexes them.
Fields
nx: Names of the asset columns, which the first step pins.
X: Buffer of the asset returns,observations × assets. It isnothingwhen a prior beneath the optimiser keeps the rows, which is every optimiser except a head that holds no prior.
nf: Names of the factor columns, which the first step pins.
F: Buffer of the factor returns,observations × factors. It isnothingwhen the returns data holds none, and when the prior beneath the optimiser keeps them in its own buffer, which it does whenever its tree reads them.
nb: Names of the benchmark columns, which the first step pins.
B: Buffer of the benchmark. It is aSampleBufferStateofobservations × assetsfor a matrix benchmark, a plain vector for a single column, andnothingwhen the returns data holds none.
ne: Names of the Exogenous Series, which the first step pins.
E: Buffer of the Exogenous Series,observations × series, ornothingwhen the returns data holds none. No prior keeps it, so this state always does.
ts: Timestamps of the folded observations, in order, ornothingwhen the returns data holds none.
pnl: The staticAssetPanelof the universe, which the first step pins, ornothing. This field never holds a time-varying panel.
max_history: Optional cap on the number of observations the buffer keeps.nothingkeeps every observation folded so far. A capped buffer drops its oldest observations as new ones arrive and holds the lastmax_historyof them. The call with no data reads those rows only. The cap is the window of the fit. An estimator wrapped inOnlinereturns the batch fit over the lastmax_historyobservations, also when its statistic has an exact update.
Constructors
ReturnsBufferState(; nx::Option{<:VecStr} = nothing, X::Option{<:SampleBufferState} = nothing, nf::Option{<:VecStr} = nothing, F::Option{<:SampleBufferState} = nothing, nb::Option{<:VecStr} = nothing, B::Option{<:Union{SampleBufferState, AbstractVector}} = nothing, ne::Option{<:VecStr} = nothing, E::Option{<:SampleBufferState} = nothing, ts::Option{<:AbstractVector} = nothing, pnl::Option{<:AssetPanel} = nothing, max_history::Option{<:Integer} = nothing) -> ReturnsBufferStateKeywords correspond to the struct's fields. The defaults give the empty seed that a first step fills, and a caller sets only the cap.
Validation
max_history > 0when it is notnothing. ADomainErroris thrown otherwise.pnlis static when it is notnothing. AnArgumentErroris thrown otherwise, because the masks of a time-varying panel belong in the returns buffer.
View parameters
port_opt_view takes the view of its fields at the selected assets:
nx,X,pnl: Sliced to the selected assets.nb,B: Sliced when the benchmark is a matrix over the assets, and copied unchanged when it is a single column.nf,F,ne,E,ts: Copied unchanged, because no asset indexes them.
Related
Functions
PortfolioOptimisers.assert_pinned_context — Function
assert_pinned_context(pinned, given, name::Symbol)
Refuses a step whose returns data disagrees with the context that the first step pinned.
The names and the static panel are context and not sample. The first step fixes them, and every later step must carry the same values, because a state that took a new asset axis would fold the next observation onto the wrong column. merge_states applies the same check to two states. Two values agree when pinned_agree answers true, and two nothing values agree.
Arguments
pinned: The value that the first step pinned.given: The value that this step carries.name: The field, for the message.
Validation
pinned_agree(pinned, given). AnArgumentErroris thrown otherwise.
Returns
nothing.
Related
PortfolioOptimisers.pinned_agree — Function
pinned_agree(a, b) -> Bool
Answers whether two pinned values are the same value, by content.
isequal is not enough for pinned context. An AssetPanel is an immutable struct that holds arrays, and a struct with no equality method of its own compares its array fields by identity. Two panels of one universe that come from two views of one ReturnsResult then compare unequal, and every step of a walk-forward hands over such a view.
Algorithm
- When
aandbare both arrays, answertruewhen their sizes are equal and each pair of elements agrees by this function. - When
aorbis a number, a string, a symbol ornothing, answerisequal(a, b). - When
aandbare structs with the same field names, answertruewhen each pair of fields agrees by this function. The two types can have different parameters, because a view of a panel holds views where the panel holds arrays. - Otherwise, answer
isequal(a, b).
Arguments
a: The value that the first step pinned.b: The value that this step carries.
Returns
agree::Bool:truewhen the two are the same value.
Related
PortfolioOptimisers.pinned_repr — Function
pinned_repr(x) -> Any
Renders a pinned value for a refusal message. A vector of names renders in full, nothing renders as nothing, and any other value renders as its summary.
Arguments
x: The pinned value.
Returns
s::String: The text for the message.
Related
PortfolioOptimisers.assert_column_presence — Function
assert_column_presence(buffer, column, name::Symbol)
Refuses a step or a merge that adds a column the first step did not carry, or drops a column it did carry.
The first step fixes whether the returns data holds a factor matrix, a benchmark, an Exogenous Series or timestamps, as a SampleBufferState fixes whether it records a mask. A column that is present for some observations and absent for others cannot be rebuilt into a matrix. The caller runs this check on every step after the first, also when the state keeps no column at all, and merge_states runs it on the columns of two states.
Arguments
buffer: The column's buffer in the state, ornothing.column: The column that the step carries, the buffer of the other state, ornothing.name: The field, for the message.
Validation
isnothing(buffer) == isnothing(column). AnArgumentErroris thrown otherwise.
Returns
nothing.
Related
PortfolioOptimisers.context_count — Function
context_count(state::ReturnsBufferState) -> Any
Number of observations a ReturnsBufferState holds.
Every column takes the same appends and the same cap, so every column holds the same number of rows, and the count comes from the first column the state keeps. A state that keeps no column answers zero. This is the state for a ReturnsResult that holds returns and names alone, beside a prior that keeps the rows. returns_result checks the count against the buffer of the rows.
Arguments
state: The state to count.
Returns
n::Int: The number of observations folded.
Related
PortfolioOptimisers.column_count — Function
column_count(buffer::Nothing)
column_count(buffer::SampleBufferState)
column_count(buffer::AbstractVector)Number of observations that one column of a ReturnsBufferState holds, or nothing for a column the state does not keep. A buffer answers its n, and a vector answers its length.
Arguments
buffer: The column, ornothing.
Returns
n::Option{<:Integer}: The number of observations, ornothing.
Related
PortfolioOptimisers.fold_column — Function
fold_column(buffer::Option{<:SampleBufferState}, column::Option{<:MatNum}, max_history)
fold_column(buffer::Option{<:AbstractVector}, column::Option{<:AbstractVector}, max_history)Appends one block of a column to its buffer, and seeds the buffer on the first step.
The vector arm writes into buffer and returns it.
Algorithm
- When the returns data does not hold the column, return
nothing. - For a matrix column, seed a
SampleBufferStatewith the cap whenbufferisnothing, and appendcolumnto it withpartial_fit!. - For a vector column, seed an empty vector with the element type of
columnwhenbufferisnothing, and appendcolumnto it. - When the vector is longer than
max_history, delete its first entries, so it keeps the lastmax_history, as a capped buffer drops its oldest rows.
Arguments
buffer: The column's buffer, ornothingbefore the first step.column: The block to append, ornothing.max_history: The cap that every column of the state shares.
Returns
- The buffer after the block, or
nothing.
Related
PortfolioOptimisers.fold_column_masked — Function
fold_column_masked(
buffer::Union{Nothing, SampleBufferState},
rd::ReturnsResult,
amsk::Union{Nothing, AbstractMatrix{<:Bool}},
max_history::Union{Nothing, Integer}
) -> SampleBufferState
Appends a block of returns to the buffer that a head with no prior owns, with the active mask of its panel and the implied volatilities of the block.
The form of fold_column for the one column that carries a mask. The buffer records the active mask of a time-varying panel with the rows, so optimise(opt) with no data can rebuild the panel from it. A static panel gives no mask. The buffer records the implied volatilities with the rows, and pins ivpa, so optimise(opt) with no data gives them back.
Algorithm
- Seed a
SampleBufferStatewith the cap whenbufferisnothing. - Append
rd.Xto the buffer withpartial_fit!, withamskas its active mask, and withrd.ivandrd.ivpa.
Arguments
buffer: The returns buffer, ornothingbefore the first step.rd: The returns data of the block. The function readsX,ivandivpa.amsk: The active mask of the block, ornothing.max_history: The cap of the state.
Returns
buffer::SampleBufferState: The buffer after the block.
Related
PortfolioOptimisers.returns_result — Method
returns_result(state::ReturnsBufferState, rows::SampleBufferState)Rebuilds the ReturnsResult that the observations folded so far describe.
The reconstitution verb of optimise(opt) with no data. rows is the buffer that holds the returns, which is the prior's buffer, or the state's own X when the head holds no prior. The state holds every other column and the pinned context. The two exceptions are the factor column of a prior whose tree reads it, and the Exogenous Series with its names of a prior whose tree reads it, which rows holds. Every array of the result is a new copy and not a view, because the next fold writes into the buffers and can reallocate them, and a ReturnsResult that held a view would then change in the hands of its caller. The Panel Fields are the one exception. Nothing writes into the arrays of a panel, and a fold makes new arrays when it appends, so the result shares them with rows.
Algorithm
- Read
nwithcontext_count, and refuse a state whose count differs from the count ofrows. - Copy the valid rows of the state's own returns into
Xwhen the state keeps them, and the valid rows ofrowsotherwise. The state keeps them for a head that holds no prior, whererowsis the same buffer, and for an optimiser whose prior folds the returns in excess of the benchmark. - When
rowsrecords an active mask, rebuild a time-varyingAssetPanelfrom it withbuffer_asset_panel. Its estimation mask is the one thatrowsrecords or, whenrowsrecords none, a copy of the active mask. The step of a refit prior records both masks, and the carry ofEmpiricalPriorand a head with no prior record the active mask alone. Its Panel Fields are the ones thatrowsrecords, which it records for a prior that reads them, and none otherwise. Whenrowsrecords no active mask, take the static panel that the state pinned, ornothing. - Take
Ffrom the state when the state keeps the factor column, and fromfactor_bufferofrowsotherwise. TakeneandEfrom the state when the state keeps the series, and fromexogenous_buffer_kwargsofrowsotherwise. - Take
ivandivpafromimplied_volatility_buffer_kwargsofrows, which records them beside the returns. - Copy
Bandtsfrom the state, and build theReturnsResult.
Arguments
state: The Fold Context.rows: The buffer that holds the returns.
Validation
rowsand the state hold the same number of observations, when the state keeps a column. ADimensionMismatchis thrown otherwise.
Returns
rd::ReturnsResult: The rebuilt returns data. Field by field, it is equal to theReturnsResultthat a batch fit over the same observations reads, with one exception. A time-varying panel comes back with the parts that the route of the prior recorded: both masks and the Panel Fields for a refit prior that reads them, both masks and no Panel Field on the carry fold of aCrossSectionalFactorPrior, and the active mask alone on the carry of another prior or with no prior. An optimiser with a prior refuses a step that carries a part its route cannot record, and a head with no prior reads none of them. A capped buffer gives back theivof the rows that it keeps, as it gives backX.
Related
PortfolioOptimisers.column_matrix — Function
column_matrix(buffer::Nothing)
column_matrix(buffer::SampleBufferState)
column_matrix(buffer::AbstractMatrix)
column_matrix(buffer::AbstractVector)Copies one column of a ReturnsBufferState, or the factor rows of a prior's buffer, into a new array for the ReturnsResult that optimise(opt) with no data rebuilds. A buffer gives a matrix of its valid rows, a matrix or a vector gives a copy of itself, and nothing gives nothing.
Arguments
buffer: The column's buffer, the valid region of the prior's factor rows, ornothing.
Returns
- The column as a new array, or
nothing.
Related
PortfolioOptimisers.merge_column — Function
merge_column(a::Nothing, b::Nothing, max_history)
merge_column(a::SampleBufferState, b::SampleBufferState, max_history)
merge_column(a::AbstractVector, b::AbstractVector, max_history)Concatenates one column of two ReturnsBufferState, and keeps the last max_history rows. A buffer merges with merge_states, which applies its own cap. A vector appends b to a copy of a with fold_column, which applies the cap, so neither input changes.
Arguments
a: The column of the first state, ornothing.b: The column of the second state, ornothing.max_history: The cap that both states share.
Returns
- The merged column, or
nothing.
Related
Base.copy — Method
copy(x::ReturnsBufferState) -> ReturnsBufferState
Copies a ReturnsBufferState, so that a fold on the copy leaves the original unchanged.
Every column and every name vector of the copy is a new array. The pinned panel is shared and not copied, because no step and no merge writes into it.
Arguments
x: The state to copy.
Returns
state::ReturnsBufferState: A new state, equal tox.
Related
PortfolioOptimisers.copy_column — Function
copy_column(x::Nothing)
copy_column(x)Copies one field of a partial-fit state with copy, and passes a field that is nothing through.
Arguments
x: The field, ornothing.
Returns
- A copy of
x, ornothing.
Related