GitHub

Preprocessing partial fit: private API

Types

PortfolioOptimisers.PricesToReturnsState — Type
struct PricesToReturnsState{__T_tail, __T_anchor} <: AbstractPartialFitState

The state a PricesToReturns keeps between two blocks of prices, so that the next block converts exactly as the whole history would.

A return reads two consecutive prices, so the conversion of a new block needs the last price row of the block before it, and nothing else of that block: tail is that row, kept as a one-row PricesResult so the span, the panel and the implied volatilities of the row ride with the prices. A CatchUpGapReturn reads further back — the observation that ends a gap books the move against the last observed price, which may lie any number of blocks behind — so anchor keeps that price per series column, NaN where none has been observed. It is nothing under the default rule, which reads no earlier price.

Fields

  • anchor: The last observed price of every series column — the assets, then the factors, then the benchmark — or NaN where none has been observed. nothing under the default gap rule.

Constructors

PricesToReturnsState(; tail::PricesResult, anchor::Option{<:AbstractVector} = nothing)

Related

source
PortfolioOptimisers.PriceGapFillState — Type
struct PriceGapFillState{__T_nx, __T_v, __T_te} <: AbstractPartialFitState

The state a PriceGapFill with a CarriedPrice keeps between two blocks: the last observed price of every asset, which is the carry the next block's gaps take and the seed the fitted result replays, and the timestamp the last block ended on, which the fitted result records as the end of its training window.

Fields

  • nx: Names of the asset columns, pinned by the first block.
  • v: The last observed price per asset, missing where none has been observed.
  • te: The last timestamp of the last block folded. A carried price precedes every observation of the next block, so the block's gaps are filled from it; the fitted result records it as the end of its training window.

Constructors

PriceGapFillState(; nx::AbstractVector{Symbol}, v::AbstractVector, te)

Related

source
PortfolioOptimisers.MissingDataFilterState — Type
struct MissingDataFilterState{__T_nx, __T_miss, __T_n} <: AbstractPartialFitState

The state a MissingDataFilter keeps between two blocks: the observations counted and the missing count per asset, from which the column filter is read out.

Fields

  • nx: Names of the asset columns, pinned by the first block.
  • miss: Missing observations per asset, over every block folded.
  • n: Observations folded.

Constructors

MissingDataFilterState(; nx::AbstractVector{Symbol}, miss::AbstractVector{<:Integer}, n::Integer)

Related

source

Functions

PortfolioOptimisers.vcat_carrier_rows — Function
vcat_carrier_rows(a::PricesResult, b::PricesResult)
vcat_carrier_rows(a::ReturnsResult, b::ReturnsResult)

Concatenates the observations of two carriers of the same universe, a first.

The row-concatenation every online form of a data step reads: PricesToReturns puts the last price row it kept in front of a new block so the conversion reads consecutive prices, and the input-carrier buffer Online(pipe) seeds appends every block it is handed. The universe is pinned by a — the asset, factor and benchmark names, the implied-volatility adjustment, a static AssetPanel — and a block carrying a different one is refused by name, as ReturnsBufferState's step refuses one. A column one carrier holds and the other does not is refused too: a factor series that comes and goes is not one series.

A time-varying panel is concatenated mask by mask. Its Panel Fields are refused, as step_active_mask refuses them: a time-varying field's rows are sample, and no concatenation of a categorical or a tensor field exists here.

Arguments

  • a: The earlier observations.
  • b: The later observations.

Validation

  • The names, the implied-volatility adjustment and a static panel agree. An ArgumentError is thrown otherwise.
  • Every optional column is held by both carriers or by neither. An ArgumentError is thrown otherwise.
  • A time-varying panel holds no Panel Field. An ArgumentError is thrown otherwise.

Returns

  • c: A carrier of the same type as a, holding the observations of a followed by those of b.

Related

source
PortfolioOptimisers.vcat_optional — Function
vcat_optional(a::Nothing, b::Nothing, name::Symbol)
vcat_optional(a, b, name::Symbol)

Concatenates one optional column of two carriers along the observation axis, or refuses a column one holds and the other does not.

Related

source
PortfolioOptimisers.vcat_panel_rows — Function
vcat_panel_rows(a::Nothing, b::Nothing)
vcat_panel_rows(a::AssetPanel, b::AssetPanel)

Concatenates the AssetPanels of two blocks along the observation axis.

A static panel carries no observation axis, so the two must agree and the first is kept. A time-varying panel is concatenated mask by mask, and one carrying a Panel Field is refused: a time-varying field's rows are sample, and this seam holds masks alone.

Related

source
PortfolioOptimisers.partial_fit_transform — Function
partial_fit_transform(est, data) -> (est′, data′)

Folds a block of observations into a data step and emits the block the step's transform gives it.

The online form of a preprocessing step that changes the rows it is handed. A Pipeline walking its steps hands each one the block it received from the step before and takes back the block the step emits, so a step's fold and its transform are one call: the transform of the new rows reads the state the earlier rows left, and the state is advanced past them. The two halves are inseparable — the first return of a new block is computed from the last price row the step kept — so the verb returns both.

partial_fit! on a data step is this verb's first element, and a step's read-out is fit_preprocessing with no data. A caller's own preprocessing estimator joins the host route by writing those three: this verb, the read-out, and supports_partial_fit answering true.

Arguments

  • est: The data step, carrying its state in cache or nothing before the first block.
  • data: The block of observations, as the step's batch verb takes it.

Returns

  • (est′, data′): The step with the block folded, and the block as the step emits it.

Related

source
partial_fit_transform(ptr::PricesToReturns, pr::PricesResult) -> (ptr′, rd)

Converts a block of prices to returns as the whole history would, and keeps what the next block needs.

Algorithm

  1. On the first block, convert it alone with prices_to_returns; the block is the history.
  2. On every later block, put the kept last price row in front of it with vcat_carrier_rows, convert the window under the step's own ret_method and padding and the default gap rule, and keep the last n return rows, n the block's observations: the extra row is the seam, and padding adds one row the same way in the window and in the history.
  3. Under a CatchUpGapReturn, resolve the writable cells of the new rows with block_gap_return!, which extends every column by the anchor and the kept row so the algorithm meets the same neighbours it meets in the history.
  4. Keep the block's last price row as the new tail, and advance the anchor past the block.

The result is exact: every return of the emitted rows is the one the batch conversion of the whole history writes at that row, because a return reads two consecutive prices and a Gap Return reads the last observed one, and the state carries both.

Related

source
partial_fit_transform(est::PriceGapFill, pr::PricesResult) -> (est′, pr′)

Fills the gaps of a block of prices as the fit over the whole history would, and carries the last observed prices forward.

Algorithm

  1. Resolve the Listing Span bounding the fill with gap_fill_span, as the batch replay does.
  2. Seed each column's walk. An asset observed in an earlier block is seeded with the carried price, which precedes every observation of this block. One not observed in an earlier block has no seed, so a gap that opens its column stays a gap until the block's own first price, as the batch replay of PriceGapFill leaves it on the training window: a price later in the block is not a seed for a gap before it. One observed nowhere yet is left alone.
  3. Walk each column with gap_fill_column! under the step's convention, the seed applying from the first observation after the state's end.
  4. Advance the carry to each column's last observed price of the block, and the end to the block's last timestamp.

The convention must be CarriedPrice; a statistic fill re-prices every earlier gap when the window grows and has no online form, which supports_partial_fit states and the Pipeline refuses at warm-up.

Related

source
partial_fit_transform(mdf::MissingDataFilter, pr::PricesResult) -> (mdf′, pr)

Counts the gaps of a block of prices and passes the block through unchanged.

The column filter is universe-only — a longer window changes which assets survive and nothing else — so it is deferred to the read-out, where fit_preprocessing with no data answers the MissingDataFilterResult of the whole history and the Pipeline applies it as a view. The row filter at row_thr = 1 drops nothing, so every row passes; a row_thr < 1 has no online form, which supports_partial_fit states and the Pipeline refuses at warm-up.

Related

source
PortfolioOptimisers.block_gap_return! — Function
block_gap_return!(
    alg::AbstractGapReturnAlgorithm,
    R::AbstractMatrix,
    P::AbstractMatrix,
    anchor::AbstractVector,
    last::AbstractMatrix,
    ret_method::Symbol
) -> AbstractMatrix

Resolves the cells a Gap Return algorithm writes in the returns of a new block, reading the anchor of the state for the price before the block.

The batch conversion hands gap_return a whole column; a block hands it the column extended by two rows in front, the anchor and the last price row kept, so the algorithm's walk meets the observed price a gap's move is booked against and the immediate predecessor of the block's first row exactly as it would in the whole history. Only the cells of the new rows are copied back, and only those gap_return_writable admits over the extended column, which is the batch invariant restated: an anchor is an earlier observed price, and a column with none admits nothing before its first observation.

Arguments

  • alg: The Gap Return algorithm.
  • R: The returns of the new block, observations × series, mutated in place.
  • P: The prices of the new block, observations × series.
  • anchor: The last observed price per series column before the block.
  • last: The last price row before the block, 1 × series.
  • ret_method: :simple or :log.

Returns

  • R: The same matrix, with the writable cells of the new rows resolved.

Related

source
PortfolioOptimisers.returns_with_series — Function
returns_with_series(
    rd::ReturnsResult,
    R::AbstractMatrix,
    _::AbstractGapReturnAlgorithm
) -> Union{ReturnsResult{_A, _B, _C, _D, _E, _F, _G, _H, _I, Nothing} where {_A, _B, _C, _D, _E, _F, _G, _H, _I}, ReturnsResult{_A, _B, _C, _D, _E, _F, _G, _H, _I, AssetPanel{__T_pf, __T_amsk, __T_emsk}} where {_A, _B, _C, _D, _E, _F, _G, _H, _I, __T_pf, __T_amsk, __T_emsk}}

Rebuilds a returns carrier from the series matrix series_values_returns laid out, re-deriving the estimation mask the way the conversion does, because a Gap Return writes a finite return where the conversion left none.

Related

source