Price gap fill: private API

PortfolioOptimisers.prices_listing_span — Function
prices_listing_span(pr::PricesResult) -> Option{<:AbstractMatrix{Bool}}

Read the Listing Span that the price data pr states, or nothing when it states none.

The price data holds the Listing Span, so a step that needs a span reads it from pr and does not derive one. A PricesResult returns its span field. price_ingestion fills that field, and a PricesResult that a caller builds by hand leaves it nothing. The step that asked then uses its fallback and reports it.

Algorithm

  1. Return the span field of pr.

Arguments

  • pr: The price data that the fill reads.

Returns

  • span::Option{<:AbstractMatrix{Bool}}: The Listing Span of pr, or nothing.

Related

source
PortfolioOptimisers.gap_fill_span — Function
gap_fill_span(span::AbstractMatrix{Bool}, X::AbstractMatrix, strict::Bool) -> AbstractMatrix{Bool}
gap_fill_span(span::Nothing, X::AbstractMatrix, strict::Bool) -> BitMatrix

Resolve the Listing Span that bounds a PriceGapFill over one window.

The span comes from the price data, because a listing calendar is a fact about the instruments and a window cannot see all of it. Price data that states no span leaves only the window, and the window cannot give the span. A suspension across the edge of a window reads there as an inception or a delisting, so a span derived from the window fills the wrong cells, not fewer cells. The fill therefore takes an all-false span, under which every cell lies outside a listing and the fill writes no price. The function reports this with a warning, or with an error under strict. It reports only when the window holds a gap, because a window with no gap has nothing to fill.

strict_diagnostic raises the warning or the error. It is the one strictness policy of the library.

Algorithm

The method that Julia selects is the algorithm.

  1. span is an AbstractMatrix{Bool}: check its shape against X and return it. A caller's own declaration and a derived PortfolioOptimisers.ListingSpan enter alike, under the public bound.
  2. span is nothing: report through strict_diagnostic when X holds a gap, then return an all-false span of the shape of X, which fills nothing.

Arguments

  • span: The listing statement that the price data holds, observations × assets, or nothing.
  • X: The price values of the window that the fill transforms, observations × assets.
  • strict: If true, a window with a gap from a PricesResult that states no Listing Span raises an ArgumentError. If false, it issues a warning.

Validation

  • size(span) == size(X). Raises a DimensionMismatch.
  • The price data states a span when X holds a gap. Raises an ArgumentError under strict.

Returns

  • span: The listing statement on the price clock of X, or an all-false span when the price data states none.

Related

source
PortfolioOptimisers.gap_fill_open — Function
gap_fill_open(span::AbstractMatrix{Bool}, j::Integer, t::Integer) -> Bool
gap_fill_open(span::Nothing, j::Integer, t::Integer) -> Bool

Tell whether the listing of column j holds from observation t to the end of a window, so that a price at t can seed the next window.

A seed from before an absence books a return across that absence. A later window cannot see an absence at the end of the window before it, so the window that observes the absence must retire the seed. PriceGapFill reads this at the end of the training window, and its online step reads it at the end of each block.

Mathematical definition

\[\begin{align} o_{i} &= \prod_{u = t}^{n} a_{u,\,i}\,. \end{align}\]

Here $i$ is the asset of column j, and $t$ is the row of its last observed price.

Where:

  • $o_{i}$: Open flag of asset $i$, $1$ when no absence from the listing follows its last observed price in the window, so that the price can seed the next window.
  • $a_{t,\,i}$: Listing indicator of asset $i$ at observation $t$ of the price clock, $1$ when the asset is listed and $0$ when it is not.
  • $n$: Price clock length, the number of price observations.

Algorithm

The method that Julia selects is the algorithm.

  1. span is an AbstractMatrix{Bool}: return true when every entry of column j from row t to the last row is true.
  2. span is nothing: return true. Price data that states no span states no absence.

Arguments

  • span: The listing statement of the price data, observations × assets, or nothing.
  • j: Index of the column.
  • t: Row of the price that seeds the next window. It is 1 when the window observes no price of the column and the seed comes from an earlier window.

Returns

  • open::Bool: true when the seed reaches the end of the window inside one listed run.

Related

source
PortfolioOptimisers.gap_fill_seed — Function
gap_fill_seed(fill::CarriedPrice, obs::VecNum, open::Bool) -> Union{Missing, Number}
gap_fill_seed(fill::Num_VecToScaM, obs::VecNum, open::Bool) -> Number

Reduce the observed training prices of one asset to the value that apply_preprocessing replays.

Mathematical definition

Under CarriedPrice:

\[\begin{align} v_{i} &= p_{\max \mathcal{O}_{i},\,i} \quad \text{when } o_{i} = 1\,. \end{align}\]

When $o_{i} = 0$, or when $\mathcal{O}_{i}$ is empty, $v_{i}$ does not exist.

Under a Num_VecToScaM:

\[\begin{align} v_{i} &= \phi\left(\left\{p_{u,\,i} : u \in \mathcal{O}_{i}\right\}\right)\,. \end{align}\]

Here $\mathcal{O}_{i}$ is the observed set of asset $i$ over the training window, and it is not empty.

Where:

  • $v_{i}$: Fitted value of asset $i$, read off its observed prices over the training window.
  • $p_{t,\,i}$: Price of asset $i$ at observation $t$ of the price clock, absent at a gap.
  • $\mathcal{O}_{i}$: Observed set of asset $i$, the observations at which its price is neither missing nor NaN.
  • $o_{i}$: Open flag of asset $i$, $1$ when no absence from the listing follows its last observed price in the window, so that the price can seed the next window.
  • $\phi$: Reduction that a Num_VecToScaM states, from the observed prices of an asset to one number. A plain number is the constant reduction.

Algorithm

The method that Julia selects is the algorithm.

  1. CarriedPrice: return the last entry of obs when open is true. It is the seed for a later window that opens inside a gap. Return missing otherwise, because the seed would cross an absence.
  2. Num_VecToScaM: return the reduction of obs through vec_to_real_measure. The value carries no price, so an absence does not retire it.

Arguments

  • fill: The convention, read off PriceGapFill.
  • obs: The observed training prices of one asset, in observation order. Under CarriedPrice it can be empty, and then open is false.
  • open: Whether the listing holds from the last observed price to the end of the training window, from PortfolioOptimisers.gap_fill_open. It is false when the asset has no observed training price.

Returns

  • v::Union{Missing, Number}: The fitted value of the asset, or missing under CarriedPrice when open is false.

Related

source
PortfolioOptimisers.gap_fill_column! — Function
gap_fill_column!(fill::CarriedPrice, X::AbstractMatrix, span::AbstractMatrix{Bool}, j::Integer, v::Union{Missing, Number}, t0::Integer) -> AbstractMatrix
gap_fill_column!(fill::Num_VecToScaM, X::AbstractMatrix, span::AbstractMatrix{Bool}, j::Integer, v::Number, t0::Integer) -> AbstractMatrix

Write the fill of one column in place, inside the listing of the asset and nowhere else.

Both methods read span before they read the price, so they never write an observation outside the listing, whatever the convention. The Span Rule puts every observation before the first listing of an asset and after its delisting outside the span, so the fill cannot make a price there.

Under CarriedPrice the seed v is a price that the training window observed last, so the method writes it only onto an observation after that window. t0 names the first such observation. A gap before t0 with no observed price before it stays a gap. On the training window, which a Pipeline transforms with the step that it just fitted, every observation comes before t0, so a gap that opens the window does not take the last price of the window.

The carry also stops at an observation outside the listing. A caller's own span can take an asset out of the listing and back in. A gap after the return takes no price from before the absence and no seed, because with such a price the next return is a move across the absence. PortfolioOptimisers.project_span books no return across an absence, and the fill keeps that statement true. The window cannot see an absence before its first row, so the fit that saw the absence gives a missing seed. The method never writes a missing seed.

Mathematical definition

Under CarriedPrice:

\[\begin{align} \tilde{p}_{t,\,i} &= \begin{cases} p_{s_{t,\,i},\,i} & a_{t,\,i} = 1\,,\ t \notin \mathcal{O}_{i}\,,\ s_{t,\,i} \text{ exists} \\ v_{i} & a_{t,\,i} = 1\,,\ t \notin \mathcal{O}_{i}\,,\ s_{t,\,i} \text{ does not exist}\,,\ t \geq t_{0}\,,\ r_{t,\,i} = 1\,,\ v_{i} \text{ exists} \\ p_{t,\,i} & \text{otherwise} \end{cases}\,. \end{align}\]

Under a Num_VecToScaM:

\[\begin{align} \tilde{p}_{t,\,i} &= \begin{cases} v_{i} & a_{t,\,i} = 1\,,\ t \notin \mathcal{O}_{i} \\ p_{t,\,i} & \text{otherwise} \end{cases}\,. \end{align}\]

Here $i$ is the asset of the column, and $\mathcal{O}_{i}$ is its observed set over the window.

Where:

  • $\tilde{p}_{t,\,i}$: Filled price of asset $i$ at observation $t$, the price after the fill.
  • $p_{t,\,i}$: Price of asset $i$ at observation $t$ of the price clock, absent at a gap.
  • $a_{t,\,i}$: Listing indicator of asset $i$ at observation $t$ of the price clock, $1$ when the asset is listed and $0$ when it is not.
  • $\mathcal{O}_{i}$: Observed set of asset $i$, the observations at which its price is neither missing nor NaN.
  • $r_{t,\,i}$: Run start of asset $i$ at observation $t$, the first observation of the unbroken listed run that holds $t$.
  • $s_{t,\,i}$: Carry source of asset $i$ at observation $t$, the last member of $\mathcal{O}_{i}$ before $t$ in the listed run that holds $t$.
  • $v_{i}$: Fitted value of asset $i$, read off its observed prices over the training window.
  • $t_{0}$: Replay start, the first observation of the window after the end of the training window, or $n + 1$ when the window holds none.
  • $n$: Price clock length, the number of price observations.

Algorithm

The method that Julia selects is the algorithm, and the two methods differ in what they write.

  1. CarriedPrice: walk the observation axis with the carry carry, which starts as the seed v, and the flag carried, which states whether the walk can write carry. A missing seed is retired from the start. At an observation outside the listing, clear carried and retire the seed. Inside the listing, set carried from t0 on while the seed is not retired. Take an observed price as the new carry and set carried, or write carry onto a gap when carried is set.
  2. Num_VecToScaM: write v onto every gap inside the listing. The value is already the reduction, so the method carries no price and reads neither an observed price nor t0.

Arguments

  • fill: The convention, read off PriceGapFillResult.
  • X: The price values of the window. The method changes them in place.
  • span: The listing statement that bounds the fill, observations × assets.
  • j: Index of the column to fill.
  • v: The fitted value of the asset. Under CarriedPrice it can be missing, and then the column fills from the prices of the window alone.
  • t0: Index of the first observation after the training window. It is size(X, 1) + 1 when the window holds none, as on the training window itself, and 1 when every observation follows the training window.

Returns

  • X::AbstractMatrix: The same matrix, with column j filled.

Related

source