Price gap fill: private API
PortfolioOptimisers.prices_listing_span — Function
prices_listing_span(pr::PricesResult) -> Option{<:AbstractMatrix{Bool}}Read the Listing Span that the price data pr states, or nothing when it states none.
The price data holds the Listing Span, so a step that needs a span reads it from pr and does not derive one. A PricesResult returns its span field. price_ingestion fills that field, and a PricesResult that a caller builds by hand leaves it nothing. The step that asked then uses its fallback and reports it.
Algorithm
- Return the
spanfield ofpr.
Arguments
pr: The price data that the fill reads.
Returns
span::Option{<:AbstractMatrix{Bool}}: The Listing Span ofpr, ornothing.
Related
PortfolioOptimisers.gap_fill_span — Function
gap_fill_span(span::AbstractMatrix{Bool}, X::AbstractMatrix, strict::Bool) -> AbstractMatrix{Bool}
gap_fill_span(span::Nothing, X::AbstractMatrix, strict::Bool) -> BitMatrixResolve the Listing Span that bounds a PriceGapFill over one window.
The span comes from the price data, because a listing calendar is a fact about the instruments and a window cannot see all of it. Price data that states no span leaves only the window, and the window cannot give the span. A suspension across the edge of a window reads there as an inception or a delisting, so a span derived from the window fills the wrong cells, not fewer cells. The fill therefore takes an all-false span, under which every cell lies outside a listing and the fill writes no price. The function reports this with a warning, or with an error under strict. It reports only when the window holds a gap, because a window with no gap has nothing to fill.
strict_diagnostic raises the warning or the error. It is the one strictness policy of the library.
Algorithm
The method that Julia selects is the algorithm.
spanis anAbstractMatrix{Bool}: check its shape againstXand return it. A caller's own declaration and a derivedPortfolioOptimisers.ListingSpanenter alike, under the public bound.spanisnothing: report throughstrict_diagnosticwhenXholds a gap, then return an all-falsespan of the shape ofX, which fills nothing.
Arguments
span: The listing statement that the price data holds,observations × assets, ornothing.X: The price values of the window that the fill transforms,observations × assets.strict: Iftrue, a window with a gap from aPricesResultthat states no Listing Span raises anArgumentError. Iffalse, it issues a warning.
Validation
size(span) == size(X). Raises aDimensionMismatch.- The price data states a span when
Xholds a gap. Raises anArgumentErrorunderstrict.
Returns
span: The listing statement on the price clock ofX, or an all-falsespan when the price data states none.
Related
PortfolioOptimisers.gap_fill_open — Function
gap_fill_open(span::AbstractMatrix{Bool}, j::Integer, t::Integer) -> Bool
gap_fill_open(span::Nothing, j::Integer, t::Integer) -> BoolTell whether the listing of column j holds from observation t to the end of a window, so that a price at t can seed the next window.
A seed from before an absence books a return across that absence. A later window cannot see an absence at the end of the window before it, so the window that observes the absence must retire the seed. PriceGapFill reads this at the end of the training window, and its online step reads it at the end of each block.
Mathematical definition
\[\begin{align} o_{i} &= \prod_{u = t}^{n} a_{u,\,i}\,. \end{align}\]
Here $i$ is the asset of column j, and $t$ is the row of its last observed price.
Where:
- $o_{i}$: Open flag of asset $i$, $1$ when no absence from the listing follows its last observed price in the window, so that the price can seed the next window.
- $a_{t,\,i}$: Listing indicator of asset $i$ at observation $t$ of the price clock, $1$ when the asset is listed and $0$ when it is not.
- $n$: Price clock length, the number of price observations.
Algorithm
The method that Julia selects is the algorithm.
spanis anAbstractMatrix{Bool}: returntruewhen every entry of columnjfrom rowtto the last row istrue.spanisnothing: returntrue. Price data that states no span states no absence.
Arguments
span: The listing statement of the price data,observations × assets, ornothing.j: Index of the column.t: Row of the price that seeds the next window. It is1when the window observes no price of the column and the seed comes from an earlier window.
Returns
open::Bool:truewhen the seed reaches the end of the window inside one listed run.
Related
PortfolioOptimisers.gap_fill_seed — Function
gap_fill_seed(fill::CarriedPrice, obs::VecNum, open::Bool) -> Union{Missing, Number}
gap_fill_seed(fill::Num_VecToScaM, obs::VecNum, open::Bool) -> NumberReduce the observed training prices of one asset to the value that apply_preprocessing replays.
Mathematical definition
Under CarriedPrice:
\[\begin{align} v_{i} &= p_{\max \mathcal{O}_{i},\,i} \quad \text{when } o_{i} = 1\,. \end{align}\]
When $o_{i} = 0$, or when $\mathcal{O}_{i}$ is empty, $v_{i}$ does not exist.
Under a Num_VecToScaM:
\[\begin{align} v_{i} &= \phi\left(\left\{p_{u,\,i} : u \in \mathcal{O}_{i}\right\}\right)\,. \end{align}\]
Here $\mathcal{O}_{i}$ is the observed set of asset $i$ over the training window, and it is not empty.
Where:
- $v_{i}$: Fitted value of asset $i$, read off its observed prices over the training window.
- $p_{t,\,i}$: Price of asset $i$ at observation $t$ of the price clock, absent at a gap.
- $\mathcal{O}_{i}$: Observed set of asset $i$, the observations at which its price is neither
missingnorNaN. - $o_{i}$: Open flag of asset $i$, $1$ when no absence from the listing follows its last observed price in the window, so that the price can seed the next window.
- $\phi$: Reduction that a
Num_VecToScaMstates, from the observed prices of an asset to one number. A plain number is the constant reduction.
Algorithm
The method that Julia selects is the algorithm.
CarriedPrice: return the last entry ofobswhenopenistrue. It is the seed for a later window that opens inside a gap. Returnmissingotherwise, because the seed would cross an absence.Num_VecToScaM: return the reduction ofobsthroughvec_to_real_measure. The value carries no price, so an absence does not retire it.
Arguments
fill: The convention, read offPriceGapFill.obs: The observed training prices of one asset, in observation order. UnderCarriedPriceit can be empty, and thenopenisfalse.open: Whether the listing holds from the last observed price to the end of the training window, fromPortfolioOptimisers.gap_fill_open. It isfalsewhen the asset has no observed training price.
Returns
v::Union{Missing, Number}: The fitted value of the asset, ormissingunderCarriedPricewhenopenisfalse.
Related
PortfolioOptimisers.gap_fill_column! — Function
gap_fill_column!(fill::CarriedPrice, X::AbstractMatrix, span::AbstractMatrix{Bool}, j::Integer, v::Union{Missing, Number}, t0::Integer) -> AbstractMatrix
gap_fill_column!(fill::Num_VecToScaM, X::AbstractMatrix, span::AbstractMatrix{Bool}, j::Integer, v::Number, t0::Integer) -> AbstractMatrixWrite the fill of one column in place, inside the listing of the asset and nowhere else.
Both methods read span before they read the price, so they never write an observation outside the listing, whatever the convention. The Span Rule puts every observation before the first listing of an asset and after its delisting outside the span, so the fill cannot make a price there.
Under CarriedPrice the seed v is a price that the training window observed last, so the method writes it only onto an observation after that window. t0 names the first such observation. A gap before t0 with no observed price before it stays a gap. On the training window, which a Pipeline transforms with the step that it just fitted, every observation comes before t0, so a gap that opens the window does not take the last price of the window.
The carry also stops at an observation outside the listing. A caller's own span can take an asset out of the listing and back in. A gap after the return takes no price from before the absence and no seed, because with such a price the next return is a move across the absence. PortfolioOptimisers.project_span books no return across an absence, and the fill keeps that statement true. The window cannot see an absence before its first row, so the fit that saw the absence gives a missing seed. The method never writes a missing seed.
Mathematical definition
Under CarriedPrice:
\[\begin{align} \tilde{p}_{t,\,i} &= \begin{cases} p_{s_{t,\,i},\,i} & a_{t,\,i} = 1\,,\ t \notin \mathcal{O}_{i}\,,\ s_{t,\,i} \text{ exists} \\ v_{i} & a_{t,\,i} = 1\,,\ t \notin \mathcal{O}_{i}\,,\ s_{t,\,i} \text{ does not exist}\,,\ t \geq t_{0}\,,\ r_{t,\,i} = 1\,,\ v_{i} \text{ exists} \\ p_{t,\,i} & \text{otherwise} \end{cases}\,. \end{align}\]
Under a Num_VecToScaM:
\[\begin{align} \tilde{p}_{t,\,i} &= \begin{cases} v_{i} & a_{t,\,i} = 1\,,\ t \notin \mathcal{O}_{i} \\ p_{t,\,i} & \text{otherwise} \end{cases}\,. \end{align}\]
Here $i$ is the asset of the column, and $\mathcal{O}_{i}$ is its observed set over the window.
Where:
- $\tilde{p}_{t,\,i}$: Filled price of asset $i$ at observation $t$, the price after the fill.
- $p_{t,\,i}$: Price of asset $i$ at observation $t$ of the price clock, absent at a gap.
- $a_{t,\,i}$: Listing indicator of asset $i$ at observation $t$ of the price clock, $1$ when the asset is listed and $0$ when it is not.
- $\mathcal{O}_{i}$: Observed set of asset $i$, the observations at which its price is neither
missingnorNaN. - $r_{t,\,i}$: Run start of asset $i$ at observation $t$, the first observation of the unbroken listed run that holds $t$.
- $s_{t,\,i}$: Carry source of asset $i$ at observation $t$, the last member of $\mathcal{O}_{i}$ before $t$ in the listed run that holds $t$.
- $v_{i}$: Fitted value of asset $i$, read off its observed prices over the training window.
- $t_{0}$: Replay start, the first observation of the window after the end of the training window, or $n + 1$ when the window holds none.
- $n$: Price clock length, the number of price observations.
Algorithm
The method that Julia selects is the algorithm, and the two methods differ in what they write.
CarriedPrice: walk the observation axis with the carrycarry, which starts as the seedv, and the flagcarried, which states whether the walk can writecarry. Amissingseed is retired from the start. At an observation outside the listing, clearcarriedand retire the seed. Inside the listing, setcarriedfromt0on while the seed is not retired. Take an observed price as the newcarryand setcarried, or writecarryonto a gap whencarriedis set.Num_VecToScaM: writevonto every gap inside the listing. The value is already the reduction, so the method carries no price and reads neither an observed price nort0.
Arguments
fill: The convention, read offPriceGapFillResult.X: The price values of the window. The method changes them in place.span: The listing statement that bounds the fill,observations × assets.j: Index of the column to fill.v: The fitted value of the asset. UnderCarriedPriceit can bemissing, and then the column fills from the prices of the window alone.t0: Index of the first observation after the training window. It issize(X, 1) + 1when the window holds none, as on the training window itself, and1when every observation follows the training window.
Returns
X::AbstractMatrix: The same matrix, with columnjfilled.
Related