Price ingestion

struct PriceIngestion{__T_join_method, __T_collapse_args, __T_span, __T_strict} <: AbstractEstimator

Estimator assembling raw price series into the span-carrying price carrier the ingestion layer converts.

PriceIngestion runs once on the whole panel, and it is deliberately not a Pipeline step. The rule that puts it outside is the observation clock: unification, the factor and benchmark join, and the frequency collapse each move or renumber the observations, folds are cut once on the carrier's clock, and a hyperparameter that changes the test set cannot be scored against one that does not. A step that only touches values leaves the clock alone and stays inside.

Running outside the Pipeline is also what lets the Span Rule read the whole panel. A step only ever sees a window, and a window-local span reads a delisting straddling the window end as an asset that was never listed. What licenses the panel-wide read where a split otherwise forbids work before it is that a listing calendar is a fact about the instruments, not an estimate from returns.

The carrier it emits holds the derived Listing Span in its span field, so the conversion can project it onto the returns clock and hand the returns carrier an AssetPanel stating the universe. A caller holding their own listing calendar passes it as span and replaces the Span Rule's answer outright.

The asset table states the clock. It states the universe, so it states the clock, and the factor, benchmark and implied-volatility series are aligned to it under the default join_method = :left, padded where they are silent. The layer names what it padded — which table, how many observations, which columns — warning by default and refusing under strict, because a carried absence is invisible where a refused one was not and those axes carry no universe that would name it later. Each field states something about the sources the layer cannot derive: which clock is authoritative, what frequency an analysis wants, which listing calendar the caller holds, and whether a padded covariate is acceptable.

Fields

• join_method: How the factor and benchmark series are joined onto the asset clock, as TimeSeries.merge takes it. :left keeps the asset clock and pads the others where they are silent; :outer takes the union of the clocks, padding every table; :inner takes their intersection, padding none.

• collapse_args: Arguments for collapsing the joined series to a lower frequency, as TimeSeries.collapse takes them. Empty leaves the clock alone.

• span: Optional listing statement of the caller's own, price observations × assets, replacing the Span Rule's answer outright. A listing calendar, or a constituency that leaves and rejoins, is any AbstractMatrix{Bool}. nothing derives the span from the gaps.

• strict: Whether a padded observation refuses the ingestion. false names what the join and the alignment padded in a warning; true raises an ArgumentError carrying the same report.

Constructors

PriceIngestion(;    join_method::Symbol = :left,    collapse_args::Tuple = (),    span::Option{<:AbstractMatrix{Bool}} = nothing,    strict::Bool = false,) -> PriceIngestion

Keywords correspond to the struct's fields.

Examples

julia> X = TimeArray(Date(2020, 1, 1):Day(1):Date(2020, 1, 3),                     [NaN 101.0; 102.0 103.0; 104.0 105.0], ["A", "B"]);julia> pr = price_ingestion(PriceIngestion(), X);julia> pr.spanListingSpan(3 × 2)julia> Matrix(pr.span)3×2 Matrix{Bool}: 0  1 1  1 1  1

Related

• price_ingestion
• PricesResult
• PricesToReturns
• listing_span
• universe_masks
• Option

price_ingestion(est::PriceIngestion, X::TimeSeries.TimeArray;
                F::Option{<:TimeSeries.TimeArray} = nothing,
                B::Option{<:TimeSeries.TimeArray} = nothing,
                iv::Option{<:TimeSeries.TimeArray} = nothing,
                ivpa::Option{<:Num_VecNum} = nothing,
                pnl::Option{<:AssetPanel} = nothing) -> PricesResult
price_ingestion(est::PriceIngestion, pr::PricesResult) -> PricesResult

Assemble raw price series into the span-carrying price carrier.

The three clock-moving steps run here rather than in a Pipeline, and the Span Rule reads the whole panel once they have. What the caller gets back is an ordinary PricesResult whose span field states which assets are listed at each observation, ready for PricesToReturns to project onto the returns clock.

The layer spells an absent price NaN, which is the library's one spelling for absence, and the conversion carries it into the returns rather than deleting the observation or the asset that holds one. An absence is carried on every axis the layer touches: a factor, benchmark or implied-volatility series silent at an observation of the emitted clock is padded there, and the padding is named — warned by default, refused under strict — because those axes carry no universe that would name it later.

The value type the carrier holds is derived from the series rather than named. The asset, factor and benchmark types are promoted to one, because the join lays them in one table, and that type is widened only as the return arithmetic would widen it: a Float32 panel stays Float32, Float32 beside Float64 joins in Float64, and an integer panel takes the floating-point type that represents it. A type that cannot spell an absence is refused by name at the first gap or padded observation it would have to spell.

Algorithm

1. Check that the asset, factor and benchmark series can still be named after the join with assert_distinct_series_names. The join renames a name two tables share, so a block would otherwise be taken apart into another block's column.
2. Derive the unification target: promote the value types the three price tables contribute through series_value_type, and widen the result with absence_type. Unify the absent-price convention of every series in that type with unify_gaps, so that a gap means one thing from here on.
3. Join the factor and the benchmark series onto the asset clock under join_method. A left join keeps the asset clock and pads the others where they are silent; an outer join adds the rows one series has and another does not, padding every table; an inner join keeps the rows every table has. A join that pads asks assert_pad_spellable first, which refuses by name a type that cannot spell the pad.
4. Write the padding report for the asset, factor and benchmark tables against the joined clock with padding_report_line, before the collapse renumbers it.
5. Collapse the joined series to a lower frequency when collapse_args is non-empty, which renumbers every observation.
6. Split the joined series back into their asset, factor and benchmark blocks, all now on one clock.
7. Align the implied volatilities to that clock with align_series, in the type absence_type derives from their own, padding the observations they are silent at; write their report line; and carry ivpa through. An implied volatility is a volatility rather than a price, so it is carried, never converted.
8. Name what was padded with assert_join_padding, which warns, or refuses under strict.
9. Put a caller's AssetPanel on the emitted clock with project_panel_clock. A caller states a Panel Field on the clock of the table they hold, and the collapse is the only step here that renumbers it, so this is where it is projected: the aggregated period takes the values of the row at its representative timestamp, which is last-observation semantics and matches LastObservation. A static panel has no observation axis and is carried through untouched.
10. Read the Listing Span off the asset block with listing_span, unless the caller declared one, in which case theirs is taken outright.
11. Return the PricesResult carrying all of it.

The emitted clock is the asset table's under the default join, so timestamp(pr.X) == timestamp(X) unless collapse_args is non-empty, and a caller declaring their own listing calendar can size it against the table they hold.

Arguments

• est: The PriceIngestion estimator.
• X: Asset prices, observations × assets.
• F: Optional factor prices.
• B: Optional benchmark prices, one column or one per asset.
• iv: Optional implied volatilities, one column per asset, on any clock: aligned to the emitted one and padded NaN where silent.
• ivpa: Optional implied volatility adjustment.
• pnl: Optional AssetPanel of Panel Fields the caller already holds. Its Panel Fields are kept; its masks are not read, because a time-varying panel carries an amsk by construction, so the field cannot carry a declaration. A listing statement is span.
• pr: A PricesResult, for the second form, whose series are re-ingested. A span the carrier states is kept unless est.span overrides it; the Span Rule runs only where neither states one.

Validation

• !isempty(X). Raises an IsEmptyError.
• The asset, factor and benchmark column names are pairwise disjoint, and none of them is timestamp. Raises a ConflictingArgumentError naming the offending columns.
• A declared span is size(values(X)) after the join and the collapse. Raises a DimensionMismatch.
• The value type carries an absence, wherever one must be spelled. Raises a DomainError naming the type, from absent_value.
• Nothing was padded, under strict. Raises an ArgumentError carrying the padding report, from assert_join_padding; a warning otherwise.

Returns

• pr::PricesResult: The price carrier, holding the Listing Span in its span field.

Examples

julia> X = TimeArray(Date(2020, 1, 1):Day(1):Date(2020, 1, 4),                     [100.0 missing; 102.0 103.0; 101.0 104.0; 103.0 102.0], ["A", "B"]);julia> pr = price_ingestion(PriceIngestion(), X);julia> values(pr.X)[1, :]2-element Vector{Float64}: 100.0 NaNjulia> Matrix(pr.span)4×2 Matrix{Bool}: 1  0 1  1 1  1 1  1

Related

• PriceIngestion
• PricesResult
• PricesToReturns
• listing_span
• unify_gaps
• series_value_type
• absence_type
• assert_pad_spellable
• align_series
• padding_report_line
• assert_join_padding
• project_panel_clock
• LastObservation