Preprocessing partial fit
Incremental updates of the data steps
A Pipeline accepts partial_fit!, and so do the data steps before its prior step, or before its optimiser step when it has no prior. A step that transforms each row on its own updates with each block. It takes a block of observations and returns the transformed block, through PortfolioOptimisers.partial_fit_transform. To do so, PricesToReturns stores the last row of prices it saw, PriceGapFill with a CarriedPrice stores the carried prices, and MissingDataFilter stores its counts of missing values. Call fit_preprocessing on such a step with no data to get its fitted result. A step whose configuration needs a whole window returns false from PortfolioOptimisers.supports_partial_fit, and the pipeline throws an error that names the step before the first update.
PortfolioOptimisers.fit_preprocessing — Method
fit_preprocessing(ptr::PricesToReturns)Reads a stepped PricesToReturns out.
The conversion keeps no fitted value. So its fitted object is the estimator with cache set to nothing, which is the object that the batch fit returns.
Returns
ptr′::PricesToReturns: The estimator with no state.
Related
PortfolioOptimisers.fit_preprocessing — Method
fit_preprocessing(est::PriceGapFill)Reads a stepped PriceGapFill out as the PriceGapFillResult that the batch fit over the same rows gives.
The result holds each asset of the folded blocks, with its last observed price, and the last folded timestamp as the end of the training window. An asset whose carried price an absence retired has a missing seed. An asset that no block priced also has a missing seed.
Algorithm
- Read the state with
partial_fit_cache. - Read into
vthe carried price of each asset whose flag inheldis set, andmissingfor the others. An asset that no block priced has no carried price, so its entry ismissingtoo. The element type ofvdropsMissingwhen no seed ismissing. - Build the
PriceGapFillResultfrom a copy of the names of the state,v, the endteof the state, and thefillandstrictof the step.
Validation
- The step keeps a state.
partial_fit_cachethrows anArgumentErrorotherwise.
Returns
res::PriceGapFillResult: The fitted gap fill of the folded observations.
Related
PortfolioOptimisers.fit_preprocessing — Method
fit_preprocessing(mdf::MissingDataFilter)Reads a stepped MissingDataFilter out as the MissingDataFilterResult that the batch fit over the same rows gives.
Algorithm
- Read the state with
partial_fit_cache. - Keep the assets whose share of missing observations
miss / ndoes not exceedcol_thr, givingkeep, and check that one asset at least survives.share_at_mostcomputes the share in the type ofcol_thr, as the batch fit does. - Build the
MissingDataFilterResultfrom the surviving asset names androw_thr.
Validation
- The step keeps a state.
partial_fit_cachethrows anArgumentErrorotherwise. - One asset at least survives the column filter. An
IsEmptyErroris thrown otherwise.
Returns
res::MissingDataFilterResult: The fitted filter of the folded observations.
Related