The online arm of the fold loop: private API

A walk-forward that declares a Fold Fit of OnlineStep sends PortfolioOptimisers.fold_loop down a third arm. It warms one estimator up on the first training window, folds each fold's new observations into it, and hands the callback a PortfolioOptimisers.Fold whose train is nothing — the estimator holds its window — so the fold reads the estimator out through optimise(opt) where a refit would have run. The run reaches the weights of the batch expanding-window walk-forward fold for fold. ADR 0140 records the decision.

PortfolioOptimisers.fit_fold_result — Function

fit_fold_result(opt::NonFiniteAllocationOptimisationEstimator, rd::ReturnsResult, train_idx::VecInt, cols)
fit_fold_result(opt::NonFiniteAllocationOptimisationEstimator, rd::ReturnsResult, ::Nothing, cols)

The fit of one fold, by whether the fold carries a training window.

The two arms of fit_and_predict's estimator form, chosen by dispatch on train_idx. A window fits the estimator over it, optimise(opt, port_opt_view(rd, train_idx, cols)), which is the refit every fold of the batch arms runs. nothing says the estimator holds its window — the online arm of fold_loop has already folded every row of it — so the fold reads the estimator out through optimise(opt) with no returns, and the read-out rebuilds the carrier from the state and runs the ordinary batch path over it. The asset view, when cols is not :, is taken by the caller before either arm, so a stepped estimator is sliced by asset as a cold one is.

Arguments

opt: The fold's estimator, asset-viewed by the caller.
rd: The carrier the fold reads.
train_idx: The fold's training window, or nothing for a stepped estimator.
cols: The fold's asset columns.

Returns

res::OptimisationResult: The fold's fitted result.

Related

source

PortfolioOptimisers.cv_online_info — Function

cv_online_info()

Build the informational message emitted when a cross-validation run takes the online arm. online_folds is the only site that emits it, as run_folds is the only site that emits cv_sequential_info, and it states the one fact that sent the run there: the scheme declares a Fold Fit.

Returns

msg::String: The message.

Related

source

PortfolioOptimisers.thread_online_folds! — Function

thread_online_folds!(predictions, fit_fold, est, folds, prev; rd, train_idx, last_end, pws)

Fold and read out the folds folds of a walk-forward, threading est from one to the next.

The per-fold body the online arm and the resumed arm of fold_loop share, written once so that a resume continues exactly the loop that started the run. For each fold i in folds, it folds the rows the training window has gained since the last fold, (last_end + 1):last(train_idx[i]), into est with partial_fit!; hands the threaded estimator to fit_fold(i, prev, est, rd, nothing), which makes the per-fold copy exactly as the batch arms do; stores the prediction at predictions[i - first(folds) + 1]; and advances prev by advance_previous_fold.

Arguments

predictions: The vector the predictions are written into, one slot per fold of folds.
fit_fold: The per-fold resolution and callback fold_loop builds.
est: The estimator, folded through last_end.
folds: The fold indices to run, a contiguous range of the scheme's enumeration.
prev: The last threadable fold's prediction before first(folds), or nothing.
rd: The carrier.
train_idx: The training windows of every fold, in split order.
last_end: The last row est has folded.
pws: The scheme's Previous-Weights Source, or nothing.

Returns

est: The estimator, folded through last(train_idx[last(folds)]).

Related

source

PortfolioOptimisers.online_folds — Function

online_folds(fit_fold, est, n::Integer, ::Type{ElT}; rd, train_idx, test_idx, fold_view, pws)

Run n folds of a walk-forward by the online step, threading one estimator from fold to fold, and emit cv_online_info.

This is the third arm of fold_loop, taken when the scheme declares a Fold Fit (fold_fit). It does five things, in order.

It refuses an estimator that is not the configuration alone, through assert_online_entry: one carrying a partial-fit state at entry, because the loop starts cold, and one carrying a TimeDependent schedule on a field that carries a state, because a schedule replaces the value a state is threaded through. Both are refused by name before any solve.
It warms up once. Under a multiple-randomised path it takes the path's asset view through fold_view(1) here, because a path is one asset subset crossed with the walk-forward's folds, and the sliced estimator is what it threads (a view slices a state by asset). It then resolves every Online wrapper through update_online_estimator, and folds the first training window train_idx[1] into the estimator with partial_fit!.
Per fold i, it folds the rows the training window has gained since the last fold, (last_end + 1):last(train_idx[i]), into the threaded estimator. A purged scheme folds a row when the training window reaches it and never drops one, so fold i has folded exactly 1:last(train_idx[i]), which are the rows the batch expanding fold reads.
It hands the threaded estimator to fit_fold, which makes the per-fold copy exactly as the batch arms do — schedules resolved against the fold's TimeDependentContext, the previous fold's weights threaded through factory — and calls the callback with a Fold whose train is nothing.
It stores the prediction and threads the estimator on.

Steps 3 to 5 are thread_online_folds!, which the resumed arm shares: a Resume in the estimator slot takes the method below, which skips the warm-up and the folds the Result holds and runs the same body from the fold after them.

The identity the arm keeps is the seam's: the run reaches the weights of the batch expanding walk-forward fold for fold, to the tolerance of the moment layer and of the solver, and the carrier the read-out rebuilds is exactly the batch fold's training window.

fit_fold takes (i, prev, est, rd, train): the fold index, the last threadable fold's prediction or nothing — advanced by threads_weights exactly as run_folds advances it, so a failed step leaves the previous weights where they were — the estimator to resolve, the carrier, and the training window, which is nothing here. ElT is the per-fold result element type, positional for the reason given in parallel_folds.

Arguments

fit_fold: The per-fold resolution and callback fold_loop builds.
est: The estimator, as the caller handed it to the loop.
n: The number of folds.
rd: The carrier.
train_idx: The training windows of every fold, in split order.
test_idx: The test windows of every fold, in split order. Unread here; the resumed arm checks the last held fold against it.
fold_view: The asset view of a multiple-randomised path, or nothing.
pws: The scheme's Previous-Weights Source, or nothing; decides what threads_weights tests.

Validation

Everything assert_online_entry refuses.

Returns

predictions::Vector{ElT}: One prediction per fold, in split order.
est: The threaded estimator, folded through last(train_idx[n]), for the Result to carry.

Related

source

online_folds(fit_fold, r::Resume, n::Integer, ::Type{ElT}; rd, train_idx, test_idx, fold_view, pws)

The resumed arm of fold_loop: continue an online walk-forward from the folds its Result holds.

Taken when the estimator slot holds a Resume. It resolves the declaration at entry — a copy of res.opt through copy_states, the last threadable fold of res.pred as the previous prediction, advanced over the folds the Result holds by advance_previous_fold exactly as the one-shot loop advanced it, and the folds to skip, n_old, read off the state's last held timestamp by resume_fold_count — and the declaration is gone before any verb below the loop meets it. The count comes from the state and not from length(res.pred), because a resumed Result holds the new folds only and the chain Resume(res2) skips every fold before it. The re-entry checks run in order: the held timestamps name a fold and equal their rows (resume_fold_count), the carrier adds a fold (assert_resume_folds), and the last held fold was not partial (assert_resume_full_fold). The folds (n_old + 1):n then run through thread_online_folds!, the body the online arm runs, from the last training end last(train_idx[n_old]). No warm-up runs. The schedule's TimeDependentContext carries the combined enumeration's i and n, and the pinned context — names, static panel, column presence — runs on every delta step through the state's own step.

Arguments

fit_fold: The per-fold resolution and callback fold_loop builds.
r: The declaration, holding the Result to continue.
n: The number of folds the scheme enumerates over the extended history.
rd: The carrier of the full history extended.
train_idx: The training windows of every fold, in split order.
test_idx: The test windows of every fold, in split order.
fold_view: Unread; a multiple-randomised path never reaches this arm.
pws: The scheme's Previous-Weights Source, or nothing.

Validation

Everything resume_fold_count, assert_resume_folds and assert_resume_full_fold refuse.

Returns

predictions::Vector{ElT}: One prediction per new fold, in split order.
est: The threaded estimator, folded through last(train_idx[n]).

Related

source