Step execution
The step execution contract is the only pipeline-aware layer over the estimator families. Each run_step method reads the PipelineContext slots its estimator needs, dispatches to that family's native verb — prior for prior estimators, clusterise for clustering, optimise for optimisers, fit_preprocessing/apply_preprocessing for preprocessing estimators — and writes the slot the family produces.
The estimators themselves live with their own families and know nothing about pipelines; the preprocessing estimators, for instance, are documented under Pre-processing.
PortfolioOptimisers.run_step — Function
run_step(est, ctx::PipelineContext) -> (fitted, ctx′)Execute one pipeline step: fit est on the PipelineContext slots it reads and return the fitted object together with a new context whose written slot is updated.
Each estimator family dispatches to its native verb — prior for prior estimators, clusterise/phylogeny_matrix for phylogeny estimators, optimise for optimisation estimators, fit_preprocessing/apply_preprocessing for preprocessing estimators. The fitted object is what apply_preprocessing later uses to transform unseen data windows; for non-preprocessing steps it is the step's ordinary result.
Estimators whose family is not steppable throw an ArgumentError directing the caller to PipelineStep.
Arguments
est: The step estimator (or aPipelineStepwrapper).ctx: The pipeline context.
Returns
(fitted, ctx′): The fitted object and the updated context.
Related
run_step(
res::NonFiniteAllocationOptimisationResult,
ctx::PipelineContext
) -> Tuple{NonFiniteAllocationOptimisationResult, PipelineContext}
Execute a precomputed optimisation result as the optimisation step: there is nothing to solve, so the result is written to the opt slot as-is and the fold predicts with its weights.
This is how a mixed TimeDependent schedule runs its result entries: the fold-loop swap (update_time_dependent_estimator) replaces the schedule step with entry i, which may be an estimator (the fold optimises) or a result (the fold only predicts). A result cannot consume computed context slots — see maybe_inject_step for the fail-closed injection rules.
Related
run_step(
tts::TrainTestSplit,
ctx::PipelineContext
) -> Tuple{TrainTestSplitResult, PipelineContext}
Execute a TrainTestSplit step: replace the input data slot with the training window, and return the fitted TrainTestSplitResult holding both windows.
The split runs at whichever level the pipeline input supplied — prices when the pipeline was fed price-level data, returns otherwise — so the same estimator serves both. Every later step therefore fits on the training window alone, which is the whole point of pinning the split to the first position.
Related
PortfolioOptimisers.require_slot — Function
require_slot(ctx::PipelineContext, slot::Symbol, est)
Validate that the PipelineContext slot slot is populated before step est runs.
Arguments
ctx: The pipeline context.slot: The required slot, one ofPIPELINE_SLOTS.est: The step about to run, used in the error message.
Returns
nothing.
Related
PortfolioOptimisers.set_slot — Function
set_slot(
ctx::PipelineContext,
slot::Symbol,
val
) -> PipelineContext
Return a new PipelineContext with slot slot set to val and every other slot unchanged.
Arguments
ctx: The pipeline context.slot: The slot to write, one ofPIPELINE_SLOTS.val: The value to write.
Returns
ctx::PipelineContext: The updated context.
Related
PortfolioOptimisers.run_uncertainty_step — Function
run_uncertainty_step(
ue::AbstractUncertaintySetEstimator,
target::Union{Nothing, Symbol},
ctx::PipelineContext
) -> Tuple{Any, PipelineContext}
Execute an uncertainty-set step pinned to a target and merge its result into the uncertainty slot.
The target comes from the PipelineStep wrapper:
:mucomputesmu_ucsand fills the mean half.:sigmacomputessigma_ucsand fills the covariance half.:bothcomputesucs, which derives both halves from one fit — sharing the prior and, for the sampling algorithms, the simulation draws — and is therefore cheaper than the two narrowed calls.
A narrowed step fills its half of the PipelineUncertaintySets pair and leaves the other untouched, so separate :mu and :sigma steps compose. Every populated half must reach the optimiser: each becomes its own routing target — :mu_ucs and :sigma_ucs — and neither is optional, so a set that cannot be routed is rejected rather than dropped and :both requires an optimiser with an ArithmeticReturn and an UncertaintySetVariance risk measure.
Arguments
ue: The uncertainty-set estimator.target::mu,:sigma, or:both; anything else throws anArgumentError.ctx: The pipeline context; requires thereturnsslot.
Returns
(res, ctx′): The computed result and the updated context. For:both,resis the(mu, sigma)PipelineUncertaintySetspair; otherwise it is the singleAbstractUncertaintySetResult.
Related
PortfolioOptimisers.run_constraint_step — Function
run_constraint_step(
ce::AbstractConstraintEstimator,
target::Union{Nothing, Symbol},
ctx::PipelineContext
) -> Tuple{Any, PipelineContext}
Execute a constraint step and append its value, paired with its routing target, to the constraints slot.
Every constraint family that computes a value is a step, and every value it computes has a declared destination — that is the whole of the rule, and pipe_constraint_targets is where it is written down. The destination travels with the value as a TargetedConstraint rather than being re-derived from the result type at injection, so a family whose result type names no unique field (a Threshold, a centrality LinearConstraint, an asset-sets matrix) routes as cleanly as one whose does.
Arguments
ce: The constraint estimator.target: ThePipelineSteptarget, ornothingfor an unwrapped step.ctx: The pipeline context.
Returns
(res, ctx′): The computed value and the updated context.resis the bare value, so what a fitted pipeline reports is the constraint result itself rather than the routing wrapper.
Related
PortfolioOptimisers.resolve_constraint_target — Function
resolve_constraint_target(
ce::AbstractConstraintEstimator,
target::Union{Nothing, Symbol}
) -> Symbol
Resolve the routing target a constraint step writes.
The family declares its targets through pipe_constraint_targets and the step's PipelineStep wrapper supplies target. The three outcomes are the three tuple lengths that declaration can have — no target means the family is not a step, one means the step needs no annotation, several mean it must carry one.
Arguments
ce: The constraint estimator.target: The step's declared target, ornothingwhen it carries no annotation.
Returns
target::Symbol: The resolved routing target.
Related
PortfolioOptimisers.constraint_step_value — Function
constraint_step_value(
ce::WeightBoundsEstimator,
ctx::PipelineContext
) -> WeightBounds
Compute the value a constraint step contributes to the constraints slot.
One method per constraint family, each calling that family's constraint-generation verb. The value is paired with its routing target by run_constraint_step; this method decides only what is computed, never where it lands.
Arguments
ce: The constraint estimator.ctx: The pipeline context.
Returns
- The computed value,
nothing, or a vector of either.
Related
constraint_step_value(
ce::ExposureConstraintEstimator,
ctx::PipelineContext
) -> Any
Compute an ExposureConstraintEstimator step's value: re-base its rows through the loadings of the pipeline's prior into an asset-space LinearConstraint.
This is the one constraint step that reads a computed slot other than :returns. The basis is ctx.prior.rr, so a prior step must come earlier; a prior that carries no regression makes constraint_space_basis throw, which is the intended failure — see ExposureConstraintEstimator.
ctx.returns is passed as well, so a space that names a regression estimator refits the loadings here rather than throwing. That is the one arrangement in which a prior carrying no factor block still admits a factor mandate — see FactorSpace.
The constraint is pinned to the pipeline's prior. Its rows were projected through the loadings this step saw, and a downstream optimiser that refits its own prior does not re-project them. Passing the estimator to the optimiser's lcse field instead recomputes the projection with the optimiser's own prior, per fold, which is what a cross-validated factor mandate needs.
Related
PortfolioOptimisers.pipeline_asset_sets — Function
pipeline_asset_sets(
ctx::PipelineContext,
est
) -> UniverseSets{String, String, String, String, String, Dict{K, V}} where {K<:AbstractString, V}
Build the UniverseSets a constraint-generation step needs from the universe names of the context's returns slot.
Every axis the returns declare is declared here: nx always, and nf whenever the returns carry factors. The factor axis is what an ExposureConstraintEstimator step resolves its names against, and taking it from rd.nf is what makes it agree with the loadings by construction — the columns of rr.M are the factors the regression was fitted on, which are the columns of rd.F.
Constraint estimators referencing groups beyond the plain universe names cannot be satisfied by this minimal set; precompute their result instead, or wrap a callable in a PipelineStep that supplies richer sets.
Arguments
ctx: The pipeline context; requires thereturnsslot.est: The step about to run, used in the error message.
Returns
sets::UniverseSets: Universe sets whosenxentry holds the asset names, plus annfentry holding the factor names when the returns carry them.
Related
PortfolioOptimisers.add_constraint_result — Function
add_constraint_result(ctx::PipelineContext, res::AbstractConstraintResult) -> PipelineContext
add_constraint_result(ctx::PipelineContext, res::AbstractVector) -> PipelineContext
add_constraint_result(ctx::PipelineContext, ::Nothing) -> PipelineContextAppend a constraint result to the constraints slot of the context.
The slot accumulates: the first result is stored as-is, later results widen it into a Vector{AbstractConstraintResult} preserving step order.
Two shapes constraint generation can return are absorbed rather than rejected, because both are ordinary outcomes of a step rather than errors:
nothing, which is whatlinear_constraintsreturns when every row was dropped — a non-strictrun whose names were all unknown, or a re-basis the loadings annihilated. The step contributed no constraint, so the slot is left untouched; the slot's job is to carry constraints, not to re-diagnose a condition generation already decided was recoverable.- a vector, which is what a step wrapping a vector of estimators returns. Its elements are appended individually, so they reach
constraint_targetsas siblings of every other step's result rather than as a nested vector it has no case for.
Arguments
ctx: The pipeline context.target: The routing target the value must land in. The value is paired with it as aTargetedConstraintonly when its own type does not already name that target (seeimplicit_constraint_target).res: The computed value to append,nothing, or a vector of either.
Returns
ctx′::PipelineContext: The updated context.
Related
A TrainTestSplit is the one step whose written slot is not a property of its type: it narrows whichever data slot the pipeline input filled, and declares the sentinel :split (see pipe_writes) so that the generic constructor machinery treats it as writing nothing. Its run_step method and slot declarations are documented with run_step, pipe_reads, and pipe_writes.