GitHub

Step execution: private API

The step execution contract is the only pipeline-aware layer over the estimator families. Each run_step method reads the PipelineContext slots its estimator needs, dispatches to that family's native verb — prior for prior estimators, clusterise for clustering, optimise for optimisers, fit_preprocessing/apply_preprocessing for preprocessing estimators — and writes the slot the family produces.

The estimators themselves live with their own families and know nothing about pipelines; the preprocessing estimators, for instance, are documented under Preprocessing.

PortfolioOptimisers.run_step — Function
run_step(est, ctx::PipelineContext) -> (fitted, ctx′)

Execute one pipeline step: fit est on the PipelineContext slots it reads and return the fitted object together with a new context whose written slot is updated.

Each estimator family dispatches to its native verb — prior for prior estimators, clusterise/phylogeny_matrix for phylogeny estimators, optimise for optimisation estimators, fit_preprocessing/apply_preprocessing for preprocessing estimators. The fitted object is what apply_preprocessing later uses to transform unseen data windows; for non-preprocessing steps it is the step's ordinary result.

Estimators whose family is not steppable throw an ArgumentError directing the caller to PipelineStep.

Arguments

  • est: The step estimator (or a PipelineStep wrapper).
  • ctx: The pipeline context.

Returns

  • (fitted, ctx′): The fitted object and the updated context.

Related

source
run_step(
    res::NonFiniteAllocationOptimisationResult,
    ctx::PipelineContext
) -> Tuple{NonFiniteAllocationOptimisationResult, PipelineContext}

Execute a precomputed optimisation result as the optimisation step: there is nothing to solve, so the result is written to the opt slot as-is and the fold predicts with its weights.

This is how a mixed TimeDependent schedule runs its result entries: the fold-loop swap (update_time_dependent_estimator) replaces the schedule step with entry i, which may be an estimator (the fold optimises) or a result (the fold only predicts). A result cannot consume computed context slots — see maybe_inject_step for the fail-closed injection rules.

Related

source
run_step(
    tts::TrainTestSplit,
    ctx::PipelineContext
) -> Tuple{TrainTestSplitResult, PipelineContext}

Execute a TrainTestSplit step: replace the input data slot with the training window, and return the fitted TrainTestSplitResult holding both windows.

The split runs at whichever level the pipeline input supplied — prices when the pipeline was fed price-level data, returns otherwise — so the same estimator serves both. Every later step therefore fits on the training window alone, which is the whole point of pinning the split to the first position.

Related

source
run_step(o::Online, _::PipelineContext)

Refuses an Online step that reached a fold-less fit, by name.

A wrapper is a declaration the online arm's warm-up resolves; fit(pipe, data) has no warm-up, so a wrapper that reaches it has no buffer to seed and no step to fold, and is refused rather than fitted as the plain estimator it wraps, which would read a batch answer under an online declaration.

Related

source
PortfolioOptimisers.run_uncertainty_step — Function
run_uncertainty_step(
    ue::AbstractUncertaintySetEstimator,
    target::Union{Nothing, Symbol},
    ctx::PipelineContext
) -> Tuple{Any, PipelineContext}

Execute an uncertainty-set step pinned to a target and merge its result into the uncertainty slot.

The target comes from the PipelineStep wrapper:

  • :mu computes mu_ucs and fills the mean half.
  • :sigma computes sigma_ucs and fills the covariance half.
  • :both computes ucs, which derives both halves from one fit — sharing the prior and, for the sampling algorithms, the simulation draws — and is therefore cheaper than the two narrowed calls.

A narrowed step fills its half of the PipelineUncertaintySets pair and leaves the other untouched, so separate :mu and :sigma steps compose. Every populated half must reach the optimiser: each becomes its own routing target — :mu_ucs and :sigma_ucs — and neither is optional, so a set that cannot be routed is rejected rather than dropped and :both requires an optimiser with an ArithmeticReturn and an UncertaintySetVariance risk measure.

Which slot the step reads is decided by reads_prior_result. An estimator with a prior of its own is fitted from the returns slot. One that reads a prior result — an AbstractPriorUncertaintySetEstimator, or a returns-data estimator with pe = nothing — is fitted from the prior slot instead, so a prior step must come earlier; the returns are not read at all, so a pipeline that writes :prior from a precomputed result needs no :returns slot for such a step.

Arguments

  • ue: The uncertainty-set estimator.
  • target: :mu, :sigma, or :both; anything else throws an ArgumentError.
  • ctx: The pipeline context; requires the returns slot, or the prior slot for an estimator that reads a prior result.

Returns

Related

source
PortfolioOptimisers.uncertainty_step_source — Function
uncertainty_step_source(
    ue::AbstractUncertaintySetEstimator,
    ctx::PipelineContext
) -> Union{Tuple{Any}, Tuple{Any, Any}}

Read the slot an uncertainty-set step fits from, as the arguments its verb takes.

An estimator that reads a prior result — reads_prior_result answers true — is fitted from the prior slot, and never touches the returns; every other estimator is fitted from the returns slot, whose X and F its verb takes. The required slot is checked here, so a missing one is refused by name before any fit. The predicate reads the type, so the branch folds.

Arguments

  • ue: The uncertainty-set estimator.
  • ctx: The pipeline context.

Validation

  • The slot the estimator reads is filled, through require_slot.

Returns

  • src::Tuple: (ctx.prior,) for a prior-reading estimator, (ctx.returns.X, ctx.returns.F) otherwise.

Related

source
PortfolioOptimisers.run_constraint_step — Function
run_constraint_step(
    ce::AbstractConstraintEstimator,
    target::Union{Nothing, Symbol},
    ctx::PipelineContext
) -> Tuple{Any, PipelineContext}

Execute a constraint step and append its value, paired with its routing target, to the constraints slot.

Every constraint family that computes a value is a step, and every value it computes has a declared destination — that is the whole of the rule, and pipe_constraint_targets is where it is written down. The destination travels with the value as a TargetedConstraint rather than being re-derived from the result type at injection, so a family whose result type names no unique field (a Threshold, a centrality LinearConstraint, an asset-sets matrix) routes as cleanly as one whose does.

Arguments

  • ce: The constraint estimator.
  • target: The PipelineStep target, or nothing for an unwrapped step.
  • ctx: The pipeline context.

Returns

  • (res, ctx′): The computed value and the updated context. res is the bare value, so what a fitted pipeline reports is the constraint result itself rather than the routing wrapper.

Related

source
PortfolioOptimisers.resolve_constraint_target — Function
resolve_constraint_target(
    ce::AbstractConstraintEstimator,
    target::Union{Nothing, Symbol}
) -> Symbol

Resolve the routing target a constraint step writes.

The family declares its targets through pipe_constraint_targets and the step's PipelineStep wrapper supplies target. The three outcomes are the three tuple lengths that declaration can have — no target means the family is not a step, one means the step needs no annotation, several mean it must carry one.

Arguments

  • ce: The constraint estimator.
  • target: The step's declared target, or nothing when it carries no annotation.

Returns

  • target::Symbol: The resolved routing target.

Related

source
PortfolioOptimisers.pipeline_asset_sets — Function
pipeline_asset_sets(
    ctx::PipelineContext,
    est
) -> UniverseSets{String, String, String, String, String, String, String, Dict{K, V}} where {K<:AbstractString, V}

Build the UniverseSets a constraint-generation step needs from the universe names of the context's returns slot.

Every axis the returns declare is declared here: nx always, and nf whenever the returns carry factors. nf is the default tfkey, the time-series factor axis, and that is the right axis because the returns carry F and nothing else: a cross-sectional model's factors are the exposures its own fit was built from, and no returns result carries them. An ExposureConstraintEstimator step resolves its names against this axis, and taking it from rd.nf is what makes it agree with the loadings by construction — the columns of rr.M are the factors the regression was fitted on, which are the columns of rd.F. A step whose loadings are a CrossSectionalFactorModel therefore finds no cfkey entry here and says so through factor_universe; such a mandate needs sets a caller supplies.

Constraint estimators referencing groups beyond the plain universe names cannot be satisfied by this minimal set; precompute their result instead, or wrap a callable in a PipelineStep that supplies richer sets.

Arguments

  • ctx: The pipeline context; requires the returns slot.
  • est: The step about to run, used in the error message.

Returns

  • sets::UniverseSets: Universe sets whose nx entry holds the asset names, plus an nf entry holding the factor names when the returns carry them.

Related

source
PortfolioOptimisers.add_constraint_result — Function
add_constraint_result(ctx::PipelineContext, res::AbstractConstraintResult) -> PipelineContext
add_constraint_result(ctx::PipelineContext, res::AbstractVector) -> PipelineContext
add_constraint_result(ctx::PipelineContext, ::Nothing) -> PipelineContext

Append a constraint result to the constraints slot of the context.

The slot accumulates: the first result is stored as-is, later results widen it into a Vector{AbstractConstraintResult} preserving step order.

Two shapes constraint generation can return are absorbed rather than rejected, because both are ordinary outcomes of a step rather than errors:

  • nothing, which is what linear_constraints returns when every row was dropped — a non-strict run whose names were all unknown, or a re-basis the loadings annihilated. The step contributed no constraint, so the slot is left untouched; the slot's job is to carry constraints, not to re-diagnose a condition generation already decided was recoverable.
  • a vector, which is what a step wrapping a vector of estimators returns. Its elements are appended individually, so they reach constraint_targets as siblings of every other step's result rather than as a nested vector it has no case for.

Arguments

Returns

  • ctx′::PipelineContext: The updated context.

Related

source

A TrainTestSplit is the one step whose written slot is not a property of its type: it narrows whichever data slot the pipeline input filled, and declares the sentinel :split (see pipe_writes) so that the generic constructor machinery treats it as writing nothing. Its run_step method and slot declarations are documented with run_step, pipe_reads, and pipe_writes.