Base Pipeline

A pipeline reifies an end-to-end workflow — price preprocessing, prices-to-returns conversion, returns preprocessing, prior estimation, phylogeny, uncertainty sets, constraint generation, and optimisation — as an ordered list of steps executed left-to-right over an accumulating context. Pipelines widen the cross-validation and hyperparameter-tuning boundary to the entire workflow, data preparation included. See docs/adr/0028-pipeline-workflow-estimator.md for the design rationale.

PortfolioOptimisers.PipelineStep — Type

struct PipelineStep{__T_est, __T_reads, __T_writes, __T_target} <: AbstractEstimator

Explicit pipeline step wrapper — used when a step's slots or its routing intent must be stated rather than inferred.

Most estimators are used as pipeline steps directly: their family determines which PipelineContext slots they read and write via pipe_reads/pipe_writes. PipelineStep covers the two cases that dispatch alone cannot settle:

Slots dispatch cannot infer: a custom callable, or an estimator routed to a nonstandard slot. reads and writes supply what the family would otherwise declare. This includes a bare-callable TimeDependent schedule of optimisers (TimeDependent(ctx -> optimiser)), whose output kind is not in its type: it enters via PipelineStep(; est = td, writes = :opt) and its output is type-checked when the fold loop swaps it in (see TD_OptE_Opt_Inferable).
Routing intent dispatch must not guess: an uncertainty-set estimator writes the uncertainty slot either way, so the slot is never in doubt; what the wrapper declares through target is which parameters you want bounded — :mu, :sigma, or :both. Since ucs derives both halves from a single fit, this is a statement of intent, not a disambiguation, and every populated half must reach the optimiser or inject_context rejects it.

Fields

est: The wrapped step: an estimator or a callable.

reads: Slots the step requires to be populated before it runs (a subset of PIPELINE_SLOTS, as a tuple).

writes: Slot the step writes (one of PIPELINE_SLOTS).

target: Optional routing annotation for heterogeneous slots (one of PIPELINE_STEP_TARGETS, or nothing).

Constructors

PipelineStep(;    est::Union{<:AbstractEstimator, <:Function},    writes::Symbol,    reads::Tuple{Vararg{Symbol}} = (),    target::Option{Symbol} = nothing,) -> PipelineStep

Keywords correspond to the struct's fields.

Validation

writes in PIPELINE_SLOTS.
all(r -> r in PIPELINE_SLOTS, reads).
isnothing(target) || target in PIPELINE_STEP_TARGETS.
A TimeDependent est must be an optimiser-position schedule (TD_OptE_Opt) and declare writes = :opt: schedules of non-optimiser families are not steppable — a per-fold prior/constraint/… is spelled as a TimeDependent field of the optimisation step instead.

Examples

julia> ps = PipelineStep(; est = NormalUncertaintySet(), reads = (:returns,),                         writes = :uncertainty, target = :mu);julia> PortfolioOptimisers.pipe_writes(ps):uncertaintyjulia> PortfolioOptimisers.pipe_reads(ps)(:returns,)

Related

source

PortfolioOptimisers.pipe_constraint_targets — Function

pipe_constraint_targets(
    _::AbstractConstraintEstimator
) -> Tuple{Symbol}

The routing targets a constraint family's step can write.

This is the one declaration of the estimator → target half of the seam, and it has three readers, which is why it exists as a table rather than as scattered knowledge:

run_constraint_step resolves the target and pairs it with the computed value.
pipe_required_targets hands it to assert_routable, so a step whose target the terminal optimiser cannot receive is refused when the Pipeline is built rather than after the first fold has run.
PipelineStep validates a declared target against it.

The tuple's length is the contract:

One target: the family names its destination, and the step needs no annotation.
Several: the destination is a real choice the result cannot express, so the step must name one through its PipelineStep wrapper. This is the same rule an uncertainty-set step follows.
None: the family has no value to contribute to the constraints slot, so it is not a step. JuMPConstraintEstimator is the case — it is configuration for the model, not a computation over data.

Arguments

ce: A constraint estimator.

Returns

A tuple of routing targets.

Examples

julia> PortfolioOptimisers.pipe_constraint_targets(WeightBoundsEstimator())(:wb,)

Related

source