Step execution: private API
A run_step method is the only code that connects an estimator family to a pipeline. It reads the slots of the PipelineContext that its estimator needs, and calls the function that the family uses outside a pipeline: prior for a prior estimator, clusterise for a clustering estimator, optimise for an optimiser, and fit_preprocessing and apply_preprocessing for a preprocessing estimator. It then writes the slot that the family produces.
The estimators do not depend on the pipeline code, and their docstrings are with their own families. The preprocessing estimators, for example, are on the preprocessing page.
PortfolioOptimisers.run_step — Function
run_step(est, ctx::PipelineContext, ps::Option{PipelineStep} = nothing) -> (fitted, ctx′)Execute one pipeline step: fit est on the PipelineContext slots it reads and return the fitted object together with a new context whose written slot is updated.
A PipelineStep checks the slots that it declares, then runs its estimator with itself in the slot ps. A bare step runs with ps = nothing. Three estimators read the wrapper. A callable writes its value to the slot that ps.writes names. An uncertainty-set estimator reads the parameters that it bounds from ps.target, and a bare one is refused. A constraint estimator resolves its routing target from ps.target, or from its family when it runs bare. Every other estimator ignores the slot.
Each estimator family dispatches to its native verb — prior for prior estimators, clusterise/phylogeny_matrix for phylogeny estimators, optimise for optimisation estimators, fit_preprocessing/apply_preprocessing for preprocessing estimators. The fitted object is what apply_preprocessing later uses to transform unseen data windows; for non-preprocessing steps it is the step's ordinary result.
Estimators whose family is not steppable throw an ArgumentError directing the caller to PipelineStep.
A step that reads the returns reads them from the source that its own reader states in its x_src field, as the same reader does in the slot of an optimiser. A PipelineStep wrapper holds no source.
Arguments
est: The step estimator (or aPipelineStepwrapper).ctx: The pipeline context.ps: The wrapper that holdsest, ornothingfor a bare step.
Returns
(fitted, ctx′): The fitted object and the updated context.
Related
run_step(
res::NonFiniteAllocationOptimisationResult,
ctx::PipelineContext
) -> Tuple{NonFiniteAllocationOptimisationResult, PipelineContext}
run_step(
res::NonFiniteAllocationOptimisationResult,
ctx::PipelineContext,
::Union{Nothing, PipelineStep}
) -> Tuple{NonFiniteAllocationOptimisationResult, PipelineContext}
Execute a precomputed optimisation result as the optimisation step: there is nothing to solve, so the result is written to the opt slot as-is and the fold predicts with its weights.
This is how a mixed TimeDependent schedule runs its result entries: the fold-loop swap (update_time_dependent_estimator) replaces the schedule step with entry i, which may be an estimator (the fold optimises) or a result (the fold only predicts). A result cannot consume computed context slots — see maybe_inject_step for the fail-closed injection rules.
Related
run_step(
ue::AbstractUncertaintySetEstimator,
ctx::PipelineContext,
ps::PipelineStep
) -> Tuple{Any, PipelineContext}
Execute an uncertainty-set step pinned to a target and merge its result into the uncertainty slot.
The target comes from the PipelineStep wrapper:
:mucomputesmu_ucsand fills the mean half.:sigmacomputessigma_ucsand fills the covariance half.:bothcomputesucs, which derives both halves from one fit — sharing the prior and, for the sampling algorithms, the simulation draws — and is therefore cheaper than the two narrowed calls.
A narrowed step fills its half of the PipelineUncertaintySets pair and leaves the other untouched, so separate :mu and :sigma steps compose. Every populated half must reach the optimiser: each becomes its own routing target — :mu_ucs and :sigma_ucs — and neither is optional, so a set that cannot be routed is rejected rather than dropped and :both requires an optimiser with an ArithmeticReturn and an UncertaintySetVariance risk measure.
Which slot the step reads is decided by reads_prior_result. An estimator with a prior of its own is fitted from the returns slot. One that reads a prior result — an AbstractPriorUncertaintySetEstimator, or a returns-data estimator with pe = nothing — is fitted from the prior slot instead, so a prior step must come earlier; the returns are not read at all, so a pipeline that writes :prior from a precomputed result needs no :returns slot for such a step.
Arguments
ue: The uncertainty-set estimator.ctx: The pipeline context; requires thereturnsslot, or thepriorslot for an estimator that reads a prior result.ps: The wrapper ofue. Itstargetis:mu,:sigma, or:both; anything else throws anArgumentError.
Returns
(res, ctx′): The computed result and the updated context. For:both,resis the(mu, sigma)PipelineUncertaintySetspair; otherwise it is the singleAbstractUncertaintySetResult.
Related
run_step(
ce::AbstractConstraintEstimator,
ctx::PipelineContext
) -> Tuple{Any, PipelineContext}
run_step(
ce::AbstractConstraintEstimator,
ctx::PipelineContext,
target::Union{Nothing, Symbol}
) -> Tuple{Any, PipelineContext}
Execute a constraint step and append its value, paired with its routing target, to the constraints slot.
Every constraint family that computes a value is a step, and every value it computes has a declared destination — that is the whole of the rule, and pipe_constraint_targets is where it is written down. The destination travels with the value as a TargetedConstraint rather than being re-derived from the result type at injection, so a family whose result type names no unique field (a Threshold, a centrality LinearConstraint, an asset-sets matrix) routes as cleanly as one whose does.
Arguments
ce: The constraint estimator.ctx: The pipeline context.target: ThePipelineSteptarget, ornothingfor an unwrapped step. The method for aPipelineSteppasses thetargetof the wrapper.
Returns
(res, ctx′): The computed value and the updated context.resis the bare value, so what a fitted pipeline reports is the constraint result itself rather than the routing wrapper.
Related
run_step(
tts::TrainTestSplit,
ctx::PipelineContext
) -> Tuple{TrainTestSplitResult, PipelineContext}
run_step(
tts::TrainTestSplit,
ctx::PipelineContext,
::Union{Nothing, PipelineStep}
) -> Tuple{TrainTestSplitResult, PipelineContext}
Execute a TrainTestSplit step: replace the input data slot with the training window, and return the fitted TrainTestSplitResult holding both windows.
The split runs at whichever level the pipeline input supplied — prices when the pipeline was fed price-level data, returns otherwise — so the same estimator serves both. Every later step therefore fits on the training window alone, which is the whole point of pinning the split to the first position.
Related
run_step(o::Online, ::PipelineContext)
run_step(
o::Online,
::PipelineContext,
::Union{Nothing, PipelineStep}
)
Refuses an Online step that reaches a fit with no folds, with an error that names it.
The warm-up of the online arm resolves a wrapper, and fit(pipe, data) runs no warm-up. A wrapper that reaches it has no buffer to seed and no step to fold. A fit of the wrapped estimator as a plain one would return a batch answer under an online declaration, so the method refuses the step instead.
Validation
- The method always throws an
ArgumentError. The message names the wrapped estimator and the two ways out: an Online Scheme, or the plain estimator.
Related
PortfolioOptimisers.require_slot — Function
require_slot(ctx::PipelineContext, slot::Symbol, est)
Validate that the PipelineContext slot slot is populated before step est runs.
Arguments
ctx: The pipeline context.slot: The required slot, one ofPIPELINE_SLOTS.est: The step about to run, used in the error message.
Returns
nothing.
Related
PortfolioOptimisers.set_slot — Function
set_slot(
ctx::PipelineContext,
slot::Symbol,
val
) -> PipelineContext
Return a new PipelineContext with slot slot set to val and every other slot unchanged.
Arguments
ctx: The pipeline context.slot: The slot to write, one ofPIPELINE_SLOTS.val: The value to write.
Returns
ctx::PipelineContext: The updated context.
Related
PortfolioOptimisers.uncertainty_step_source — Function
uncertainty_step_source(
ue::AbstractUncertaintySetEstimator,
ctx::PipelineContext
) -> Union{Tuple{Any, Tuple{Any, Any}}, Tuple{AbstractUncertaintySetEstimator, Tuple{Any}}}
Read the slot an uncertainty-set step fits from, as the arguments its verb takes.
An estimator that reads a prior result — reads_prior_result answers true — is fitted from the prior slot, and never touches the returns; every other estimator is fitted from the returns slot, whose X and F its verb takes. The estimator reads the source that its own prior states, returns_source of ue.pe, as the verb over a ReturnsResult reads it. The observations of a Shared Series gap that the fit reads are dropped first with shared_series_fit: a gap of a vector benchmark under ExcessSource(), and a gap of the factor returns when the prior of the estimator reads them. Each fixed observation weight in the tree of the estimator loses the same observations, so the function returns the cut estimator too. When the optimiser receives the set, the returns slot already has the gaps of its group dropped, see run_group_step, so a raw prior in an excess group reads the same observations as the other readers of the group. The asset returns are then read with benchmark_relative_returns under the same source. The required slot is checked here, so a missing one is refused with an error that names it before any fit. The predicate reads the type, so the branch folds.
Arguments
ue: The uncertainty-set estimator.ctx: The pipeline context.
Validation
- The slot the estimator reads is filled, through
require_slot. - The rules of
benchmark_relative_returnsandshared_series_fit.
Returns
ue::AbstractUncertaintySetEstimator: The estimator that the fit reads, cut to the observations that the fit keeps. A prior-reading estimator comes back with no change.src::Tuple:(ctx.prior,)for a prior-reading estimator,(X, F)otherwise, whereXis the asset returns from the source ofue.peand both hold the observations that the fit keeps.
Related
PortfolioOptimisers.resolve_constraint_target — Function
resolve_constraint_target(
ce::AbstractConstraintEstimator,
target::Union{Nothing, Symbol}
) -> Symbol
Resolve the routing target a constraint step writes.
The family declares its targets through pipe_constraint_targets and the step's PipelineStep wrapper supplies target. The three outcomes are the three tuple lengths that declaration can have — no target means the family is not a step, one means the step needs no annotation, several mean it must carry one.
Arguments
ce: The constraint estimator.target: The step's declared target, ornothingwhen it carries no annotation.
Returns
target::Symbol: The resolved routing target.
Related
PortfolioOptimisers.pipeline_asset_sets — Function
pipeline_asset_sets(
ctx::PipelineContext,
est
) -> UniverseSets{String, String, String, String, String, String, String, Dict{K, V}} where {K<:AbstractString, V}
Build the UniverseSets a constraint-generation step needs from the universe names of the context's returns slot.
Every axis the returns declare is declared here: nx always, and nf whenever the returns carry factors. nf is the default tfkey, the time-series factor axis, and that is the right axis because the returns carry F and nothing else: a cross-sectional model's factors are the exposures its own fit was built from, and no returns result carries them. An ExposureConstraintEstimator step resolves its names against this axis, and taking it from rd.nf is what makes it agree with the loadings by construction — the columns of rr.M are the factors the regression was fitted on, which are the columns of rd.F. A step whose loadings are a CrossSectionalFactorModel therefore finds no cfkey entry here and says so through factor_universe; such a mandate needs sets a caller supplies.
Constraint estimators referencing groups beyond the plain universe names cannot be satisfied by this minimal set; precompute their result instead, or wrap a callable in a PipelineStep that supplies richer sets.
Arguments
ctx: The pipeline context; requires thereturnsslot.est: The step about to run, used in the error message.
Returns
sets::UniverseSets: Universe sets whosenxentry holds the asset names, plus annfentry holding the factor names when the returns carry them.
Related
PortfolioOptimisers.add_constraint_result — Function
add_constraint_result(ctx::PipelineContext, res::AbstractConstraintResult) -> PipelineContext
add_constraint_result(ctx::PipelineContext, res::AbstractVector) -> PipelineContext
add_constraint_result(ctx::PipelineContext, ::Nothing) -> PipelineContextAppend a constraint result to the constraints slot of the context.
The slot accumulates: the first result is stored as-is, later results widen it into a Vector{AbstractConstraintResult} preserving step order.
Two shapes constraint generation can return are absorbed rather than rejected, because both are ordinary outcomes of a step rather than errors:
nothing, which is whatlinear_constraintsreturns when every row was dropped — a non-strictrun whose names were all unknown, or a re-basis the loadings annihilated. The step contributed no constraint, so the slot is left untouched; the slot's job is to carry constraints, not to re-diagnose a condition generation already decided was recoverable.- a vector, which is what a step wrapping a vector of estimators returns. Its elements are appended individually, so they reach
constraint_targetsas siblings of every other step's result rather than as a nested vector it has no case for.
Arguments
ctx: The pipeline context.target: The routing target the value must land in. The value is paired with it as aTargetedConstraintonly when its own type does not already name that target (seeimplicit_constraint_target).res: The computed value to append,nothing, or a vector of either.
Returns
ctx′::PipelineContext: The updated context.
Related
The frame of a step
A step that reads the returns reads them from the source that its own reader states in its x_src field. A PipelineStep holds no source. The steps whose values the last optimiser receives join the group of that optimiser, and the readers that those values replace leave it. The Pipeline constructor warns on that group, and it refuses a prior step that fits on the excess returns and feeds an optimiser with a data benchmark.
PortfolioOptimisers.reader_frames — Method
reader_frames(ps::PipelineStep, path::AbstractString, refit::Bool) -> ReturnsFramesReturn the frames that the estimator of a PipelineStep adds to the group of a Pipeline.
The wrapper holds no source. The estimator that it wraps states its own, so the walk enters the field est.
Arguments
ps: The step wrapper.path: The path of the value from the head, for example"pe","cle"or"r[2].pe". A warning names a reader by its path.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.
Returns
fr::ReturnsFrames: The frames ofps.est, at the path"<path>.est".
Related
PortfolioOptimisers.step_prior — Function
step_prior(step) -> Option{<:AbstractPriorEstimator}Return the prior estimator that a step of a Pipeline fits into the prior slot, or nothing.
The answer sees through a PipelineStep and an Online wrapper. A nested Pipeline answers for its last step. A callable and a precomputed Prior Result state no source, so they answer nothing.
Internal machinery — not part of the user-facing API.
Arguments
step: A step of aPipeline.
Returns
pe::Option{<:AbstractPriorEstimator}: The prior estimator of the step, ornothing.
Related
PortfolioOptimisers.pipe_step_targets — Function
pipe_step_targets(step) -> TupleReturn the routing targets that the value of a step of a Pipeline fills in the last optimiser.
inject_context puts the prior slot into pe, and a clustering result in the phylogeny slot into cle. Every other target comes from pipe_required_targets. A network step writes a phylogeny matrix, which reaches no target.
Internal machinery — not part of the user-facing API.
Algorithm
- A step that writes
:priorfills:pe. - A step that writes
:phylogenyfills:clewhen it is a clustering estimator or a clustering result, and nothing otherwise. - Every other step fills the targets that
pipe_required_targetsgives.
Arguments
step: A step of aPipeline.
Returns
targets::Tuple: The routing targets of the value of the step.
Related
PortfolioOptimisers.pipe_target_paths — Function
pipe_target_paths(x, ::Val{target}) -> Vector{String}Return the paths, from the optimiser x, of the slots that a value at a routing target replaces.
The walk follows pipe_route. A target goes into the field of the same name, or into the configuration that @pipe_delegates names. :mu_ucs goes into ret.ucs of a JuMPOptimiser, and :sigma_ucs goes into the ucs of each UncertaintySetVariance in the field r. The frame walk of a Pipeline leaves these slots out of the group of the optimiser, because the value of the step replaces the reader that they hold.
Internal machinery — not part of the user-facing API.
Algorithm
:sigma_ucs: the path"r.ucs", or"r[i].ucs"for each elementiof a vector that is anUncertaintySetVariance. Only an optimiser that@pipe_route_sigma_ucsdeclares accepts this target, and each one has a fieldr.- A target with a field of the same name in
x: the name of the field. - Otherwise the paths in the configuration of
x, after its field name, seepipe_config_paths.
Arguments
x: The optimiser or the optimiser configuration.::Val{target}: The routing target.
Returns
paths::Vector{String}: The paths of the replaced slots. It is empty whenxhas no slot for the target.
Related
PortfolioOptimisers.pipe_config_paths — Function
pipe_config_paths(x, t::Val) -> Vector{String}Return the paths of pipe_target_paths in the configuration of x, after the name of the field that holds it, or an empty vector when x delegates to no configuration.
A target that the ConstraintBlock of x accepts goes into the block, as pipe_route puts it there. So its path starts with the field that pipe_block_field names, such as "cbe.ple", which is the path that the frame walk gives to the slot.
Internal machinery — not part of the user-facing API.
Arguments
x: The optimiser.t: The routing target.
Returns
paths::Vector{String}: The paths fromx.
Related
PortfolioOptimisers.pipe_prior_estimator — Function
pipe_prior_estimator(x)Return the prior that the optimiser x holds in the field that receives the :pe routing target, or nothing when it has none.
The walk follows pipe_route: the field pe of x, or of the configuration that @pipe_delegates names.
Internal machinery — not part of the user-facing API.
Arguments
x: The optimiser or the optimiser configuration.
Returns
pe: The prior estimator, the Prior Result or the schedule in the field, ornothing.
Related
PortfolioOptimisers.cut_frames — Function
cut_frames(x, path::AbstractString, refit::Bool, cut::AbstractVector{<:AbstractString}) -> ReturnsFramesReturn the frames that x adds to a group, without the slots at the paths in cut.
The walk of a Pipeline calls this function on its last optimiser. A step value replaces each slot in cut, so the reader that the slot holds never runs. The function enters only the values on the way to a cut slot, field by field and element by element, and gives every other value to reader_frames. The optimiser itself is at the empty path, and it always enters, because a head has no method of reader_frames.
Internal machinery — not part of the user-facing API.
Algorithm
- A value at a path in
cutadds nothing. - A value at a path that no cut path starts below adds
reader_frames(x, path, refit). - Otherwise join the frames of each element of a vector, or of each field, with
cut_children.
Arguments
x: The value to walk: a reader of the returns, a value that holds readers, a vector of them, or any other value.path: The path of the value from the head, for example"pe","cle"or"r[2].pe". A warning names a reader by its path.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.cut: The paths of the replaced slots, frompipe_target_paths.
Returns
fr::ReturnsFrames: The frames ofxwithout the cut slots.
Related
PortfolioOptimisers.cut_children — Function
cut_children(x, path::AbstractString, refit::Bool, cut::AbstractVector{<:AbstractString})
cut_children(x::AbstractVector, path::AbstractString, refit::Bool,
cut::AbstractVector{<:AbstractString})Return the frames of each field of x, or of each element of a vector, through cut_frames.
cut_frames calls this function on a value on the way to a cut slot. A field is at the path that join_path gives, and an element is at "<path>[i]", as reader_frames names it.
Internal machinery — not part of the user-facing API.
Arguments
x: The value to walk: a reader of the returns, a value that holds readers, a vector of them, or any other value.path: The path of the value from the head, for example"pe","cle"or"r[2].pe". A warning names a reader by its path.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.cut: The paths of the replaced slots.
Returns
fs::Vector{ReturnsFrames}: The frames of each field or element, in order.
Related
PortfolioOptimisers.pipe_terminal_optimiser — Function
pipe_terminal_optimiser(
ests
) -> Union{Nothing, OptimisationEstimator}
Return the optimiser of the last step of a Pipeline, or nothing when the pipeline has none that a construction check can read.
The checks that compare the earlier steps with the optimiser read it here: assert_routable, assert_pipeline_data_benchmark and pipeline_frames.
Algorithm
- Read the optimiser from the last step, without its
PipelineStepwrapper. - Return
nothingwhen the pipeline has one step, or when the optimiser is not anOptimisationEstimator. ATimeDependentschedule and a precomputed result are such steps, and the fold loop resolves them later. The type is checked first, becausefitalso reads the optimiser for the gap drop of the group, seepipeline_gap_sources, and in a fold of a mixed schedule the last step is a precomputed result, whose written slotpipe_writescannot infer. - Return the optimiser when its step writes
:opt, andnothingotherwise.
Arguments
ests: The step estimators, with the optimisation step last, seeassert_opt_last.
Returns
opt::Option{<:OptimisationEstimator}: The optimiser, ornothing.
Related
PortfolioOptimisers.pipeline_frames — Function
pipeline_frames(ests) -> ReturnsFramesReturn the frames of the group that the steps of a Pipeline join: the group of its last optimiser.
inject_context puts the value of each step into a slot of the last optimiser, so the step reads the returns for that optimiser. The steps whose values the optimiser receives join its group, and the readers that those values replace leave it. A network step writes a phylogeny matrix that no slot receives, so it stays out. A prior step decides refit for the optimiser when the optimiser receives its value, because that prior then replaces the prior of the optimiser. A step reads the prior slot, so the last prior step decides refit for the steps. With no prior step a step reads a Prior Result that the input gave, which reads as raw. group_frames of a Pipeline calls this function.
Internal machinery — not part of the user-facing API.
Algorithm
- Read the optimiser
optof the last step withpipe_terminal_optimiser. Answer that the group holds no reader when there is none. - For each earlier step, keep the targets of
pipe_step_targetsthatoptaccepts (pipe_accepts). The value of a step with no such target does not reachopt. - Find
refitfor the steps and foroptwithpipeline_refits. - Walk each step that
optreceives withreader_frames, at the path"steps[i]". - Walk
optwithout the slots that the steps replace withterminal_frames, and join every part withmerge_frames.
Arguments
ests: The step estimators, with the optimisation step last, seeassert_opt_last.
Returns
fr::ReturnsFrames: The frames of the group of the last optimiser, with the steps that it receives.
Related
PortfolioOptimisers.pipeline_received — Function
pipeline_received(ests, opt) -> Vector{Vector{Symbol}}Return, for each step of a Pipeline before the last, the routing targets of the step that the last optimiser opt accepts.
A step with no such target does not reach opt, so it stays out of the group of opt.
Internal machinery — not part of the user-facing API.
Arguments
ests: The step estimators, with the optimisation step last, seeassert_opt_last.opt: The optimiser of the last step.
Returns
received::Vector{Vector{Symbol}}: The targets thatoptreceives from each earlier step.
Related
PortfolioOptimisers.pipeline_gap_sources — Function
pipeline_gap_sources(ests) -> Vector{Option{ExcessSource}}Return, for each step of a Pipeline, the source under which the Pipeline drops a gap of a vector benchmark before the step runs.
The steps whose values the last optimiser receives join its group, see pipeline_frames. A head drops a gap of a vector benchmark when any reader of its group states ExcessSource, see group_source, so every reader of the head pairs the same observations. The Pipeline applies the same rule: each step of the group, and the optimiser, reads the returns data with the gaps of the group dropped. A raw reader of the group loses the observations too. The rule decides which observations a step reads, and not its frame: each reader still reads the frame that its own x_src states. A step outside the group gets nothing, so it drops by the rule of its own reader alone.
Internal machinery — not part of the user-facing API.
Algorithm
- Read the optimiser
optof the last step withpipe_terminal_optimiser. When there is none, answernothingfor every step, except for a Pipeline of one optimiser step: that step is a head alone, so it getsgroup_sourceof its optimiser. - Find the source of the group with
group_sourceofpipeline_frames, givinggsrc. - Answer
gsrcfor each step thatoptreceives (pipeline_received) and for the last step, andnothingfor every other step.
Arguments
ests: The step estimators, with the optimisation step last, seeassert_opt_last.
Returns
gsrcs::Vector{Option{ExcessSource}}: One source for each step.
Related
PortfolioOptimisers.run_group_step — Function
run_group_step(step, ctx::PipelineContext, gsrc::Nothing) -> (fitted, ctx′)
run_group_step(step, ctx::PipelineContext, gsrc::ExcessSource) -> (fitted, ctx′)Run one step of a Pipeline with the gaps of the group of its optimiser dropped from the returns data that the step reads.
The step reads the returns slot with each observation at which a vector benchmark has a gap dropped, through shared_series_view under gsrc. The returns data is then put back in the slot, so the next step, and the context of the PipelineResult, hold the returns data that the caller gave. A step of the group never writes the returns slot, because the optimiser receives no Data Slot. The reader of the step then drops a gap of the factor returns, and a gap of the benchmark under its own source, as it does in a head. Both drops are idempotent on a view with no gap left.
Internal machinery — not part of the user-facing API.
Algorithm
The method that Julia selects is the algorithm.
gsrcisnothing: run the step withrun_step.gsrcisExcessSource: view thereturnsslot withgroup_returns_view, run the step on that context, and put the returns data back.
Arguments
step: The step, aftermaybe_inject_step.ctx: The pipeline context.gsrc: The source of the gap drop of the step, seepipeline_gap_sources.
Returns
(fitted, ctx′): The fitted object and the updated context.
Related
PortfolioOptimisers.group_returns_view — Function
group_returns_view(step, rd::ReturnsResult, gsrc::ExcessSource) -> Tuple{Any, ReturnsResult}
group_returns_view(step, rd, gsrc::ExcessSource)Return the step and the returns data that a step of the group of a Pipeline reads: rd with each gap of a vector benchmark dropped, and step with each fixed observation weight of the same observations dropped, see shared_series_fit.
A slot that holds no ReturnsResult, such as an empty slot before a step that reads prices, is returned unchanged with the step.
Internal machinery — not part of the user-facing API.
Related
PortfolioOptimisers.pipeline_refits — Function
pipeline_refits(ests, received, opt) -> Tuple{Bool, Bool}Return refit for the steps of a Pipeline and refit for its last optimiser, see refits_regression.
A step reads the prior slot, so the last prior step decides refit for the steps. With no prior step a step reads a Prior Result that the input gave, which reads as raw. A prior step decides refit for the optimiser when the optimiser receives its value, because that prior then replaces the prior of the optimiser.
Internal machinery — not part of the user-facing API.
Algorithm
- Find the last prior step with
step_prior.refitfor the steps isrefits_regressionof its prior, orfalsewhen there is none. - Find the last step whose targets in
receivedhold:pe.refitforoptisrefits_regressionof its prior, or of the prior ofopt(pipe_prior_estimator) when there is none.
Arguments
ests: The step estimators, with the optimisation step last.received: The routing targets thatoptreceives from each earlier step.opt: The optimiser of the last step.
Returns
(step_refit, opt_refit)::Tuple{Bool, Bool}:refitfor the steps and foropt.
Related
PortfolioOptimisers.terminal_frames — Function
terminal_frames(step, n::Integer, opt, received, refit::Bool) -> ReturnsFramesReturn the frames of the last optimiser of a Pipeline, without the slots that the earlier steps replace, at the path of the last step.
Internal machinery — not part of the user-facing API.
Algorithm
- Find the paths of the slots that the earlier steps replace with
pipe_target_paths. - When there are none, take
group_framesofopt. Otherwise walkoptwithout those slots withcut_frames. - Put the path of the last step,
"steps[n]"or"steps[n].est"for aPipelineStep, before each raw path.
Arguments
step: The last step,optor its wrapper.n: The position of the last step.opt: The optimiser of the last step.received: The routing targets thatoptreceives from each earlier step.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.
Returns
fr::ReturnsFrames: The frames ofopt, with paths from the pipeline.
Related
PortfolioOptimisers.pipe_data_benchmark_path — Function
pipe_data_benchmark_path(x) -> Option{<:String}Return the path of the first DataBenchmarkTracking in a measure that reads the prior of the optimiser x, or nothing.
Every tracking form compares pr.X * w with its benchmark series, so the prior that the measure reads decides its frame. The measures that read the prior of a head are the tracking errors tr of its configuration and the risk measures r, ri and ro of the head. The walk follows the path of the :pe routing target through pipe_config_field, so it reads the fields that hold the prior and the fields beside them. It does not enter an inner optimiser or a fallback, because each one checks its own prior.
Internal machinery — not part of the user-facing API.
Algorithm
- For each of the fields
tr,r,riandroofx, in order, return the path thatdata_benchmark_pathfinds below it, after the name of the field. - Otherwise return the answer for the
ConstraintBlockofx, after the name of its field, seepipe_block_field. The block holds the tracking errorstrof aJuMPOptimiser. - An
xwith no block returns the answer for its configuration, after the name of its field. Anxwith neither answers fornothing, which isnothing.
Arguments
x: The optimiser or the optimiser configuration.
Returns
path::Option{<:String}: The path of the data benchmark fromx, ornothing.
Related
PortfolioOptimisers.assert_pipeline_data_benchmark — Function
assert_pipeline_data_benchmark(ests) -> NothingRefuse at construction a Pipeline whose prior step states x_src = ExcessSource() when the optimiser that receives the prior tracks a DataBenchmarkTracking.
The optimiser receives the Prior Result of the step in its field pe. In the Excess Frame pr.X holds X - B, so a data benchmark gives the series (X - B) * w - B, and the benchmark is subtracted two times. The constructor of the optimiser refuses this pair for its own prior. It cannot see the source of the step, because the step value replaces its prior only when the pipeline runs. So the Pipeline constructor refuses it. The configuration alone decides the refusal. A prior step that holds a precomputed Prior Result, or a callable, states no source, so it reads as raw and passes.
Internal machinery — not part of the user-facing API.
Algorithm
- Read the optimiser
optof the last step withpipe_terminal_optimiser. Stop when there is none, or when it has no field for the:perouting target. - Find the last prior step, whose Prior Result the
priorslot holds whenoptruns, withstep_prior. Stop when there is none, or when its source is notExcessSource(). - Find a data benchmark in a measure of
optthat reads its prior withpipe_data_benchmark_path.
Arguments
ests: The step estimators, with the optimisation step last, seeassert_opt_last.
Validation
- The last prior step does not state
x_src = ExcessSource(), or the optimiser holds noDataBenchmarkTrackingin a measure that reads its prior. AnArgumentErrorthat names the step, the field and the two remedies is thrown otherwise.
Returns
nothing.
Related
A TrainTestSplit is the one step whose written slot does not follow from its type. It narrows whichever data slot the pipeline input filled, the prices or the returns. pipe_writes returns :split for it, which is not a slot, so the pipeline treats the step as one that writes nothing. The docstrings of run_step, pipe_reads and pipe_writes cover the run_step method of the split and its slot declarations.