The source files can be found in examples/.
Pipelines
The earlier examples start from a returns matrix. Several choices come before that matrix: which assets have enough data to keep, how to fill the gaps in their prices, and whether the returns are simple or logarithmic. Each choice has parameters, and so far the examples made them once, on the full sample, before cross-validation started.
A choice made on the full sample uses the prices of the test windows. If a gap takes a fill value computed from the whole price history, the prices of every test fold enter that value. If you choose the assets by their missing data over all four years, the backtest keeps only assets that you already know have a full history.
A Pipeline fits the whole workflow as one estimator. The workflow can clean the prices, convert them to returns, estimate the prior and the phylogeny, make the constraints and optimise. A pipeline fitted on a training window learns its preprocessing state from that window, and applies that state unchanged to a test window. Cross-validation of a pipeline splits the input rows, and every fold fits the cleaning steps again.
We build a pipeline one step at a time and show what each step changes. Then we tune the preprocessing parameters under walk-forward cross-validation.
using PortfolioOptimisers, PrettyTablestsfmt = (v, i, j) -> begin if j == 1 return Date(v) else return v endend;resfmt = (v, i, j) -> begin if j == 1 return v else return isa(v, AbstractFloat) ? "$(round(v*100, digits=3)) %" : v endend;1. Setting up
We use four years of daily data, so that walk-forward cross-validation has enough rows for several folds. We start from prices, not returns, because the pipeline does the conversion.
using CSV, TimeSeries, DataFrames, Clarabel, Statistics, StableRNGsX = TimeArray(CSV.File(joinpath(@__DIR__, "..", "SP500.csv.gz")); timestamp = :Date)[(end - 252 * 4):end]X = X[Symbol.(["AAPL", "MSFT", "JNJ", "JPM", "XOM", "PG", "KO", "WMT", "PEP", "MRK"])...]pretty_table(X[(end - 5):end]; formatters = [tsfmt])slv = Solver(; name = :clarabel, solver = Clarabel.Optimizer, settings = Dict("verbose" => false), check_sol = (; allow_local = true, allow_almost = true));┌────────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ timestamp │ AAPL │ MSFT │ JNJ │ JPM │ XOM │ PG │ KO │ WMT │ PEP │ MRK │
│ Date │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │
├────────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ 2022-12-20 │ 131.916 │ 240.67 │ 173.109 │ 127.844 │ 104.964 │ 147.661 │ 61.841 │ 142.919 │ 178.765 │ 108.229 │
│ 2022-12-21 │ 135.057 │ 243.287 │ 175.09 │ 129.282 │ 106.312 │ 149.015 │ 62.836 │ 144.04 │ 180.017 │ 109.611 │
│ 2022-12-22 │ 131.846 │ 237.077 │ 174.45 │ 127.814 │ 104.168 │ 149.359 │ 62.383 │ 142.354 │ 178.627 │ 109.739 │
│ 2022-12-23 │ 131.477 │ 237.614 │ 174.893 │ 128.421 │ 106.922 │ 149.781 │ 62.855 │ 142.641 │ 179.781 │ 110.35 │
│ 2022-12-27 │ 129.652 │ 235.852 │ 174.844 │ 128.871 │ 108.408 │ 151.086 │ 63.24 │ 142.681 │ 180.58 │ 110.607 │
│ 2022-12-28 │ 125.674 │ 233.434 │ 174.085 │ 129.575 │ 106.627 │ 149.133 │ 62.609 │ 140.181 │ 179.278 │ 109.581 │
└────────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘1.1 PricesResult, the price-level container
PricesResult is the price-level counterpart of ReturnsResult. It stores the asset prices and, when you have them, factor, benchmark and implied-volatility series. A pipeline that starts at the price level takes one as its input.
So that the cleaning steps change something, we delete some prices. JNJ loses the first half of its history, and AAPL and XOM lose one price each.
listing_span reads the listing of each asset off its prices. A run of gaps at the start means that the asset is not listed yet, a run at the end means that it is delisted, and a gap in between is a gap inside the listing. price_ingestion makes this span for you, and we call listing_span only because we built the prices by hand. The cell ends with the fraction of missing prices of each asset.
vals = Matrix{Float64}(values(X))vals[1:(end ÷ 2), 3] .= NaNvals[10, 1] = NaNvals[25, 5] = NaNXm = TimeArray(timestamp(X), vals, colnames(X))span = listing_span(vals)pr = PricesResult(; X = Xm, span = span)miss = DataFrame(; asset = string.(colnames(Xm)), missing_frac = vec(count(isnan, vals; dims = 1)) ./ size(vals, 1))pretty_table(miss; formatters = [resfmt])┌────────┬──────────────┐
│ asset │ missing_frac │
│ String │ Float64 │
├────────┼──────────────┤
│ AAPL │ 0.099 % │
│ MSFT │ 0.0 % │
│ JNJ │ 49.95 % │
│ JPM │ 0.0 % │
│ XOM │ 0.099 % │
│ PG │ 0.0 % │
│ KO │ 0.0 % │
│ WMT │ 0.0 % │
│ PEP │ 0.0 % │
│ MRK │ 0.0 % │
└────────┴──────────────┘2. Building a pipeline one step at a time
A pipeline is an ordered list of steps, and a step is an ordinary estimator. The pipeline keeps the data that its steps pass on in a context, a set of named slots such as :prices, :returns, :prior or :opt. You read the context of a fitted pipeline from res.ctx. Every step writes one slot. The kind of estimator decides the slot, so a step needs no wrapper. You can name a step with "name" => estimator. A step with no name takes the name of its slot, with a suffix _1, _2 when two steps write the same slot.
2.1 A pipeline from prices to returns
PricesToReturns is the step form of prices_to_returns. It learns nothing when you fit it, so it runs the same conversion on any window.
pipe = Pipeline(; steps = (PricesToReturns(), EmpiricalPrior(), EqualWeighted()))pipe.names("returns", "prior", "opt")Fitting runs the steps from left to right. The result stores the fitted output of every step, the context after the last step, and the portfolio weights in res.w.
res = fit(pipe, pr)pretty_table(DataFrame(; asset = res.ctx.returns.nx, weight = res.w); formatters = [resfmt])[ Info: 3 asset(s) left the investable universe and are excluded from this optimisation: ["AAPL", "JNJ", "XOM"]. A constraint, bound or rate stated for one of them is dropped, and a liquidation charge is priced over them.
┌────────┬──────────┐
│ asset │ weight │
│ String │ Float64 │
├────────┼──────────┤
│ AAPL │ 0.0 % │
│ MSFT │ 14.286 % │
│ JNJ │ 0.0 % │
│ JPM │ 14.286 % │
│ XOM │ 0.0 % │
│ PG │ 14.286 % │
│ KO │ 14.286 % │
│ WMT │ 14.286 % │
│ PEP │ 14.286 % │
│ MRK │ 14.286 % │
└────────┴──────────┘AAPL, JNJ and XOM get no weight. A gap in the prices reaches the returns as a missing value, and the optimiser leaves out an asset whose returns have one.
We compare the number of price rows with the number of return rows. Cross-validation of a pipeline sizes its windows in input rows, which are prices here, and not in returns.
size(values(pr.X), 1), size(res.ctx.returns.X, 1)(1009, 1008)2.2 The choice of assets is fitted
MissingDataFilter drops every asset whose fraction of missing prices is above col_thr. The training window decides which assets stay, and the fitted result stores their names. Applied to another window, the fitted result selects the same assets there. The training weights and the test returns then cover the same assets.
JNJ misses half of its history, so col_thr = 0.4 drops it.
pipe = Pipeline(; steps = ("filter" => MissingDataFilter(; col_thr = 0.4), PricesToReturns(), EmpiricalPrior(), EqualWeighted()))res = fit(pipe, pr)res["filter"].nx9-element Vector{Symbol}:
:AAPL
:MSFT
:JPM
:XOM
:PG
:KO
:WMT
:PEP
:MRKWith a higher threshold, JNJ stays. The threshold is a parameter, and section 4 tunes it instead of fixing it by hand.
PricesToReturns learns nothing. It computes returns and deletes no observation and no asset. A gap reaches the returns as a missing value, and its row stays. The conversion therefore never gives a training window and a test window different assets.
The choice of assets is fitted, and a MissingDataFilter step makes it. Put that step before the conversion. Use it when you want an asset removed. Use a PriceGapFill step when you want a gap inside a listing to take a price.
res_lax = fit(Pipeline(; steps = ("filter" => MissingDataFilter(; col_thr = 0.9), PricesToReturns(), EmpiricalPrior(), EqualWeighted())), pr)res_lax["filter"].nx10-element Vector{Symbol}:
:AAPL
:MSFT
:JNJ
:JPM
:XOM
:PG
:KO
:WMT
:PEP
:MRK2.3 Fill values are fitted
PriceGapFill gives a price to a gap inside the listing of an asset. It fills nothing outside the listing. JNJ's missing first half comes before its first price. The step leaves it empty and does not invent a history. The single gaps of AAPL and XOM are inside their listings, and the step fills them.
The rule for the fill is the parameter fill. The default, CarriedPrice, gives a gap the last price before it, so a holding keeps its value over the gap. When you fit the step, it records the last price of every asset in the field v. We fit the same step on two windows and compare the two values for AAPL.
pipe_imp = Pipeline(; steps = ("gap_fill" => PriceGapFill(), PricesToReturns(), EmpiricalPrior()))res_train = fit(pipe_imp, PricesResult(; X = Xm[1:500], span = span[1:500, :]))res_test = fit(pipe_imp, PricesResult(; X = Xm[501:end], span = span[501:end, :]))j = findfirst(==(:AAPL), res_train["gap_fill"].nx)k = findfirst(==(:AAPL), res_test["gap_fill"].nx)fills = DataFrame(; window = ["train (1:500)", "test (501:end)"], aapl_fill = [res_train["gap_fill"].v[j], res_test["gap_fill"].v[k]])pretty_table(fills)┌────────────────┬───────────┐
│ window │ aapl_fill │
│ String │ Float64 │
├────────────────┼───────────┤
│ train (1:500) │ 124.794 │
│ test (501:end) │ 125.674 │
└────────────────┴───────────┘Every window learns its own last price, so the two values differ. A pipeline fitted on the training window stores the training value. When it predicts on a later window, it uses that value only for a gap at the start of the window, before the window has a price of its own. A later gap takes the last price of the window itself.
PriceGapFill(; fill = MeanValue()) and PriceGapFill(; fill = MedianValue()) give every gap of an asset one constant, which the step computes from the training prices. A pipeline applies the training constant to the test window. If you fit such a fill on the full sample before the split, the constant includes the test prices, and every test score then uses information from its own test window. Section 4 tunes the choice between the two.
3. The full workflow
We now build the full workflow. It filters the assets, fills the gaps, converts the prices to returns, estimates the prior, makes the phylogeny constraints and the weight bounds, and ends with a MeanRisk optimisation.
The pipeline connects the steps in two ways.
- Every step writes the slot that its kind of estimator owns. A prior estimator writes
:prior. A phylogeny constraint estimator, such asSemiDefinitePhylogenyEstimator, and a weight-bounds estimator write:constraints. A phylogeny estimator writes:phylogeny. - Before the optimisation step runs, the pipeline puts the computed slots into the optimiser. The prior replaces the optimiser's
pe. A constraint result goes to the field of the optimiser that its kind names, such aswb,lcse,cte,ple,rkbor a threshold field.
A buy-in threshold can go to six different fields, so a ThresholdEstimator step must name one. Wrap it in a PipelineStep with a target, for example target = :lt for the long threshold or target = :st for the short threshold. If the optimiser has no field for a constraint step, the Pipeline constructor throws an error. Every step is optional. Without a prior step, the optimiser estimates its own prior, as it does outside a pipeline.
pipe = Pipeline(; steps = ("filter" => MissingDataFilter(; col_thr = 0.4), "gap_fill" => PriceGapFill(), PricesToReturns(), EmpiricalPrior(), SemiDefinitePhylogenyEstimator(), "wb" => WeightBoundsEstimator(; lb = nothing, ub = 0.4), MeanRisk(; opt = JuMPOptimiser(; slv = slv))))pipe.namesres = fit(pipe, pr)pretty_table(DataFrame(; asset = res.ctx.returns.nx, weight = res.w); formatters = [resfmt])┌────────┬──────────┐
│ asset │ weight │
│ String │ Float64 │
├────────┼──────────┤
│ AAPL │ 0.0 % │
│ MSFT │ 0.0 % │
│ JPM │ 0.0 % │
│ XOM │ 6.29 % │
│ PG │ 15.706 % │
│ KO │ 19.947 % │
│ WMT │ 31.434 % │
│ PEP │ 0.0 % │
│ MRK │ 26.623 % │
└────────┴──────────┘We compare the largest weight with the upper bound of 0.4.
maximum(res.w) <= 0.4 + 1e-8true3.1 Predicting on an unseen window
predict applies the fitted preprocessing to a test window in step order. It selects the training assets, fills the gaps with the rule it fitted, and converts the prices to returns. Then it runs the ordinary prediction from the weights. Every scorer and risk measure then works on the result as it does without a pipeline. The third argument gives the rows of the input to predict on, here the prices after row 800.
T = size(values(Xm), 1)res_train = fit(pipe, PricesResult(; X = Xm[1:800], span = span[1:800, :]))pred = predict(res_train, pr, 801:T)expected_risk(ConditionalValueatRisk(), pred)0.025999979503071144. Tuning the whole workflow
search_cross_validation splits the input rows into windows of consecutive rows. For every candidate, it fits the whole pipeline on the training window and scores it on the test window. One grid can hold parameters of the preprocessing and of the optimiser, and the preprocessing of a candidate never sees its test window.
A key of the grid names a step in one of three ways.
- A step name with a property path,
"filter.col_thr", changes one field of that step. - A step name alone,
"gap_fill", or the position of the step as an integer key, replaces the whole step. - A property path from the pipeline,
"steps[1].col_thr", works as it does for an optimiser outside a pipeline. The path starts atsteps.
The table after the search gives the mean test score of each candidate. The search negates a risk measure, so the highest score belongs to the lowest mean CVaR.
pipe = Pipeline(; steps = ("filter" => MissingDataFilter(; col_thr = 0.4), "gap_fill" => PriceGapFill(), PricesToReturns(), EmpiricalPrior(), "opt" => MeanRisk(; opt = JuMPOptimiser(; slv = slv))))p = ["filter.col_thr" => [0.0, 0.4], "gap_fill" => [PriceGapFill(; fill = MeanValue()), PriceGapFill(; fill = MedianValue())]]gscv = GridSearchCrossValidation(p; cv = IndexWalkForward(500, 250), r = ConditionalValueatRisk())tuned = search_cross_validation(pipe, gscv, pr)scores = DataFrame(; candidate = 1:length(tuned.val_grid), col_thr = [v[1] for v in tuned.val_grid], gap_fill = [string(nameof(typeof(v[2].fill))) for v in tuned.val_grid], mean_score = vec(mean(tuned.test_scores; dims = 1)))pretty_table(scores)┌───────────┬─────────┬─────────────┬────────────┐
│ candidate │ col_thr │ gap_fill │ mean_score │
│ Int64 │ Float64 │ String │ Float64 │
├───────────┼─────────┼─────────────┼────────────┤
│ 1 │ 0.0 │ MeanValue │ -0.0218637 │
│ 2 │ 0.4 │ MeanValue │ -0.0218836 │
│ 3 │ 0.0 │ MedianValue │ -0.0218637 │
│ 4 │ 0.4 │ MedianValue │ -0.021892 │
└───────────┴─────────┴─────────────┴────────────┘A threshold of zero drops every asset that has a gap in the training window. The first training window, rows 1 to 500, holds the gaps of AAPL and XOM. There the threshold of zero drops them, and the threshold of 0.4 keeps them and fills their gaps. JNJ fails both thresholds in both training windows.
At the threshold of zero no gap is left to fill, so the two fills give the same score. That threshold scores highest. The search keeps the first candidate with the highest mean score, and tuned.opt is that pipeline. We fit it on the full sample, where it drops AAPL, JNJ and XOM.
tuned.idx, tuned.opt.steps[1].col_thr, nameof(typeof(tuned.opt.steps[2].fill))final = fit(tuned.opt, pr)pretty_table(DataFrame(; asset = final.ctx.returns.nx, weight = final.w); formatters = [resfmt])┌────────┬──────────┐
│ asset │ weight │
│ String │ Float64 │
├────────┼──────────┤
│ MSFT │ 0.0 % │
│ JPM │ 0.078 % │
│ PG │ 14.503 % │
│ KO │ 24.516 % │
│ WMT │ 32.462 % │
│ PEP │ 0.0 % │
│ MRK │ 28.442 % │
└────────┴──────────┘4.1 Replacing a whole estimator
A grid value can be any object, so a grid can replace a whole step. Here the grid replaces the prior estimator. The last step is a MeanRisk optimisation, which minimises the variance of the prior, so the two priors give different weights.
pipe_struct = Pipeline(; steps = (MissingDataFilter(; col_thr = 0.4), PriceGapFill(), PricesToReturns(), "prior" => EmpiricalPrior(), MeanRisk(; opt = JuMPOptimiser(; slv = slv))))p_struct = ["prior" => [EmpiricalPrior(), EmpiricalPrior(; ce = PortfolioOptimisersCovariance(; ce = GerberCovariance()))]]tuned_struct = search_cross_validation(pipe_struct, GridSearchCrossValidation(p_struct; cv = IndexWalkForward(500, 250), r = ConditionalValueatRisk()), pr)tuned_struct.idx, vec(mean(tuned_struct.test_scores; dims = 1))(2, [-0.021899206075784994, -0.019645751472496593])The prior with the Gerber covariance scores higher, so the search keeps the second candidate.
4.2 Randomised search
RandomisedSearchCrossValidation samples candidates from the grid and then runs the grid search on them, as it does outside a pipeline. When you give seed, the search seeds a copy of rng with it.
rscv = RandomisedSearchCrossValidation(p; cv = IndexWalkForward(500, 250), r = ConditionalValueatRisk(), rng = StableRNG(42), n_iter = 2, seed = 1)tuned_rand = search_cross_validation(pipe, rscv, pr)size(tuned_rand.test_scores)(2, 4)5. Cross-validation with many paths
The walk-forward search above gives one backtest path. CombinatorialCrossValidation and MultipleRandomised give many paths, and with them a distribution of test results instead of one number.
A pipeline that starts from prices runs both schemes. The combinatorial scheme joins groups of rows that are not consecutive, and at the price level every join makes one return between two prices that are not neighbours. We start from returns here and drop the price-level cleaning, so every return comes from two consecutive prices.
rd = prices_to_returns(X)rpipe = Pipeline(; steps = (EmpiricalPrior(), MeanRisk(; obj = MinimumRisk(), opt = JuMPOptimiser(; slv = slv))))Pipeline
names ┼ Tuple{String, String}: ("prior", "opt")
steps ┴ Tuple{EmpiricalPrior{PortfolioOptimisersCovariance{Covariance{SimpleExpectedReturns{Nothing, Nothing, Nothing}, GeneralCovariance{SimpleCovariance, Nothing, Nothing}, FullMoment, Nothing, Nothing, Nothing}, MatrixProcessing{Posdef{UnionAll, @NamedTuple{}}, Nothing, Nothing, Nothing, NTuple{4, Symbol}}, Nothing}, SimpleExpectedReturns{Nothing, Nothing, Nothing}, Nothing, Nothing, Nothing, Nothing, Nothing}, MeanRisk{JuMPOptimiser{EmpiricalPrior{PortfolioOptimisersCovariance{Covariance{SimpleExpectedReturns{Nothing, Nothing, Nothing}, GeneralCovariance{SimpleCovariance, Nothing, Nothing}, FullMoment, Nothing, Nothing, Nothing}, MatrixProcessing{Posdef{UnionAll, @NamedTuple{}}, Nothing, Nothing, Nothing, NTuple{4, Symbol}}, Nothing}, SimpleExpectedReturns{Nothing, Nothing, Nothing}, Nothing, Nothing, Nothing, Nothing, Nothing}, Solver{Symbol, UnionAll, Dict{String, Bool}, @NamedTuple{allow_local::Bool, allow_almost::Bool}, Bool}, WeightBounds{Float64, Float64}, Float64, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ArithmeticReturn{JuMPReturnsSettings{Float64, Nothing, Bool, Bool, Bool}, Nothing, Nothing, EigenFallbackSquareRoot}, SumScalariser, Nothing, Nothing, Int64, Int64, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Bool, Bool, Nothing}, Variance{RiskMeasureSettings{Float64, Nothing, Bool}, Nothing, Nothing, Nothing, SquaredSOCRiskExpr, EigenFallbackSquareRoot, SquareRootRatio, Nothing}, MinimumRisk, Nothing, Nothing}}: (EmpiricalPrior
ce ┼ PortfolioOptimisersCovariance
│ ce ┼ Covariance
│ │ me ┼ SimpleExpectedReturns
│ │ │ w ┴ nothing
│ │ ce ┼ GeneralCovariance
│ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ w ┴ nothing
│ │ alg ┼ FullMoment()
│ │ w ┴ nothing
│ mp ┼ MatrixProcessing
│ │ pdm ┼ Posdef
│ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ dn ┼ nothing
│ │ dt ┼ nothing
│ │ alg ┼ nothing
│ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
me ┼ SimpleExpectedReturns
│ w ┴ nothing
horizon ┼ nothing
fill_limit ┼ nothing
x_src ┴ nothing
, MeanRisk
opt ┼ JuMPOptimiser
│ pe ┼ EmpiricalPrior
│ │ ce ┼ PortfolioOptimisersCovariance
│ │ │ ce ┼ Covariance
│ │ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ │ w ┴ nothing
│ │ │ │ ce ┼ GeneralCovariance
│ │ │ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ │ │ w ┴ nothing
│ │ │ │ alg ┼ FullMoment()
│ │ │ │ w ┴ nothing
│ │ │ mp ┼ MatrixProcessing
│ │ │ │ pdm ┼ Posdef
│ │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ │ │ dn ┼ nothing
│ │ │ │ dt ┼ nothing
│ │ │ │ alg ┼ nothing
│ │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
│ │ me ┼ SimpleExpectedReturns
│ │ │ w ┴ nothing
│ │ horizon ┼ nothing
│ │ fill_limit ┼ nothing
│ │ x_src ┴ nothing
│ slv ┼ Solver
│ │ name ┼ Symbol: :clarabel
│ │ solver ┼ UnionAll: Clarabel.MOIwrapper.Optimizer
│ │ settings ┼ Dict{String, Bool}: Dict{String, Bool}("verbose" => 0)
│ │ check_sol ┼ @NamedTuple{allow_local::Bool, allow_almost::Bool}: (allow_local = true, allow_almost = true)
│ │ add_bridges ┴ Bool: true
│ wb ┼ WeightBounds
│ │ lb ┼ Float64: 0.0
│ │ ub ┴ Float64: 1.0
│ bgt ┼ Float64: 1.0
│ sbgt ┼ nothing
│ gbgt ┼ nothing
│ lt ┼ nothing
│ st ┼ nothing
│ lcse ┼ nothing
│ cte ┼ nothing
│ gcarde ┼ nothing
│ sgcarde ┼ nothing
│ smtx ┼ nothing
│ sgmtx ┼ nothing
│ slt ┼ nothing
│ sst ┼ nothing
│ sglt ┼ nothing
│ sgst ┼ nothing
│ tn ┼ nothing
│ fees ┼ nothing
│ sets ┼ nothing
│ tr ┼ nothing
│ ple ┼ nothing
│ ret ┼ ArithmeticReturn
│ │ settings ┼ JuMPReturnsSettings
│ │ │ scale ┼ Float64: 1.0
│ │ │ lb ┼ nothing
│ │ │ rte ┼ Bool: true
│ │ │ fee ┼ Bool: true
│ │ │ mic ┴ Bool: true
│ │ ucs ┼ nothing
│ │ mu ┼ nothing
│ │ mtx_sqrt ┴ EigenFallbackSquareRoot()
│ sca ┼ SumScalariser()
│ ccnt ┼ nothing
│ cobj ┼ nothing
│ sc ┼ Int64: 1
│ so ┼ Int64: 1
│ ss ┼ nothing
│ card ┼ nothing
│ scard ┼ nothing
│ l2c ┼ nothing
│ lpc ┼ nothing
│ linfc ┼ nothing
│ l1 ┼ nothing
│ l2 ┼ nothing
│ lp ┼ nothing
│ linf ┼ nothing
│ xbgt ┼ Bool: false
│ strict ┴ Bool: false
r ┼ Variance
│ settings ┼ RiskMeasureSettings
│ │ scale ┼ Float64: 1.0
│ │ ub ┼ nothing
│ │ rke ┴ Bool: true
│ sigma ┼ nothing
│ chol ┼ nothing
│ rc ┼ nothing
│ alg ┼ SquaredSOCRiskExpr()
│ mtx_sqrt ┼ EigenFallbackSquareRoot()
│ ratio_alg ┼ SquareRootRatio()
│ corner ┴ nothing
obj ┼ MinimumRisk()
wi ┼ nothing
fb ┴ nothing
)
5.1 Combinatorial paths
Every split trains on its training groups, which need not be consecutive, and predicts the groups it holds out. The test groups of the splits join into paths. expected_risk over the population gives one risk value per path. The spread of those values shows how much the result depends on the path.
comb = CombinatorialCrossValidation(; n_folds = 5, n_test_folds = 2)pp = cross_val_predict(rpipe, rd, comb)cvar = ConditionalValueatRisk()pretty_table(DataFrame(; path = [p.id for p in pp.pred], folds = [length(p.pred) for p in pp.pred], cvar = expected_risk(cvar, pp)); formatters = [resfmt])┌───────┬───────┬─────────┐
│ path │ folds │ cvar │
│ Int64 │ Int64 │ Float64 │
├───────┼───────┼─────────┤
│ 1 │ 5 │ 2.679 % │
│ 2 │ 5 │ 2.662 % │
│ 3 │ 5 │ 2.669 % │
│ 4 │ 5 │ 2.678 % │
└───────┴───────┴─────────┘5.2 Paths over random subsets of assets
MultipleRandomised draws a random subset of assets for every path and runs an inner walk-forward on it. It draws only from the assets with a finite price or return at every row of that path's window. It applies the subset to the input. The pipeline then fits from the start on each subset.
mr = MultipleRandomised(IndexWalkForward(500, 250); subset_size = 6, n_subsets = 4, rng = StableRNG(20240607))pm = cross_val_predict(rpipe, rd, mr)pretty_table(DataFrame(; path = [p.id for p in pm.pred], folds = [length(p.pred) for p in pm.pred], assets = [length(p.pred[1].res.w) for p in pm.pred]))┌───────┬───────┬────────┐
│ path │ folds │ assets │
│ Int64 │ Int64 │ Int64 │
├───────┼───────┼────────┤
│ 1 │ 2 │ 6 │
│ 2 │ 2 │ 6 │
│ 3 │ 2 │ 6 │
│ 4 │ 2 │ 6 │
└───────┴───────┴────────┘6. What a pipeline does not do
- A pipeline is not an optimisation estimator.
optimisethrows an error for a pipeline, so fit it withfit. - A meta-optimiser such as
NestedClustered,StackingorSubsetResamplingdoes not take a pipeline as an inner estimator, and its constructor throws an error. The other way round works, and a meta-optimiser can be the optimisation step of a pipeline. - A pipeline with no optimisation step is valid, for example one that fits only a prior.
predictthrows an error for it, because it has no weights.
The first call prints the error that optimise gives for a pipeline.
try optimise(pipe, res.ctx.returns)catch e println(e.msg)enda Pipeline is a workflow, not an OptimisationEstimator: fit it with fit(pipeline, data). Wrapping a Pipeline inside a meta-optimiser is not supported.The second call fits a pipeline that ends with a prior step, and predicts with it. It prints the error that predict gives, because the pipeline has no weights.
try predict(fit(Pipeline(; steps = (PricesToReturns(), EmpiricalPrior())), pr), pr, 801:T)catch e println(e.msg)endthe pipeline produced no optimisation result; add a terminal optimisation step before predicting7. Summary
A Pipeline makes the data preparation part of the fitted model, so you can tune it.
- A step is an ordinary estimator, and its kind decides the slot of the context it writes.
- A preprocessing step learns its state on the training window, such as the assets to keep and the fill values. A prediction applies that state to the test window.
- The pipeline puts the fitted prior and the fitted constraints into the optimiser, so the optimiser does not estimate them again.
- Cross-validation splits the input rows. The pipeline fits its cleaning steps again in every fold, and no test window enters an asset filter or a fill constant.
- One grid can hold the parameters of the preprocessing and of the optimiser, and a grid value can replace a whole step.
docs/adr/0028-pipeline-workflow-estimator.md gives the reasons for the design and lists the features it leaves for later.
This page was generated using Literate.jl.