The source files can be found in user_guide/.
Validation and tuning
Cross-validation scores a strategy on data that it was not fitted to, and a search chooses the parameters of the strategy by that score. PortfolioOptimisers.jl has cross-validation splitters, and a cross-validated search over parameters, that work with any optimiser. This page shows the minimal path. For the other splitters and searches, see the validation and tuning examples.
using PortfolioOptimisers, CSV, TimeSeries, Clarabel, StatsPlots, GraphRecipesX = TimeArray(CSV.File(joinpath(@__DIR__, "../examples/SP500.csv.gz")); timestamp = :Date)[(end - 252):end]rd = prices_to_returns(X)slv = Solver(; name = :clarabel, solver = Clarabel.Optimizer, settings = Dict("verbose" => false), check_sol = (; allow_local = true, allow_almost = true))mr = MeanRisk(; opt = JuMPOptimiser(; slv = slv))MeanRisk
opt ┼ JuMPOptimiser
│ pe ┼ EmpiricalPrior
│ │ ce ┼ PortfolioOptimisersCovariance
│ │ │ ce ┼ Covariance
│ │ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ │ w ┴ nothing
│ │ │ │ ce ┼ GeneralCovariance
│ │ │ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ │ │ w ┴ nothing
│ │ │ │ alg ┼ FullMoment()
│ │ │ │ w ┴ nothing
│ │ │ mp ┼ MatrixProcessing
│ │ │ │ pdm ┼ Posdef
│ │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ │ │ dn ┼ nothing
│ │ │ │ dt ┼ nothing
│ │ │ │ alg ┼ nothing
│ │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
│ │ me ┼ SimpleExpectedReturns
│ │ │ w ┴ nothing
│ │ horizon ┼ nothing
│ │ fill_limit ┴ nothing
│ slv ┼ Solver
│ │ name ┼ Symbol: :clarabel
│ │ solver ┼ UnionAll: Clarabel.MOIwrapper.Optimizer
│ │ settings ┼ Dict{String, Bool}: Dict{String, Bool}("verbose" => 0)
│ │ check_sol ┼ @NamedTuple{allow_local::Bool, allow_almost::Bool}: (allow_local = true, allow_almost = true)
│ │ add_bridges ┴ Bool: true
│ wb ┼ WeightBounds
│ │ lb ┼ Float64: 0.0
│ │ ub ┴ Float64: 1.0
│ bgt ┼ Float64: 1.0
│ sbgt ┼ nothing
│ gbgt ┼ nothing
│ xbgt ┼ Bool: false
│ lt ┼ nothing
│ st ┼ nothing
│ lcse ┼ nothing
│ cte ┼ nothing
│ gcarde ┼ nothing
│ sgcarde ┼ nothing
│ smtx ┼ nothing
│ sgmtx ┼ nothing
│ slt ┼ nothing
│ sst ┼ nothing
│ sglt ┼ nothing
│ sgst ┼ nothing
│ tn ┼ nothing
│ fees ┼ nothing
│ sets ┼ nothing
│ tr ┼ nothing
│ ple ┼ nothing
│ ret ┼ ArithmeticReturn
│ │ settings ┼ JuMPReturnsSettings
│ │ │ scale ┼ Float64: 1.0
│ │ │ lb ┼ nothing
│ │ │ rte ┼ Bool: true
│ │ │ fee ┼ Bool: true
│ │ │ mic ┴ Bool: true
│ │ ucs ┼ nothing
│ │ mu ┴ nothing
│ sca ┼ SumScalariser()
│ ccnt ┼ nothing
│ cobj ┼ nothing
│ sc ┼ Int64: 1
│ so ┼ Int64: 1
│ ss ┼ nothing
│ card ┼ nothing
│ scard ┼ nothing
│ l2c ┼ nothing
│ lpc ┼ nothing
│ linfc ┼ nothing
│ l1 ┼ nothing
│ l2 ┼ nothing
│ lp ┼ nothing
│ linf ┼ nothing
│ brt ┼ Bool: false
│ x_src ┼ Symbol: :prior
│ strict ┴ Bool: false
r ┼ Variance
│ settings ┼ RiskMeasureSettings
│ │ scale ┼ Float64: 1.0
│ │ ub ┼ nothing
│ │ rke ┴ Bool: true
│ sigma ┼ nothing
│ chol ┼ nothing
│ rc ┼ nothing
│ alg ┴ SquaredSOCRiskExpr()
obj ┼ MinimumRisk()
wi ┼ nothing
fb ┴ nothing
1. Cross-validation
A cross-validation splitter divides the dates into folds. KFold is the simplest. It makes n folds of consecutive dates, and each fold is the test fold once, while the other folds train the optimiser. cross_val_predict fits the optimiser on the training folds of each split, predicts the returns of the portfolio on the test fold, and joins the predictions. So you can score the strategy on data that it did not see.
kfold = KFold(; n = 3)pred = cross_val_predict(mr, rd, kfold)MultiPeriodPredictionResult
pred ┼ 3-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 252-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ts ┼ 252-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
We compute the risk of the joined predictions with expected_risk. The measure here is the variance of the predicted returns.
cv_risk = expected_risk(LowOrderMoment(; alg = SecondMoment()), pred)0.00010006296287395223CombinatorialCrossValidation takes every combination of a number of test folds out of the folds, and joins the test folds into several paths through the dates. You get several out-of-sample paths in place of one, for more fits. See Cross Validation. A walk-forward can also update one estimator from fold to fold instead of refitting it on each fold, with OnlineIndexWalkForward. See The online walk-forward.
2. Hyperparameter tuning
GridSearchCrossValidation tries every combination of a grid of parameters, and keeps the one with the best score on the test folds. The grid is a list of "path" => values pairs, and the cell passes a list that holds one such grid. The path is a string that names a field inside the estimator, and the library turns it into a lens of Accessors.jl. A scoring measure, such as MeanReturnRiskRatio, ranks the candidates. search_cross_validation runs the search and returns the tuned estimator in its opt field. We tune the strength of the L1 penalty of our MeanRisk.
score = MeanReturnRiskRatio(; rk = LowOrderMoment(; alg = SecondMoment()))grid = [["opt.l1" => [0.001, 0.01, 0.05]]]gs_res = search_cross_validation(mr, GridSearchCrossValidation(grid; r = score), rd)SearchCrossValidationResult
opt ┼ MeanRisk
│ opt ┼ JuMPOptimiser
│ │ pe ┼ EmpiricalPrior
│ │ │ ce ┼ PortfolioOptimisersCovariance
│ │ │ │ ce ┼ Covariance
│ │ │ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ │ │ w ┴ nothing
│ │ │ │ │ ce ┼ GeneralCovariance
│ │ │ │ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ │ │ │ w ┴ nothing
│ │ │ │ │ alg ┼ FullMoment()
│ │ │ │ │ w ┴ nothing
│ │ │ │ mp ┼ MatrixProcessing
│ │ │ │ │ pdm ┼ Posdef
│ │ │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ │ │ │ dn ┼ nothing
│ │ │ │ │ dt ┼ nothing
│ │ │ │ │ alg ┼ nothing
│ │ │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
│ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ w ┴ nothing
│ │ │ horizon ┼ nothing
│ │ │ fill_limit ┴ nothing
│ │ slv ┼ Solver
│ │ │ name ┼ Symbol: :clarabel
│ │ │ solver ┼ UnionAll: Clarabel.MOIwrapper.Optimizer
│ │ │ settings ┼ Dict{String, Bool}: Dict{String, Bool}("verbose" => 0)
│ │ │ check_sol ┼ @NamedTuple{allow_local::Bool, allow_almost::Bool}: (allow_local = true, allow_almost = true)
│ │ │ add_bridges ┴ Bool: true
│ │ wb ┼ WeightBounds
│ │ │ lb ┼ Float64: 0.0
│ │ │ ub ┴ Float64: 1.0
│ │ bgt ┼ Float64: 1.0
│ │ sbgt ┼ nothing
│ │ gbgt ┼ nothing
│ │ xbgt ┼ Bool: false
│ │ lt ┼ nothing
│ │ st ┼ nothing
│ │ lcse ┼ nothing
│ │ cte ┼ nothing
│ │ gcarde ┼ nothing
│ │ sgcarde ┼ nothing
│ │ smtx ┼ nothing
│ │ sgmtx ┼ nothing
│ │ slt ┼ nothing
│ │ sst ┼ nothing
│ │ sglt ┼ nothing
│ │ sgst ┼ nothing
│ │ tn ┼ nothing
│ │ fees ┼ nothing
│ │ sets ┼ nothing
│ │ tr ┼ nothing
│ │ ple ┼ nothing
│ │ ret ┼ ArithmeticReturn
│ │ │ settings ┼ JuMPReturnsSettings
│ │ │ │ scale ┼ Float64: 1.0
│ │ │ │ lb ┼ nothing
│ │ │ │ rte ┼ Bool: true
│ │ │ │ fee ┼ Bool: true
│ │ │ │ mic ┴ Bool: true
│ │ │ ucs ┼ nothing
│ │ │ mu ┴ nothing
│ │ sca ┼ SumScalariser()
│ │ ccnt ┼ nothing
│ │ cobj ┼ nothing
│ │ sc ┼ Int64: 1
│ │ so ┼ Int64: 1
│ │ ss ┼ nothing
│ │ card ┼ nothing
│ │ scard ┼ nothing
│ │ l2c ┼ nothing
│ │ lpc ┼ nothing
│ │ linfc ┼ nothing
│ │ l1 ┼ Float64: 0.05
│ │ l2 ┼ nothing
│ │ lp ┼ nothing
│ │ linf ┼ nothing
│ │ brt ┼ Bool: false
│ │ x_src ┼ Symbol: :prior
│ │ strict ┴ Bool: false
│ r ┼ Variance
│ │ settings ┼ RiskMeasureSettings
│ │ │ scale ┼ Float64: 1.0
│ │ │ ub ┼ nothing
│ │ │ rke ┴ Bool: true
│ │ sigma ┼ nothing
│ │ chol ┼ nothing
│ │ rc ┼ nothing
│ │ alg ┴ SquaredSOCRiskExpr()
│ obj ┼ MinimumRisk()
│ wi ┼ nothing
│ fb ┴ nothing
test_scores ┼ 5×3 Matrix{Float64}
train_scores ┼ nothing
lens_grid ┼ 3-element Vector{Vector{ComposedFunction{Accessors.PropertyLens{:l1}, Accessors.PropertyLens{:opt}}}}
val_grid ┼ Vector{Tuple{Float64}}: [(0.001,), (0.01,), (0.05,)]
idx ┴ Int64: 3
We optimise the tuned estimator on the full sample, with the same call as any other estimator.
res_tuned = optimise(gs_res.opt, rd)MeanRiskResult
jr ┼ JuMPOptimisationResult
│ pa ┼ ProcessedJuMPOptimiserAttributes
│ │ pr ┼ LowOrderPrior
│ │ │ X ┼ 252×20 Matrix{Float64}
│ │ │ o_X ┼ nothing
│ │ │ mu ┼ 20-element Vector{Float64}
│ │ │ sigma ┼ 20×20 Matrix{Float64}
│ │ │ chol ┼ nothing
│ │ │ w ┼ nothing
│ │ │ ens ┼ nothing
│ │ │ kld ┼ nothing
│ │ │ ow ┼ nothing
│ │ │ rr ┼ nothing
│ │ │ fpr ┴ nothing
│ │ wb ┼ WeightBounds
│ │ │ lb ┼ 20-element StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}
│ │ │ ub ┴ 20-element StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}
│ │ lt ┼ nothing
│ │ st ┼ nothing
│ │ lcsr ┼ nothing
│ │ ctr ┼ nothing
│ │ gcardr ┼ nothing
│ │ sgcardr ┼ nothing
│ │ smtx ┼ nothing
│ │ sgmtx ┼ nothing
│ │ slt ┼ nothing
│ │ sst ┼ nothing
│ │ sglt ┼ nothing
│ │ sgst ┼ nothing
│ │ tn ┼ nothing
│ │ fees ┼ nothing
│ │ plr ┼ nothing
│ │ ret ┼ ArithmeticReturn
│ │ │ settings ┼ JuMPReturnsSettings
│ │ │ │ scale ┼ Float64: 1.0
│ │ │ │ lb ┼ nothing
│ │ │ │ rte ┼ Bool: true
│ │ │ │ fee ┼ Bool: true
│ │ │ │ mic ┴ Bool: true
│ │ │ ucs ┼ nothing
│ │ │ mu ┴ 20-element Vector{Float64}
│ │ sca ┼ SumScalariser()
│ │ imsk ┴ nothing
│ retcode ┼ OptimisationSuccess
│ │ res ┴ Dict{Any, Any}: Dict{Any, Any}()
│ sol ┼ JuMPOptimisationSolution
│ │ w ┴ 20-element Vector{Float64}
│ model ┼ A JuMP Model
│ │ ├ solver: Clarabel
│ │ ├ objective_sense: MIN_SENSE
│ │ │ └ objective_function_type: QuadExpr
│ │ ├ num_variables: 22
│ │ ├ num_constraints: 5
│ │ │ ├ AffExpr in MOI.EqualTo{Float64}: 1
│ │ │ ├ Vector{AffExpr} in MOI.Nonnegatives: 1
│ │ │ ├ Vector{AffExpr} in MOI.Nonpositives: 1
│ │ │ ├ Vector{AffExpr} in MOI.NormOneCone: 1
│ │ │ └ Vector{AffExpr} in MOI.SecondOrderCone: 1
│ │ └ Names registered in the model
│ │ └ :G, :T, :bgt, :cdev_soc_1, :cl1_noc, :dev_1, :k, :l1, :lw, :obj_expr, :op, :ret, :ret_1, :ret_vec, :risk, :risk_minimised, :risk_vec, :sc, :so, :t_l1, :variance_flag, :variance_risk_1, :w, :w_gross_ub, :w_lb, :w_ub
r ┼ Variance
│ settings ┼ RiskMeasureSettings
│ │ scale ┼ Float64: 1.0
│ │ ub ┼ nothing
│ │ rke ┴ Bool: true
│ sigma ┼ 20×20 Matrix{Float64}
│ chol ┼ nothing
│ rc ┼ nothing
│ alg ┴ SquaredSOCRiskExpr()
fb ┴ nothing
RandomisedSearchCrossValidation draws candidates from the grid, or from distributions, instead of trying every one. It costs less on a large grid. See Hyperparameter Tuning.
3. Time-dependent inputs
Under cross-validation, each fold is a separate optimisation over its own dates. An input that defines the problem, such as a constraint, a prior, a risk measure, an objective or the fallback optimiser, can change from fold to fold. Put a vector with one entry for each fold, or a function of the fold's context, in a TimeDependent, which this page calls a schedule. Then store the schedule in the field that it changes. The fold loop uses entry i for fold i. An input that controls how the problem runs, such as a solver or a random number generator, stays the same on every fold. We tighten the cap on the weight of each asset as a walk-forward moves forward.
wf = IndexWalkForward(126, 42)n = n_splits(wf, rd)bounds = [WeightBounds(; lb = 0.0, ub = ub) for ub in range(0.35, 0.2, n)]caps = TimeDependent(bounds)mr_caps = MeanRisk(; opt = JuMPOptimiser(; slv = slv, wb = caps))pred_caps = cross_val_predict(mr_caps, rd, wf)MultiPeriodPredictionResult
pred ┼ 3-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 126-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ts ┼ 126-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
An OnlinePortfolioSelection takes a schedule of its allocation set, the set of weights that its rule can hold, in the same way. The loop puts entry i in place before it processes the rows of fold i. So the weights at each row of fold i, and the weights that the fold reports, lie inside the set of entry i. Each entry is a whole allocation set. To change one bound and keep the others, write the whole set in each entry, or build it in a function of the fold's context.
caps_online = TimeDependent([BoundedAllocationSet(; wb = wb) for wb in bounds])ops_caps = OnlinePortfolioSelection(; alg = ExponentiatedGradient(), set = caps_online)pred_ops = cross_val_predict(ops_caps, rd, OnlineIndexWalkForward(126, 42))MultiPeriodPredictionResult
pred ┼ 3-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Vector{Int64}}, false}
│ X ┼ 126-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ts ┼ 126-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┼ OnlinePortfolioSelection
│ alg ┼ MirrorDescent
│ │ eta ┼ Float64: 0.05
│ │ proj ┼ EntropicProjection()
│ │ alpha ┼ Int64: 0
│ │ obj ┼ LogWealth()
│ │ grad ┴ PlainGradient()
│ set ┼ TimeDependent
│ │ val ┼ 3-element Vector{BoundedAllocationSet}
│ │ │ BoundedAllocationSet ⋯
│ │ │ BoundedAllocationSet ⋯
│ │ │ BoundedAllocationSet ⋯
│ │ bind ┼ Symbol: :outermost
│ │ default ┴ NoDefault()
│ w0 ┼ nothing
│ fees ┼ nothing
│ fb ┼ nothing
│ strict ┴ Bool: false
The entries of a schedule can be whole optimisers, so that the strategy changes from fold to fold. You can pass such a schedule to cross_val_predict as the optimiser. Outside a fold there is then no fixed optimiser, so the schedule names the optimiser that optimise runs there, in its default keyword.
iv = InverseVolatility()strategies = TimeDependent([isodd(i) ? mr : iv for i in 1:n]; default = mr)pred_switch = cross_val_predict(strategies, rd, wf)MultiPeriodPredictionResult
pred ┼ 3-element Vector{PredictionResult}
│ PredictionResult ⋯
│ PredictionResult ⋯
│ PredictionResult ⋯
mrd ┼ PredictionReturnsResult
│ nx ┼ 20-element SubArray{String, 1, Vector{String}, Tuple{Base.Slice{Base.OneTo{Int64}}}, true}
│ X ┼ 126-element Vector{Float64}
│ nf ┼ nothing
│ F ┼ nothing
│ nb ┼ nothing
│ B ┼ nothing
│ ts ┼ 126-element Vector{Date}
│ iv ┼ nothing
│ ivpa ┴ nothing
id ┼ nothing
opt ┴ nothing
A schedule in a field such as wb needs no default. Outside a fold loop, the field then takes the value that the constructor gives it when you do not pass it. For schedules in the fields of a meta-optimiser, functions that read the data of the fold, and schedules that hold computed results, see Time Dependent Constraints and Time Dependent Optimisers.
4. Cross-validation scores
plot_cv_scores plots the out-of-sample score of each fold, so you can see how much the score of the strategy changes from fold to fold.
plot_cv_scores(LowOrderMoment(; alg = SecondMoment()), pred)
This page was generated using Literate.jl.