Base meta optimisation: private API
PortfolioOptimisers.FullUniverse — Type
struct FullUniverse <: SubPortfolioUniverseMakes each inner optimiser a sub-portfolio that sees the whole universe.
This is the enumeration of Stacking. The module views nothing onto a sub-portfolio, and an inner weight vector already covers every asset, so fold_weight_matrix adds no zeros.
Related
PortfolioOptimisers.ClusterUniverse — Type
struct ClusterUniverse{__T_cls} <: SubPortfolioUniverseMakes each cluster a sub-portfolio that sees only its own assets.
This is the enumeration of NestedClustered. One inner optimiser serves every sub-portfolio, viewed onto the assets of that sub-portfolio, so an inner weight vector is as long as its cluster. fold_weight_matrix puts zeros at the other assets.
Fields
cls: Asset indices of each sub-portfolio. They partition the universe, so a column with zeros at the other assets is the real weight of the sub-portfolio over the whole asset axis.
Related
PortfolioOptimisers.sub_portfolio_cv — Function
sub_portfolio_cv(cv)Give one sub-portfolio its own copy of the cross-validation scheme.
The module cross-validates the sub-portfolios in parallel. A scheme that draws its splits from a random number generator holds mutable state, and two sub-portfolios that advance one generator at the same time get different folds on each run. So the method copies a scheme that has an rng field, and each copy starts from the same state. It returns a scheme with no rng field unchanged.
No scheme that the cv field of a meta-optimiser accepts (OptCVE_Onl) has an rng field, so every scheme a meta-optimiser runs today takes the second branch. A scheme with an rng field must implement Base.copy.
Arguments
cv: Cross-validation scheme.
Returns
- A copy of
cvwhen it has anrngfield, otherwisecvitself.
Related
PortfolioOptimisers.outer_optimisation_finaliser — Function
outer_optimisation_finaliser(wb::WeightBounds, wf::WeightFinaliser, resi::VecOpt,
rco::OptimisationReturnCode, w::VecNum, wi::MatNum)
outer_optimisation_finaliser(wb::WeightBounds, wf::WeightFinaliser, resi::VecOpt,
rcos::VecOptRetCode, ws::VecVecNum, wi::MatNum)Combine the inner and outer weights of a meta-optimiser, and finalise them under its weight bounds.
The return code reports a failure at any of the three stages: an inner solve, the outer solve, and the finalisation of the combined weights.
Mathematical definition
\[\begin{align} \boldsymbol{w} &= \mathbf{W} \boldsymbol{v}\,. \end{align}\]
Where:
- $\boldsymbol{w}$: Combined weights over the $N$ assets, before the finalisation.
- $\mathbf{W}$: Inner weight matrix
assets × sub-portfolios. Column $k$ holds the weights of sub-portfolio $k$ over all $N$ assets, and zero at an asset outside the sub-portfolio. - $\boldsymbol{v}$: Outer weight vector, the weights that the outer optimiser gives the $K$ synthetic assets.
Stackingpasses the coefficients ofcombination_weightsin its place. - $N$: Number of assets.
Algorithm
- Combine the weights, giving
w = wi * w. - Finalise
wwithwfunderwbthroughfinalise_weight_bounds, givingretcodeand the finalisedw. - Read the return code of every inner result, giving
resi_retcodes. - When an inner return code,
rcoorretcodeis anOptimisationFailure, replaceretcodewith anOptimisationFailurewhoseresis(; msg, opti, opto, wb).msghas one line for each stage that failed, in the order inner, outer, finalisation.optiisresi_retcodes,optoisrco, andwbis the return code of step 2. - Return
(retcode, w). The method returns the weights of step 2 when a stage fails too.
The method for an efficient frontier runs these steps once for each pair of rcos and ws.
Arguments
wb: Weight bounds over the $N$ assets.wf: Weight finaliser.resi: The results of the inner optimisations.rco: The return code of the outer optimisation.w: Outer weights, one entry for each sub-portfolio.rcos,ws: The return codes and outer weights of the points of an efficient frontier.wi: Inner weights,assets × sub-portfolios.
Returns
(retcode, w): The return code and the finalised weights. The frontier method returns a vector of return codes and a vector of weight vectors.
Related
PortfolioOptimisers.combination_weights — Function
combination_weights(scale::Nothing, w::VecNum_VecVecNum)
combination_weights(scale::VecNum, w::VecNum)
combination_weights(scale::VecNum, w::VecVecNum)Apply a Combination Weight to the outer weights of a meta-optimiser.
The outer optimiser decides how much of each sub-portfolio to hold. A Combination Weight is a fixed belief about the same quantity, so the two multiply, and the method rescales the products to the total that the outer optimiser chose. Schur Complement Hierarchical Risk Parity blends its parameter bundles in the same way.
The rescale makes only the ratios between the entries of a Combination Weight matter. A common factor cancels, so the weight needs no normalised form. Three cases give back w unchanged:
- a uniform weight, whatever its sum;
- a lone sub-portfolio, because one element is not a combination;
- an outer total other than one keeps its total, so the method does not overrule a
bgtof0.9.
The outer problem never sees the weight. A common factor on every synthetic return column moves the trade-off between return and risk of MaximumUtility, so a uniform weight in the outer problem is not neutral. A weight in the outer problem also needs every predict_outer_returns method to apply it, and a method that drops it makes a cross-validated run disagree with a fold-less run.
Mathematical definition
\[\begin{align} c_k &= \begin{cases} \dfrac{s_k v_k}{\sum_{j=1}^{K} s_j v_j} \sum_{j=1}^{K} v_j & \text{if } \sum_{j=1}^{K} s_j v_j \neq 0 \text{ and } \sum_{j=1}^{K} v_j \neq 0\,,\\ s_k v_k & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $c_k$: Coefficient of sub-portfolio $k$ in the combination.
- $s_k$: Combination Weight of sub-portfolio $k$.
- $\boldsymbol{v}$: Outer weight vector, the weights that the outer optimiser gives the $K$ synthetic assets.
- $K$: Sub-portfolio count of a meta-optimiser, which is also the number of synthetic assets of its outer problem.
The first case keeps the total, $\sum_k c_k = \sum_k v_k$, and the ratios, $c_k / c_j = s_k v_k / (s_j v_j)$. In the second case no scalar rescale can keep a zero total and apply the tilt at the same time, so the tilted products stand. They are finite and keep their ratios. A denominator near zero is not a degenerate case. The large factor is the correct rescale of a combination that nearly cancels.
Algorithm
- Multiply the weight into the outer weights, giving
c = scale .* w. - Divide the two totals, giving the factor
f = sum(w) / sum(c). - Return
cwhenfis zero or not finite, otherwisec * f. A factor that overflows is not finite, so it takes the second case too.
A scale of nothing returns w, and a frontier applies the steps to each point.
Arguments
scale: Combination Weight, one entry for each sub-portfolio, ornothing.w: Outer weights, one entry for each sub-portfolio, or a vector of them on an efficient frontier.
Returns
wwhenscaleisnothing, otherwise the coefficients $\boldsymbol{c}$.
Related
PortfolioOptimisers.prepare_outer_rd — Function
prepare_outer_rd(
rd::ReturnsResult,
wi::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
alg::AbstractPanelCollapseAlgorithm
) -> Tuple{Any, Any, Any, Any, Union{Nothing, AssetPanel}, Matrix}
Prepare the returns data of the outer problem from the inner weights wi.
The method collapses every per-asset quantity of rd onto the synthetic assets, and it allocates the buffer that the caller fills with the synthetic returns. Benchmark returns are extensive and collapse as a weighted sum. The implied volatilities and the Asset Panel are intensive and collapse as convex combinations, so the gross exposure of a sub-portfolio does not scale them. On a time-varying panel, the implied volatilities read the active members of each row, under the rule alg of the panel collapse, as the Asset Panel does (see collapse_asset_panel).
Destructure all six returned values. Julia discards trailing values without an error, so a caller that names five binds pnl to the name it means for the buffer X, and its first write into that name fails.
Mathematical definition
\[\begin{align} \mathbf{B}^{o} &= \mathbf{B} \mathbf{W}\,,\\ V^{o}_{tk} &= \frac{\sum_{i=1}^{N} \tilde{W}_{ik} m_{ti} V_{ti}}{d_{tk}}\,,\\ a^{o}_{k} &= \frac{\sum_{i=1}^{N} \tilde{W}_{ik} m_{Ti} a_{i}}{d_{Tk}}\,. \end{align}\]
Where:
- $\mathbf{B}$, $\mathbf{B}^{o}$: Benchmark returns,
observations × assetsandobservations × sub-portfolios. A benchmark that is not a matrix is kept as it is. - $V_{ti}$, $V^{o}_{tk}$: Implied volatilities,
observations × assetsandobservations × sub-portfolios. - $a_{i}$, $a^{o}_{k}$: Implied volatility risk premium adjustment, one entry for each asset and one for each sub-portfolio. A scalar adjustment is kept as it is.
- $m_{ti}$: Active mask of the Asset Panel, $1$ when asset $i$ is in the universe at observation $t$ and $0$ otherwise. With no panel, or a static one, $m_{ti} = 1$.
- $T$: The last observation of
rd, the one at which the outer fit stands. - $d_{tk}$: Divisor of the rule
alg, seeactive_weight_divisor. It is $1$ with no panel, with a static one, and underInactiveAsCash. - $\mathbf{W}$: Inner weight matrix
assets × sub-portfolios. Column $k$ holds the weights of sub-portfolio $k$ over all $N$ assets, and zero at an asset outside the sub-portfolio. - $\tilde{\mathbf{W}}$: Normalised inner weight matrix, $\tilde{W}_{ik} = |W_{ik}| / \sum_{j=1}^{N} |W_{jk}|$. A column of zeros stays a column of zeros.
- $N$: Number of assets.
A column of zeros in $\mathbf{W}$ gives a column of zeros in $\mathbf{V}^{o}$ and a zero entry in $\boldsymbol{a}^{o}$.
Algorithm
- When
rd.Bis a matrix, collapse the cells thatbenchmark_cellsreads, givingB = benchmark_cells(rd) * wi, and name its columnsnb = ["_b1", …]. A gap in the benchmark or in the returns of an asset, or an inactive cell, gives zero and notNaN. Otherwise keeprd.Bandrd.nb. - Read the active mask of
rd.pnl, givingm. It isnothingwith no panel or a static one. - When
rd.ivis present orrd.ivpais a vector, normalisewiwithsynthetic_asset_weights, givingwn. Collapserd.ivandrd.ivpawithcollapse_rate, onwn,mandalg. - Collapse the Asset Panel with
collapse_asset_panelunderalg, givingpnl. - Allocate the buffer
X,observations × sub-portfolios, with the element type that promotes the element types ofrd.Xandwi.
Arguments
rd: The returns data of the meta-optimiser.wi: Inner weights,assets × sub-portfolios.alg: The rule of the panel collapse, thepcolfield of the meta-optimiser.
Returns
nb: Names of the benchmark columns.B: Benchmark returns.iv: Implied volatilities, ornothing.ivpa: Implied volatility risk premium adjustment, ornothing.pnl: Asset Panel over the synthetic assets, ornothing.X: Uninitialised buffer for the outer returns matrix.
Related
PortfolioOptimisers.outer_prior_rows — Function
outer_prior_rows(
rd::ReturnsResult,
pr::AbstractPriorResult
) -> ReturnsResult
View the returns data of a meta-optimiser onto the observations its Prior Result answers on.
The outer problem reads the net return of each sub-portfolio off the scenarios of the Prior Result, one row per scenario, so its returns data must hold one row per scenario too. A prior that answers on every observation leaves rd whole. A CrossSectionalFactorPrior answers on the observations it fitted, which are the last ones of rd, because its Descriptor warm-up and its exposure lag consume the first. So a prior with fewer scenarios than rd has observations takes the last rows of rd, and the timestamps, the benchmark, the Exogenous Series and the Asset Panel of the outer problem follow them. A prior that dropped the observations of a Shared Series gap from the rd that it read takes the last of the observations that it kept, see prior_data_rows.
Arguments
rd: The returns data of the meta-optimiser.pr: The Prior Result over the whole universe.
Validation
size(pr.X, 1) <= size(rd.X, 1). Raises aDimensionMismatch, because a prior with more scenarios than observations states no row ofrdfor each of them.
Returns
rd::ReturnsResult:rditself, or its view onto thesize(pr.X, 1)observations thatprior_data_rowspairs with the scenarios.
Related
PortfolioOptimisers.assert_fold_alignment — Function
assert_fold_alignment(predictions) -> VecPredResCheck that fold f of every sub-portfolio covers the same observations, and return the folds of the first sub-portfolio.
A meta-optimiser runs the same cross-validation scheme over the same returns data for every sub-portfolio, so fold f covers the same observations for each of them. rebuild_returns_result needs this. Its reshape(X, :, N) lines up the N stacked return vectors row by row, and the weight matrix of a fold exists only when they agree. The check matters most on the combinatorial path, where the scorer of each sub-portfolio selects its path on its own.
Algorithm
- Read the folds of the first sub-portfolio, giving
pred1, and their numbernf. - For each sub-portfolio, check that it has
nffolds. - For each fold, compare the timestamps with those of
pred1when the returns data has a clock, and the number of return entries when it has none.
The number of entries is the strongest check available without a clock, and it is the one reshape needs.
Arguments
predictions: OneMultiPeriodPredictionResultfor each sub-portfolio.
Validation
- Every sub-portfolio has the same number of folds as the first, else
DimensionMismatch. - Fold
fof every sub-portfolio has the timestamps of foldfof the first, or the same number of return entries when there are no timestamps, elseDimensionMismatch.
Returns
- The per-fold
PredictionResultobjects of the first sub-portfolio.
Related
PortfolioOptimisers.fold_row_indices — Function
fold_row_indices(rd, pred) -> VecVecIntFind the rows of the original returns data that each cross-validation fold covers.
A fold records its rows in the idx of its PredictionResult, and rebuild_fold_rates reads them there. This method finds them from the timestamps. port_opt_view slices ts with the test_idx of the fold, so the rd.ts of a fold is its slice of the original clock, and feature_row_indices finds the rows from it. This works on the combinatorial path too, where sort_predictions puts the folds of a path in split order and not in time order. The timestamps carry the order that the folds have.
Only a time-varying Asset Panel needs the rows. A static panel has no observation axis, so only the time-varying shape needs the clock.
Arguments
rd: The originalReturnsResult. The observation axis of a time-varyingrd.pnlis parallel to itsts.pred: The per-foldPredictionResultobjects of one sub-portfolio.
Validation
rd.tsis notnothing, elseIsNothingError.
Returns
- One vector of row indices for each fold.
Related
PortfolioOptimisers.fold_asset_panel — Function
fold_asset_panel(pnl::AssetPanel, nx, wi, anchor, alg) -> AssetPanelCollapse the original AssetPanel onto the synthetic assets of one fold, with an observation axis.
The collapse is collapse_asset_panel of the original panel with the weights of the fold, under the rule alg of the panel collapse. The anchor of the fold depends on the shape of the panel:
- A static panel has no observation axis, so its anchor is the number of observations of the fold. The method repeats its one collapsed panel over them. The collapse depends on the weights of the fold, so it is constant in one fold and changes in the next, and the outer problem sees a time-varying panel.
- For a time-varying panel, the method first cuts the panel to the rows of the fold in the original clock, and the collapse keeps that observation axis and the collapsed universe masks of those rows. The anchor is those rows, and
fold_row_indicesfinds them from the timestamps of the fold.
Algorithm
- For a static panel, collapse it, giving
c, and lift every field ofcto theanchorobservations of the fold withpanel_field_lift. Both universe masks are alltrue. - For a time-varying panel, view the rows
anchorwithport_opt_viewand collapse that view.
rebuild_asset_panel returns before this method when the returns data has no panel.
Arguments
pnl: The original Asset Panel, not sliced.nx: The asset names of the returns data, ornothing. Only the square case reads them.wi: The weights of the fold,assets × sub-portfolios.anchor: The number of observations of the fold for a static panel, its rows for a time-varying one.alg: The rule of the panel collapse, thepcolfield of the meta-optimiser.
Returns
- A time-varying Asset Panel over the observations of the fold.
Related
PortfolioOptimisers.fold_feature_anchors — Function
fold_feature_anchors(rd, pred)Give each fold the anchor that fold_asset_panel needs: the number of observations for a static panel, and the rows for a time-varying one.
Only a time-varying panel needs rows, so only that shape asks the returns data for timestamps. A static panel runs on the fold sizes alone.
Arguments
rd: The originalReturnsResult.pred: The per-foldPredictionResultobjects of one sub-portfolio.
Returns
- One anchor for each fold: an
Integerfor a static panel or no panel, a vector of row indices for a time-varying one.
Related
PortfolioOptimisers.panel_field_stack — Method
panel_field_stack(fs::AbstractVector{<:NumericPanelField}) -> NumericPanelField
panel_field_stack(fs::AbstractVector{<:TensorPanelField}) -> TensorPanelFieldStack the collapses of one Panel Field over the folds, along the observation axis.
Every fold collapses the same source field onto the same synthetic assets, so the fold results agree on every axis except the observations, and the stack concatenates them there. The observed masks stack with the values. collapse_panel_field turns a categorical field into a tensor field, so a fold result is never categorical.
Algorithm
- Concatenate the values of every fold along the observation axis.
- Concatenate the observed masks in the same way, or keep
nothingwhen the field carries none. Concatenate the placeholder masks withpanel_pmsk_vcat. - Build the field again with its keyword constructor, which runs every check again.
Arguments
fs: One collapsed Panel Field for each fold, in fold order.
Returns
- The stacked Panel Field.
Related
PortfolioOptimisers.rebuild_asset_panel — Function
rebuild_asset_panel(rd, predictions, u, pred1, alg)Compute the AssetPanel of the outer problem on the cross-validated path.
For each fold, the method makes the same collapse_asset_panel call that prepare_outer_rd makes on the fold-less path, with the same asset names, the same original panel and the same rule alg, and the weights of that fold. Thus cv, which controls execution only, does not change what the outer optimiser measures.
The stacked panel is time-varying. The fold panels cover disjoint windows of one synthetic universe, and the method stacks their universe masks with their values. So a sub-portfolio with no active member at an observation is inactive there, on this path as on the fold-less one. A static panel has no inactive member, and its folds give universe masks that are all true.
Algorithm
- Return
nothingwhenrdcarries no panel. - For each fold, build its weight matrix with
fold_weight_matrixand collapse the original panel withfold_asset_panelat the anchor fromfold_feature_anchors, givingps. - Stack each Panel Field over the folds with
panel_field_stack, givingpf. - Stack the active masks and the estimation masks of the folds, and build the panel.
Arguments
rd: The originalReturnsResult. The method collapses its panel without slicing it first.predictions: OneMultiPeriodPredictionResultfor each sub-portfolio.u: Sub-portfolio enumeration, aSubPortfolioUniverse.pred1: The folds of the first sub-portfolio, fromassert_fold_alignment. Every sub-portfolio agrees with them, so they set the fold boundaries.alg: The rule of the panel collapse, thepcolfield of the meta-optimiser.
Returns
pnl::Option{AssetPanel}: The Asset Panel over the synthetic assets, ornothingwhenrdcarries none.
Related
PortfolioOptimisers.rebuild_fold_rates — Function
rebuild_fold_rates(rd, predictions, u, pred1, alg, iv, ivpa)Compute the implied volatilities of the outer problem on the cross-validated path, under the rule of the panel collapse.
Each fold collapses rd.iv and rd.ivpa with the weights of its sub-portfolio through collapse_fold_rate, which reads the active mask and the finite cells under RenormaliseActive, whatever the rule. The fold-less path weighs the members that it reads under the rule (see prepare_outer_rd). So this method collapses the original rd.iv and rd.ivpa again for each fold, with collapse_rate on the weights of that fold, as rebuild_asset_panel does for the panel. It does this with no panel, a static one and a time-varying one: under InactiveAsCash the two rules differ wherever iv holds a cell that is not finite. The rows of a fold are the idx that its PredictionResult records, so the method needs no timestamps. Under RenormaliseActive with no active mask, the values of the predictions are already the collapse of the rule, and the method keeps them bit for bit.
Algorithm
- Read the active mask of
rd.pnlwithpanel_moment_masks, givingam. It isnothingwith no panel or a static one. - Return
ivandivpawhenrdcarries neitherrd.ivnor a vectorrd.ivpa, or whenalgisRenormaliseActiveandamisnothing. - For each fold, normalise its weight matrix from
fold_weight_matrixwithsynthetic_asset_weights, and collapse the rowsidxofrd.ivand the vectorrd.ivpawithcollapse_rate, on the rowsidxof the active mask. - Stack the collapsed
ivof the folds in fold order, and takeivpafrom the last fold, as each prediction keeps the adjustment of its last fold.
Arguments
rd: The originalReturnsResult.predictions: OneMultiPeriodPredictionResultfor each sub-portfolio.u: Sub-portfolio enumeration, aSubPortfolioUniverse.pred1: The folds of the first sub-portfolio, fromassert_fold_alignment.alg: The rule of the panel collapse, thepcolfield of the meta-optimiser.iv: The stacked implied volatilities of the predictions,observations × sub-portfolios, ornothing.ivpa: The adjustments of the predictions, one for each sub-portfolio, ornothing.
Validation
- Every fold of
pred1records its rows inidx, elseIsNothingError.
Returns
(iv, ivpa): The implied volatilities and the adjustments of the outer problem.
Related
PortfolioOptimisers.rebuild_returns_result — Function
rebuild_returns_result(rd, predictions, u, alg)Build the returns data of the outer problem from the cross-validation predictions of the sub-portfolios.
Column k of the result is the out-of-sample prediction of sub-portfolio k, and its rows are the rows of the folds, in fold order. u is the sub-portfolio enumeration: a ClusterUniverse for NestedClustered and a FullUniverse for Stacking. It states where the weights of a fold sit on the asset axis.
u is positional and has no default. With a full-universe default, a two-argument call still runs. That is correct for Stacking, but for NestedClustered it writes the weights of every cluster to the wrong rows and gives a wrong Asset Panel with no error. alg is the rule of the panel collapse, and it has no default for the same reason: the fold-less path reads the rule of the meta-optimiser, and this path must read the same one.
The folds carry no Asset Panel. The method collapses the original rd.pnl again for each fold, with the same collapse_asset_panel call as prepare_outer_rd makes on the fold-less path, and stacks the fold results along the observation axis (see rebuild_asset_panel). The stack has the assets × features × observations shape of a time-varying panel, and a FeatureDistance with its default LastRow reads the collapse of the last fold. The inner solves still see the panel of their own cluster. The collapse of the original panel also serves a square panel under NestedClustered. Its folds see the returns of one cluster each, so their own panels have different feature axes and do not stack.
Algorithm
- Copy the returns
X, andBandivwhen present, of the first prediction. Wrap itsivpain a vector when present. Each prediction keeps theivpaof its last fold as one number. - Check the folds with
assert_fold_alignment, givingpred1. - Append the returns,
ivandBof every other prediction, and push itsivpa. - Reshape the returns to
observations × sub-portfolios, givingX. - Check that the folds cover
size(X, 1)observations,nobs. - Build the Asset Panel with
rebuild_asset_panelunderalg, givingpnl. - Reshape
Bandivtoobservations × sub-portfolios, and name the benchmark columns_b1,_b2, …. - Collapse
ivandivpaagain underalgwithrebuild_fold_rates. - Build the
ReturnsResult, with the asset names_1,_2, … and the factors and timestamps of the first prediction.
The method reads predictions and does not change them, so two calls on one vector give the same result.
Arguments
rd: The originalReturnsResult.predictions: OneMultiPeriodPredictionResultfor each sub-portfolio.u: Sub-portfolio enumeration, aSubPortfolioUniverse.alg: The rule of the panel collapse, thepcolfield of the meta-optimiser.
Validation
- The checks of
assert_fold_alignment. - The folds of the first sub-portfolio cover as many observations as the stacked returns have rows, else
DimensionMismatch. - The checks of
rebuild_fold_rates.
Returns
- The returns data of the outer problem, one synthetic asset for each sub-portfolio.
Related
PortfolioOptimisers.sub_portfolio_predictions — Function
sub_portfolio_predictions(::Type{T}, opti, u, rd, cv, ex) where {T}Cross-validate every sub-portfolio in parallel, over the same returns data.
Every sub-portfolio runs the same cross-validation over the same returns data, which is the condition that assert_fold_alignment checks.
Algorithm
- Allocate one slot of type
Tfor each of thesub_portfolio_countsub-portfolios, givingpredictions. - For each sub-portfolio
i, in parallel underex, give it its own scheme withsub_portfolio_cvand write the result ofsub_portfolio_predicttopredictions[i].
Arguments
T: Element type of the prediction vector. It isMultiPeriodPredictionResulton the non-combinatorial path andPopulationPredictionResulton the combinatorial one.opti: The inner optimiser field of the meta-optimiser.u: Sub-portfolio enumeration, aSubPortfolioUniverse.rd: Returns data.cv: Cross-validation scheme.ex: FLoops executor that controls parallelism.
Returns
- One prediction result for each sub-portfolio.
Related
PortfolioOptimisers.outer_returns_source — Function
outer_returns_source(pr::AbstractPriorResult, rd::ReturnsResult,
x_src::Option{<:AbstractReturnsSource})
outer_returns_source(pr::AbstractPriorResult, rd::ReturnsResult, x_src::ExcessSource)Return the returns on which the outer problem of a meta-optimiser scores its sub-portfolios, in the raw frame of rd.
The outer problem reads the raw returns of the sub-portfolios, as a cross-validated prediction reads them, and the readers of the outer optimiser state their own sources. The returns of the Prior Result are in the frame that the prior of the meta-optimiser was fitted in, and the Prior Result records that frame, see prior_returns_source. When it records ExcessSource(), the frame is the excess over the benchmark, so this function adds the benchmark back. The source comes from the Result and not from the estimator, because three paths put a Result in place of the prior estimator: the call with no data on an online fold, a prior step of a Pipeline, and a Prior Result that the caller computed. A Prior Result that a returns-matrix method of prior fitted records no source, so the outer problem reads it as raw.
Mathematical definition
\[\begin{align} \mathbf{S} &= \begin{cases} \mathbf{X}_{\mathcal{P}} & \text{the prior reads the raw returns}\,,\\ \mathbf{X}_{\mathcal{P}} + \mathbf{B} & \text{the prior states the Excess Frame}\,. \end{cases} \end{align}\]
Where:
- $\mathbf{S}$: Returns that the sub-portfolios are scored on,
observations × assets. - $\mathbf{X}_{\mathcal{P}}$: Returns of the Prior Result,
pr.X. - $\mathbf{B}$: Benchmark returns of
rd. A vector benchmark adds the same value to each asset of an observation.
Arguments
pr: The Prior Result of the meta-optimiser.rd: The returns data of the meta-optimiser, on the observations ofpr.x_src: The source thatprrecords, seeprior_returns_source.
Returns
- The matrix $\mathbf{S}$ when
x_srcisExcessSource, andprotherwise.
Related
PortfolioOptimisers.predict_outer_returns — Function
predict_outer_returns(cv::Nothing, opt, u::SubPortfolioUniverse, rd::ReturnsResult,
pr::AbstractPriorResult, fees::Option{<:Fees}, wi::MatNum,
resi::VecOpt)
predict_outer_returns(cv::OptCVE_Onl, opt, u::SubPortfolioUniverse,
rd::ReturnsResult, pr::AbstractPriorResult,
fees::Option{<:Fees}, wi::MatNum, resi::VecOpt)
predict_outer_returns(cv::CombinatorialCrossValidation, opt,
u::SubPortfolioUniverse, rd::ReturnsResult,
pr::AbstractPriorResult, fees::Option{<:Fees}, wi::MatNum,
resi::VecOpt)Predict the returns of the sub-portfolios of a meta-optimiser as the ReturnsResult of the outer problem.
One module serves every meta-optimiser that owns an outer optimiser. The sub-portfolio enumeration is the only part that changes between them, and it arrives as u. NestedClustered passes a ClusterUniverse and Stacking a FullUniverse. No method reads the type of the meta-optimiser.
The methods dispatch on cv, which selects the prediction, so a custom cross-validation scheme is a method on the first argument:
- Without folds, the column of each sub-portfolio is the net return of its own solve in
resi, on the Prior Result and the Fees viewed onto it withsub_portfolio_view. - A scheme of
OptCVE_Onlpredicts each sub-portfolio out of sample and stacks the folds withrebuild_returns_result. An Online Scheme steps each sub-portfolio from fold to fold and does not refit it. - A
CombinatorialCrossValidationdoes the same, and first thescorerof the meta-optimiser selects one path from the population of each sub-portfolio. Whenscorerisnothing,NearestQuantilePredictionselects it.
wi holds the weights of the inner optimisers alone. A Combination Weight acts at the combination after the outer solve, so no method applies it here (see combination_weights).
Mathematical definition
Without folds:
\[\begin{align} R_{tk} &= \boldsymbol{x}_t^\intercal \mathbf{W}_{\cdot k}\,. \end{align}\]
Where:
- $R_{tk}$: Return of synthetic asset $k$ at observation $t$, before Fees.
- $\boldsymbol{x}_t$: Asset returns for observation $t$, the $t$-th row of the returns matrix.
- $\mathbf{W}$: Inner weight matrix
assets × sub-portfolios. Column $k$ holds the weights of sub-portfolio $k$ over all $N$ assets, and zero at an asset outside the sub-portfolio.
With Fees, calc_net_returns subtracts the fee of sub-portfolio $k$ from its column.
Algorithm
Without folds:
- View
rdonto the observations the Prior Result answers on withouter_prior_rows. Prepare the returns data of the outer problem from it withprepare_outer_rdunder the ruleopt.pcol, givingnb,B,iv,ivpa,pnland the bufferX. - Take the returns that the sub-portfolios are scored on with
outer_returns_sourceunder the source thatprrecords, givingsrc. It is the Prior Result, or the returns of the Prior Result with the benchmark added back whenprrecordsx_src = ExcessSource(). - For each inner result
resinresi, write its net returns withcalc_net_returnsto columniofX, onsrcand Fees viewed onto sub-portfolioi. - Build the
ReturnsResult, with the asset names_1,_2, … and the factors and timestamps ofrd.
With cross-validation:
- Cross-validate every sub-portfolio with
sub_portfolio_predictionsunder the schemecv, givingpredictions. - On the combinatorial path, select one path of each population with
opt.scorer, or withNearestQuantilePredictionwhen it isnothing. - Stack the predictions with
rebuild_returns_resultunder the ruleopt.pcol.
Arguments
cv: The cross-validation scheme of the meta-optimiser, ornothing.opt: The meta-optimiser. Every method reads itspcolfield, the rule of the panel collapse. The method without folds also reads the source of its prior estimatorpe. The cross-validated methods read itsoptiandexfields, and the combinatorial method reads itsscorerfield.u: Sub-portfolio enumeration.rd: Returns data.pr: Prior Result over the whole universe.fees: Fees over the whole universe.wi: Inner weights,assets × sub-portfolios.resi: The results of the inner optimisations.
Returns
- The
ReturnsResultof the outer problem, one synthetic asset for each sub-portfolio.
Related
PortfolioOptimisers.assert_meta_scorer — Function
assert_meta_scorer(cv::Option{<:OptCVE_Onl},
scorer::Option{<:PredictionCrossValScorer})Refuse a scorer that the cross-validation scheme of a meta-optimiser never reads.
predict_outer_returns reads the scorer on the combinatorial path alone, where it selects one path from the population of each sub-portfolio. Under any other scheme, and with no scheme, a scorer has no effect. So the constructors of NestedClustered and Stacking refuse it, and the user learns at construction that the configuration does nothing.
Validation
isnothing(scorer) || isa(cv, CombinatorialCrossValidation). The function throws anArgumentErrorotherwise.
Arguments
cv: Thecvfield of the meta-optimiser.scorer: Thescorerfield of the meta-optimiser.
Returns
nothing.
Related