GitHub

Base Cross Validation (a)

PortfolioOptimisers.PredictionReturnsResult — Type
struct PredictionReturnsResult{__T_nx, __T_X, __T_nf, __T_F, __T_nb, __T_B, __T_ts, __T_iv, __T_ivpa} <: AbstractReturnsResult

Stores the portfolio return series of a cross-validation prediction, with the data that is aligned to it.

X is the portfolio series, not the asset returns: one vector for a single portfolio, or one vector per member of a population. Beside it the carrier holds the factor returns, the benchmark series, the timestamps, the implied volatilities and the implied volatility risk premium adjustment of the same observations.

The carrier holds no feature matrix. rebuild_returns_result computes the collapse of the outer problem from the original rd.pnl and the weights of each fold, which is the call the path without cross-validation makes. The weights of a fold are on the res of its PredictionResult, and ts is the slice of the original clock that the fold covers, so each fold can be rebuilt from what the result holds.

Fields

  • nx: Asset name vector.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • nf: Factor name vector.
  • F: Data matrix observations × factors if the dims keyword does not exist or dims = 1, factors × observations when dims = 2.
  • nb: Benchmark name vector.
  • B: Benchmark returns.
  • ts: Timestamp vector.
  • iv: Implied volatilities.
  • ivpa: Implied volatility risk premium adjustment.

Constructors

PredictionReturnsResult(;    nx::Option{<:VecStr} = nothing,    X::Option{<:VecNum_VecVecNum} = nothing,    nf::Option{<:VecStr} = nothing,    F::Option{<:MatNum} = nothing,    nb::Option{<:VecStr} = nothing,    B::Option{<:VecNum_VecVecNum} = nothing,    ts::Option{<:VecDate} = nothing,    iv::Option{<:VecNum_VecVecNum} = nothing,    ivpa::Option{<:Num_VecNum} = nothing) -> PredictionReturnsResult

Keywords correspond to the struct's fields.

Validation

  • nf and F must be consistent: both nothing, or F has length(nf) columns.
  • If X and F are given, F has one row per observation of each series in X.
  • If B and X are given, they have the same shape (VecNum or VecVecNum) and matching lengths. A mixed pair raises an ArgumentError.
  • If ts is given, it is not empty, at least one of X and F is not nothing, and its length matches X, F and B where they are given.
  • If iv is given, X is given too, else an IsNothingError is raised.
  • If iv is a VecNum, ivpa is a scalar or nothing, iv is non-empty, non-negative and finite, ivpa is positive and finite, and length(iv) == length(X).
  • If iv is a VecVecNum, ivpa is a VecNum or nothing, length(iv) == length(X), and length(ivpa) == length(X) when ivpa is given. Each entry of ivpa is positive and finite, and each vector of iv is non-empty, non-negative, finite, and as long as its series in X.

Related

source
PortfolioOptimisers.PredictionResult — Type
struct PredictionResult{__T_res, __T_rd, __T_hw} <: AbstractPredictionResult

Stores the result of a single cross-validation fold prediction. It pairs an optimisation result with the portfolio return series of the test period.

The rows of the fold are not stored. rd.ts is the slice of the original clock that the fold covers, because port_opt_view slices it with the test_idx of the fold, so feature_row_indices finds the absolute rows by their timestamps. rebuild_returns_result is the consumer that needs them. The timestamps also stay correct on the combinatorial path, where the folds of a path come in split order and not in time order.

Fields

  • res: Optimisation result from the training fold.
  • rd: The returns result to use.
  • hw: Held-weights record of the fold, or nothing when the fold held its target weights on every observation.

Constructors

PredictionResult(;    res::NonFiniteAllocationOptimisationResult,    rd::PredictionReturnsResult,    hw::Option{<:HeldWeightsResult} = nothing) -> PredictionResult

Keywords correspond to the struct's fields. res and rd are required, because a fold prediction needs both. hw defaults to nothing, which is a fold that held its target weights on every observation.

hw is present only when a Weight Drift or a Previous-Weights Source ran over the fold, and the fold-taking consumers dispatch on its type. It carries the asset returns of the fold, the weights the drift started from, the weights held after the last observation and the drift that made them. weight_path rebuilds the weight path from it.

Related

source
PortfolioOptimisers.MultiPeriodPredictionResult — Type
struct MultiPeriodPredictionResult{__T_pred, __T_mrd, __T_id, __T_opt} <: AbstractPredictionResult

Stores the predictions of the folds of one path, with their return data concatenated into one PredictionReturnsResult.

The quantities with one value per observation (X, F, B, ts, iv) are concatenated over the folds. ivpa has one value per synthetic asset, and the reconstruct_rd of each fold has already collapsed it with the weights of that fold, so it cannot be concatenated. The constructor keeps the value of the last fold. This matches predict_realised_vols, which reads the last row of the concatenated iv, so the premium divisor pairs with the implied volatility it divides.

A feature matrix is not among the concatenated quantities, because the folds do not carry one. rebuild_returns_result computes the outer collapse from the original rd.pnl, and reaches each fold through pred[f].res.w and pred[f].rd.ts. That is why the result keeps pred.

The Result of an online run also carries, in opt, the estimator that the fold loop threaded, folded through the last training end last(train_idx[end]). A batch run writes nothing. Resume hands the Result back to the loop over a longer history, and the loop continues from the fold after the last one held. A hand step in the value form, partial_fit(res.opt, rows), leaves the Result resumable. The bang form, partial_fit!(res.opt, rows), writes the arrays of the state in place, the held timestamps among them. The Result then names a row at which no fold ends, and Resume refuses it. partial_fit! states this contract for a kept estimator.

Fields

  • pred: Collection of fold predictions.
  • mrd: Aggregated multi-period returns result.
  • id: Path or fold identifier.
  • opt: The estimator the online arm of the fold loop threaded, folded through the last training end, or nothing on a batch run. Resume re-enters the loop from it.

Constructors

MultiPeriodPredictionResult(;    pred::VecPredRes,    id::Any = nothing,    opt::Option{<:AbstractEstimator} = nothing) -> MultiPeriodPredictionResult

Keywords correspond to the struct's fields. pred is required. The constructor concatenates the return data of the folds into mrd, and a concatenation of no folds has no names and no clock.

Validation

  • !isempty(pred).

Related

source
PortfolioOptimisers.PopulationPredictionResult — Type
struct PopulationPredictionResult{__T_pred} <: AbstractPredictionResult

Stores the paths of a cross-validation scheme that makes more than one path. Each element of pred is one path: a random asset subset of MultipleRandomised, or a path of CombinatorialCrossValidation. A member is usually a MultiPeriodPredictionResult, and a population built by hand can hold a single-fold PredictionResult.

Fields

  • pred: Collection of fold predictions.

Constructors

PopulationPredictionResult(;    pred::VecPredRes_MultiPredRes = Vector{PredRes_MultiPredRes}(undef, 0)) -> PopulationPredictionResult

Keywords correspond to the struct's fields. An empty pred is admitted: a population from which every path was dropped is a valid, if empty, answer.

Every multi-period member carries an id

A MultiPeriodPredictionResult member whose id is nothing takes its position in pred as its id, through population_ids. The schemes that make a population already number their paths that way, so the rule changes nothing for them. It gives an identifier to a population built by hand from cross_val_predict streams, so the path that NearestQuantilePrediction or sort_by_measure selects names its place in the population. A member that already carries an id keeps it. A single-fold PredictionResult has no id field, and is kept as it is.

Related

source
Base.split — Method
split(
    res::CrossValidationResult,
    args...
) -> CrossValidationResult

Give a split result back unchanged.

A CrossValidationResult holds folds that are already split, so an entry point that calls split(cv, rd) on a scheme reads the same folds when it holds a result in place of the scheme. The data argument is not read.

Arguments

  • res: The split result.
  • args...: The data the scheme would split, which is ignored.

Returns

  • res, the same object.
source
StatsAPI.predict — Method
predict(res::NonFiniteAllocationOptimisationResult, rd::ReturnsResult)
predict(res, rd, test_idx, cols = :)
predict(res, rd, test_idxs::VecVecInt, cols = :)

Apply an optimisation result res to returns data rd to produce a PredictionResult or a vector of prediction results.

When test_idx is given, only the rows of rd that test_idx indexes, and the columns that cols selects, are used for the prediction.

The fee holds no horizon. charge_fees receives the length of the series it charges, so a fold spreads a one-off cost over its own observations and the whole-sample method spreads it over the whole sample. fees.fa names the clock alone: a nothing or FirstObservationFees charges the two fixed terms on the first observation of the series, and an AmortisedFees spreads them evenly over it. The fa keyword overrides that clock for the series this method builds, and changes nothing that the result holds. A report can then charge a fixed fee as a fund saw it, while the optimiser prices the same fee as its own objective needs.

A test window over a point-in-time universe holds two kinds of gap. The column of a non-investable asset has weight 0, and the view at the Investable Mask removes it before anything reads it. A Held Gap is an (observation, asset) pair at which the weight is not zero and the return is missing, which an asset that delists inside the test window makes. The mask comes from the fit and cannot see it. Nothing is renormalised, so the missing weight is held in cash on that observation. Without a drift, the series of the fold is

returns[t] == sum_i w_i * (isfinite(X[t, i]) ? X[t, i] : 0) - fee

with the fee taken over the whole weight vector. Under a drift, w_i is the weight of asset i on the weight path at observation t.

A failed fold carries NaN weights, so its series is NaN and res.w and rd.X show the failure to a scorer. Under a drift the fold still held a book. held_start_weights starts the drift from the previous weights w_prev, member by member under a population, so the next fold reads that book. With no w_prev, which is fold 1 or a scheme whose folds are not a timeline, the record is NaN and nothing throws.

Algorithm

  1. For the index methods, view the rows test_idx and the columns cols of rd with port_opt_view.
  2. View the weights and the window at the Investable Mask of res with investable_fold_view. The fee of the result is on the investable universe already, and is not viewed.
  3. Override the clock of the fee with fa through override_fee_amortisation.
  4. Set every non-finite entry of the window to zero with filter_held_gaps, giving Xf. A Held Gap warns, or raises an ArgumentError under strict, through strict_diagnostic.
  5. Compute the net return series from the weights, Xf, the fee and wd with calc_net_returns.
  6. Choose the weights the drift starts from with held_start_weights, from the return code, the weights and w_prev viewed at the mask.
  7. Drift them over Xf under hwd with held_weights_result, giving the record and the ruined members.
  8. Warn about the ruined members with warn_ruined_members, and mark them failed with mark_ruined_members.
  9. Collapse the data aligned to the series with reconstruct_rd.
  10. Expand the record to the caller's universe with expand_held_weights, because the turnover of the next fold reads it, and build the PredictionResult.

The method over test_idxs runs the method over one test_idx for each fold.

Arguments

  • res::NonFiniteAllocationOptimisationResult: Fitted optimisation result.
  • rd::ReturnsResult: Returns data for the prediction period.
  • test_idx: Observation index or vector of observation indices for the test fold.
  • cols: Column selector. Defaults to : (all assets).

Keyword Arguments

  • wd::Option{<:AbstractWeightDrift} = nothing: The Weight Drift of the series, or nothing for a series formed from the target weights.
  • hwd::Option{<:AbstractWeightDrift} = wd: The drift of the Held Weights record, or nothing for no record. The schemes pass held_weights_drift(wd, pws), so a Previous-Weights Source records the held weights of a series that is not drifted.
  • fa::Option{<:AbstractFeeAmortisation} = nothing: The clock the series charges the two fixed fee terms on, or nothing to inherit the clock the fee itself states.
  • store_weight_path::Bool = false: Whether the record stores the weight path, in place of rebuilding it on demand.
  • strict::Bool = false: Whether a Held Gap raises an ArgumentError rather than warning.
  • w_prev::Option{<:VecNum_VecVecNum} = nothing: The previous weights the fold was handed, which a failed fold holds under a drift, or nothing.

Returns

Related

source
PortfolioOptimisers.sort_by_measure — Function
sort_by_measure(ppred::PopulationPredictionResult, r::BaseRM_VecBaseRM; kwargs...)

Sort the successful paths in a PopulationPredictionResult by their expected risk under r. A path in which any fold returned a failure code is left out.

A path whose measure is not finite goes after every finite one, under both directions of the ranking, because a non-finite number is not a best and must not be the answer of a first. The finite members are sorted among themselves, and the non-finite ones keep their own order at the end.

The direction of the ranking comes from bigger_is_better, which throws on a vector of measures that disagree on polarity. quantile_by_measure takes an explicit sign instead, so it admits a mixed vector.

Algorithm

  1. Keep the successful members of ppred with successful_members.
  2. Compute the expected risk of each member under r, giving rks.
  3. Sort the members whose rks is finite by rks, in descending order when bigger_is_better holds of r and in ascending order otherwise.
  4. Append the members whose rks is not finite, in their own order.

Arguments

  • ppred::PopulationPredictionResult: Population prediction to sort.
  • r::BaseRM_VecBaseRM: Risk measure used for ranking, or a vector of them. A vector is scalarised by kwargs.sca, defaulting to SumScalariser.
  • kwargs...: Keyword arguments forwarded to expected_risk.

Validation

Returns

Related

source
PortfolioOptimisers.previous_weights — Function
previous_weights(pws::Any, prev::Nothing)
previous_weights(pws::Nothing, prev::PredictionResult)
previous_weights(pws::AbstractPreviousWeightsSource, prev::PredictionResult)

Read the weights a fold threads into the fold that follows it.

This is the one function that reads the Previous-Weights Source. The first fold of a run has no fold behind it, so it threads nothing whatever the source is. A later fold threads the target weights of the previous fold by default, and the weights that fold held after its last observation when the scheme sets a source.

prev is the last fold whose weights can be threaded, which is not always the fold before. The sequential loops advance it only when threads_weights holds of a fold, so a failed fold is skipped and the fold before it is read. The weights this function gives are therefore finite whenever it gives any.

Algorithm

  1. With no previous fold, give nothing.
  2. With no source, give the target weights of the previous fold, prev.res.w.
  3. With a source, give the held weights of the previous fold, prev.hw.w.

Arguments

  • pws: Previous-weights source of the scheme, or nothing.
  • prev: Prediction result of the last threadable fold, or nothing.

Returns

  • Option{<:VecNum_VecVecNum}: The weights the next fold reads through factory.

Related

source