Subset resampling

PortfolioOptimisers.SubsetResamplingResult — Type
struct SubsetResamplingResult{__T_pr, __T_wb, __T_fees, __T_ress, __T_idx, __T_retcode, __T_w, __T_imsk, __T_fb} <: NonJuMPOptimisationResult

Result of a SubsetResampling optimisation.

ress[m] is the result of the subset whose asset indices are idx[:, m]. w is the average of the subset weights over all the assets, after the weight finaliser. The average is the last step of Algorithm 1 of [4].

idx, pr, wb and fees are on the universe that the subsets come from. When imsk is not nothing, this universe is the investable universe. w is on the full asset universe.

Fields

  • pr: Prior result.
  • wb: Weight bounds.
  • fees: Fees estimator or result.
  • ress: Optimisation result of each asset subset, one entry for each subset, in the column order of idx.
  • idx: Asset indices of the subsets, one column for each subset and one row for each asset of a subset. When imsk is not nothing, the indices are into the investable universe.
  • retcode: Optimisation return code.
  • w: Averaged and finalised portfolio weights over the full asset universe, or one vector for each point of an efficient frontier.
  • imsk: The Investable Mask the optimisation reduced on: true at every asset whose prior moments were finite. It is nothing when every asset was investable, and that sentinel is what skips both the reduction and the expansion. investable_mask derives it once from the full-universe prior result, and the result carries it, because the reduced prior can no longer yield it.
  • fb: The fallback chain that answered this result: the (estimator, result) pair of every attempt optimise made before this one, in the order they ran, or nothing when the estimator it was asked of answered (see FbChain).

Constructors

SubsetResamplingResult(;    pr::Option{<:AbstractPriorResult},    wb::Option{<:WeightBounds},    fees::Option{<:Fees},    ress::AbstractVector{<:NonFiniteAllocationOptimisationResult},    idx::MatNum,    retcode::OptRetCode_VecOptRetCode,    w::VecNum_VecVecNum,    imsk::Option{<:BitVector} = nothing,    fb::Option{<:OptE_Opt_FbChain}) -> SubsetResamplingResult

Keywords correspond to the struct's fields. The keyword constructor expands w onto the full asset universe with expand_investable_weights, and _optimise calls it. The positional constructor does not expand w. set_retcode and factory call it, because their w is already on the full asset universe.

Related

References

  • [4] W. Shen and J. Wang. Portfolio selection via subset resampling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (2017); pp. 1517–1523. Algorithm 1.
source
PortfolioOptimisers.PowerSubsetSize — Type
struct PowerSubsetSize{__T_p} <: SubsetSizeEstimator

Sizes each asset subset as a power of the number of assets, the subset size of [4].

An instance is the default subset_size of SubsetResampling. A call of the instance on a prior result or a returns result gives the subset size.

Mathematical definition

\[\begin{align} k &= \mathrm{round}\left(N^{p}\right)\,. \end{align}\]

Where:

  • $k$: Number of assets in each subset.
  • $p$: Exponent of the number of assets, p.
  • $N$: Number of assets.

$\mathrm{round}$ gives the nearest integer, and the even integer at a tie. When $N \geq 1$ and $0 < p \leq 1$, $1 \leq N^{p} \leq N$, so $1 \leq k \leq N$. So $p = 0.8$ on $N = 100$ assets gives $k = 40$, and $p = 0.7$ gives $k = 25$.

Shen and Wang [4] state the subset size as $b = n^{p}$ and give no rounding rule. The simulation study uses $p = 0.8$, and the empirical study uses $p = 0.7$.

Fields

  • p: The exponent $p$ of the number of assets.

Constructors

PowerSubsetSize(; p::Real = 0.8) -> PowerSubsetSize

Keywords correspond to the struct's fields. The default is the exponent of the simulation study of [4].

Validation

  • 0 < p <= 1. A DomainError is thrown otherwise.

Examples

julia> PowerSubsetSize()PowerSubsetSize  p ┴ Float64: 0.8

Related

References

  • [4] W. Shen and J. Wang. Portfolio selection via subset resampling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (2017); pp. 1517–1523. Page 1520, Figure 1 on page 1521, and the note of Table 2 on page 1522.
source
PortfolioOptimisers.SubsetResampling — Type
struct SubsetResampling{__T_pe, __T_wb, __T_fees, __T_sets, __T_opt, __T_wf, __T_ex, __T_subset_size, __T_n_subsets, __T_max_comb, __T_rng, __T_seed, __T_fb, __T_strict, __T_cache} <: BaseSubsetResamplingOptimisationEstimator

Averages the weights of one base optimiser over random subsets of the assets.

The estimator draws n_subsets distinct subsets of subset_size assets, solves opt on each subset, and averages the subset weights. The average then goes through the weight finaliser wf under the bounds wb. Algorithm 1 of [4] states the method. It solves the mean-variance problem on each subset, and this type takes any base optimiser. It samples the assets of each subset without replacement, and it does not require the subsets to differ. This type draws distinct subsets. When the number of possible subsets is larger than max_comb, the draw is approximate, and two subsets can be equal.

Mathematical definition

Draw $M$ distinct subsets $S_1, \ldots, S_M$ of $k$ assets each from the $N$ assets. Solve the base optimiser on each subset, and average the subset weights over all the assets:

\[\begin{align} k &= \begin{cases} \mathrm{subset\_size} & \text{if } \mathrm{subset\_size} \in \mathbb{Z}\,, \\ \max\!\left(\mathrm{round}(s N),\, 1\right) & \text{if } s = \mathrm{subset\_size} \in (0, 1)\,, \\ f(\mathcal{P}) & \text{if } f = \mathrm{subset\_size} \text{ is callable}\,, \end{cases} \\ \boldsymbol{w}^* &= \frac{1}{M} \sum_{m=1}^{M} \boldsymbol{e}_{S_m}(\boldsymbol{w}_{S_m})\,, \\ \left[\boldsymbol{e}_{S}(\boldsymbol{v})\right]_i &= \begin{cases} v_j & \text{if } i \text{ is the } j\text{-th asset of } S\,, \\ 0 & \text{if } i \notin S\,. \end{cases} \end{align}\]

Where:

  • $\boldsymbol{w}^*$: Averaged portfolio weights, before the weight finaliser.
  • $M$: Number of subsets, n_subsets. A callable n_subsets gives its value on $\mathcal{P}$.
  • $k$: Number of assets in each subset.
  • $s$: Fractional subset_size.
  • $f$: Callable subset_size.
  • $\mathcal{P}$: Prior result of pe on the investable universe.
  • $S_m$: The $m$-th subset, with its assets in ascending order.
  • $\boldsymbol{w}_{S_m}$: Weights that opt gives the assets of $S_m$.
  • $\boldsymbol{e}_{S}(\cdot)$: Map that puts the weights of a subset $S$ at the assets of $S$, and zero at every other asset.
  • $\boldsymbol{v}$: Weight vector over the assets of a subset $S$, with entries $v_j$.
  • $i$: Index of an asset of the $N$ assets.
  • $N$: Number of assets.

The subsets are distinct, so $M \leq \binom{N}{k}$. When each $\boldsymbol{w}_{S_m}$ sums to one, $\boldsymbol{w}^*$ sums to one. An asset that is in no subset has zero weight.

$\mathrm{round}$ gives the nearest integer, and the even integer at a tie. So $s = 0.8$ on $N = 12$ assets gives $k = 10$, and $s = 0.5$ on $N = 5$ assets gives $k = 2$.

Fields

  • pe: Prior estimator.
  • wb: Weight bounds estimator or weight bounds.
  • fees: Fees estimator.
  • sets: Sets used to map estimator values to assets.
  • opt: Base portfolio optimiser applied to each asset subset.
  • wf: Weight finaliser.
  • ex: Parallel execution strategy.
  • subset_size: Size of each random subset.
  • n_subsets: Number of random subsets.
  • max_comb: Largest number of possible asset subsets for which the draw is exact. When binomial(N, subset_size) is larger, the draw takes each subset alone, and two subsets can be equal.
  • rng: Random number generator.
  • seed: Optional seed. If set, the draws come from a copy of rng seeded with it, so rng does not advance and every call draws the same values. If nothing, the draws come from rng itself.
  • fb: Fallback result or estimator.
  • strict: Whether a constraint that names an asset or a group that sets does not hold raises an error. When it is false, the constraint generator warns and skips the entry.
  • cache: Optional ReturnsBufferState, the fold context of the online step. It is nothing until partial_fit! writes one, and optimise(opt) with no returns reads it. The returns themselves are carried by the prior, which owns the rows once; this holds every other column of the returns data and the context pinned at the first step. factory carries it unchanged and port_opt_view slices it to the selected assets.

Constructors

SubsetResampling(;    pe::Onl{<:TD{<:PrE_Pr}} = EmpiricalPrior(),    wb::TD_Option{<:WbE_Wb} = nothing,    fees::TD_Option{<:FeesE_Fees} = nothing,    sets::TD_Option{<:UniverseSets} = nothing,    opt::OptE_TD,    wf::TD{<:WeightFinaliser} = IterativeWeightFinaliser(),    ex::FLoops.Transducers.Executor = FLoops.ThreadedEx(),    subset_size::TD{<:SubsetSizeE} = PowerSubsetSize(),    n_subsets::TD{<:NumberSubsetsE} = 2,    max_comb::Integer = 1_000_000_000,    rng::Random.AbstractRNG = Random.default_rng(),    seed::Option{<:Integer} = nothing,    fb::TDO_Option{<:OptE_Opt} = nothing,    strict::Bool = false,    cache::Option{<:ReturnsBufferState} = nothing) -> SubsetResampling

Keywords correspond to the struct's fields. The default subset_size is the subset size of the simulation study of [4], $k = \mathrm{round}\left(N^{0.8}\right)$, see PowerSubsetSize.

Time-dependent fields

pe, wb, fees, sets, opt, wf, subset_size, n_subsets and fb can hold a TimeDependent schedule with one entry for each fold. The other fields cannot hold a schedule.

The loop inside SubsetResampling is over asset subsets, not over time folds. No inner fold loop exists for a bind = :nearest schedule in opt or fb, so the constructor refuses one. A fold loop that reaches the estimator resolves each schedule. A solve with no fold loop resets each schedule to its own default. A schedule with no default resets pe, wf, subset_size and n_subsets to their keyword defaults, and wb, fees, sets and fb to nothing. A schedule in opt with no default throws a TimeDependentDefaultError.

Validation

  • opt passes assert_internal_optimiser. A schedule applies the check to each entry and to its default.
  • If wb is keyed by name (name_keyed), sets is not nothing. The constructor throws an IsNothingError otherwise.
  • If fees is a FeesEstimator, sets is not nothing. The constructor throws an IsNothingError otherwise.
  • A plain number in subset_size passes count_or_fraction, and the field then holds a CountOf or a FractionOf.
  • If n_subsets is an Integer: n_subsets >= 2.
  • max_comb > 0 and finite.
  • A schedule in opt or fb has bind !== :nearest. Each entry of a schedule passes the check of its field.
  • A reader that states no source and reads the raw returns data, beside a reader that states x_src = ExcessSource(), logs one warning that names its path, see warn_raw_readers. The head then builds as configured.
A search grid can turn a plain count into a fraction

A field that takes a CountOf or a FractionOf also takes a plain number, and count_or_fraction reads an Integer as a count and any other real number as a fraction. Julia promotes the elements of a vector literal to one type before the library reads them. So ["opt.ret.ucs.alg.method.active" => [3, 5, 8], "r.alpha" => [0.05, 0.1]] is a Vector{Pair{String, Vector{Float64}}}, and the counts arrive as 3.0, 5.0 and 8.0. The field then reads them as fractions, and the check of FractionOf refuses them. State the grid in the wrappers, such as [CountOf(3), CountOf(5), FractionOf(0.2)]. A wrapper keeps its type in every container.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • fees: Recursively updated through factory.
  • opt: Recursively updated through factory.
  • fb: Recursively updated through factory.
  • cache: Carried unchanged through factory.

View parameters

SubsetResampling defines its own port_opt_view method rather than deriving one from field tags.

  • The method reads the returns matrix X as its third argument. When pe is a prior result, the method uses pe.X in place of X.
  • pe, wb, sets and cache recurse through port_opt_view with the index alone.
  • fees and opt recurse with the index and X.
  • fb recurses through view_child with the index and X.

Observation weight parameters

obs_weights_view indexes these fields to the selected observations:

Related

References

  • [4] W. Shen and J. Wang. Portfolio selection via subset resampling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (2017); pp. 1517–1523. Algorithm 1.
source
PortfolioOptimisers.factory — Method
factory(
    sr::SubsetResamplingResult,
    fb::Union{Nothing, NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult, AbstractVector{<:Tuple{OptimisationEstimator, OptimisationResult}}}
) -> SubsetResamplingResult

Rebuild a SubsetResamplingResult with the fallback chain fb.

optimise calls this method when a SubsetResampling fallback gives the answer. The method copies every other field unchanged, and it does not expand sr.w again.

Related

source
PortfolioOptimisers.factory — Method
factory(a, args...; kwargs...) -> a
factory(a::AbstractArray{<:Union{Nothing, AbstractEstimator, AbstractAlgorithm,
                                 AbstractResult, StatsBase.CovarianceEstimator}},
        args...; kwargs...) -> AbstractArray

Rebuild an estimator, an algorithm or a result with runtime values. These two methods are the leaf and the array step of the traversal.

factory and port_opt_view are the two propagation mechanisms of this library. factory passes runtime values, such as prior moments, observation weights and previous portfolio weights, down a tree of composed structs. port_opt_view passes an index selection, a subset of the assets, down the same tree.

factory is a traversal, so it owns the identity on every leaf, as fmap in Functors.jl and adapt_structure in Adapt.jl do. A value that has no method of its own comes back with no change: a data field, a scalar, nothing, a set of observation weights and a covariance estimator of StatsBase. So a @propagatable struct sends each @fprop field to factory directly, whatever the field holds.

The array method is the one forwarding contract for every propagation field that holds an array of estimators, algorithms, results or covariance estimators. It applies factory to each element, and it forwards args... and kwargs... with no change. A family that accepts an array of them needs no method of its own. A family that needs more than the forward defines its own, more specific method. A concrete element type, through concrete_typed_array_if_abstract, is an example. An array of data, such as a vector of numbers, is a leaf, and it comes back with no change and with no copy.

Algorithm

The leaf method:

  1. Return a with no change, and drop args... and kwargs.... This method is the leaf of the recursion. Because of it, a call of the verb on a type with no tags returns the value and raises no error.

The array method:

  1. For each element ai of a, call factory on ai, and forward args... and kwargs... with no change.
  2. Collect the results into a new array of the shape of a, in the order of a, and return it.

A @propagatable struct always has a generated method that is more specific than the leaf method. If the struct has at least one @fprop or @wprop field, that method builds the struct again with its keyword constructor. It sends each @fprop field through factory and each @wprop field through _wprop. If the struct has no such field, that method returns x.

The leaf method does not fill a nothing field. A weights field that must take the incoming weights when it holds nothing carries @wprop, and it goes to _wprop.

Arguments

  • a: Any value, which the leaf method returns with no change. For the array method, an array of estimators, algorithms, results, covariance estimators or nothing, which it rebuilds one element at a time.
  • args...: Positional arguments. The leaf method ignores them, and the array method forwards them.
  • kwargs...: Keyword arguments. The leaf method ignores them, and the array method forwards them.

Returns

  • a: The input with no change, for the leaf method.
  • v::AbstractArray: The rebuilt elements, in an array of the shape of a, for the array method.

Examples

julia> factory(nothing, 1, 2; x = 3)julia> factory([1.0, 2.0], 1, 2; x = 3)2-element Vector{Float64}: 1.0 2.0julia> factory(MeanValue())MeanValue  w ┴ nothing

Related

source
factory(
    opt::Union{NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult},
    _
) -> EqualWeighted{_A, _B, _C, _D, _E, Bool} where {_A, _B, _C, _D, _E}

Returns opt unchanged.

This is the fallback factory of an optimisation estimator or result. An estimator whose parameters change at each step of an optimisation adds its own method. A precomputed result in a field of an estimator, for example in fb, reaches this method when factory gives the estimator the weights of the previous fold. A precomputed result does not change.

Related

source
PortfolioOptimisers.port_opt_view — Method
port_opt_view(
    sr::SubsetResampling,
    i,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    args...
) -> SubsetResampling{_A, _B, _C, _D, _E, _F, var"#s185", _G, _H, var"#s1851", var"#s1852", _I, _J, Bool} where {_A, _B, _C, _D, _E, _F, var"#s185"<:Transducers.Executor, _G, _H, var"#s1851"<:Integer, var"#s1852"<:AbstractRNG, _I, _J}

Return a copy of a SubsetResampling on the assets i.

X is the returns matrix that the views of fees, opt and fb read. When sr.pe is a prior result, the method uses sr.pe.X in place of X. The ## View parameters subsection of SubsetResampling lists what the method does to each field.

Related

source
PortfolioOptimisers._optimise — Method
_optimise(
    sr::SubsetResampling,
    rd::ReturnsResult;
    branchorder,
    str_names,
    save,
    kwargs...
) -> Any

Run the subset resampling optimisation, and return the averaged portfolio.

optimise calls this method. The struct SubsetResampling states the mathematics.

Algorithm

  1. Reset the schedules of sr with reset_time_dependent_estimator.
  2. Fit the prior sr.pe on rd with prior, giving pr, and read its investable mask with investable_mask, giving imsk. The prior reads the returns in the frame that its own x_src states. rd keeps its raw returns, and the readers of the subset optimisers state their own sources.
  3. Resolve the fees on the full asset universe with fees_constraints, and put them on the investable universe with investable_fees_view, giving fees.
  4. Reduce pr, sr and rd to the investable universe with investable_reduction, giving rdr. N is the number of investable assets.
  5. Resolve subset_size and n_subsets on pr with get_subset_size and get_n_subsets.
  6. Draw the subsets with sample_unique_assets, giving asset_idx. When binomial(N, subset_size) is at most max_comb, the subsets are distinct. Otherwise the function draws each subset alone, and two subsets can be equal.
  7. For each column of asset_idx, view sr.opt and rdr on the subset with port_opt_view and optimise the view, giving ress. The loop runs on the executor sr.ex.
  8. Build the weight bounds wb over the $N$ assets with weight_bounds_constraints.
  9. Average and finalise the subset weights with subset_resampling_finaliser, giving retcode and w.
  10. Return a SubsetResamplingResult. Its keyword constructor expands w onto the full asset universe.

Arguments

  • sr: The subset resampling optimiser to use.
  • rd: The returns result to use.
  • branchorder: Branch order, passed to each subset optimisation.
  • str_names: Whether each subset optimisation uses string names for the assets.
  • save: Whether each subset optimisation saves its JuMP model in its result.
  • kwargs: Other keyword arguments, passed to each subset optimisation.

Validation

Returns

  • res::SubsetResamplingResult: The averaged portfolio. retcode is an OptimisationFailure when a subset optimisation fails, or when the weight finaliser fails.

Related

source

References

[4]
W. Shen and J. Wang. Portfolio selection via subset resampling. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (2017); pp. 1517–1523.