Base clustering optimisation

PortfolioOptimisers.HierarchicalResult — Type
struct HierarchicalResult{__T_pr, __T_clr, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk} <: BaseHierarchicalOptimisationResult

Holds the fields that the results of hierarchical risk parity and hierarchical equal risk contribution share.

HierarchicalRiskParityResult and HierarchicalEqualRiskContributionResult each hold it as their first field, hr, as a JuMP result holds a JuMPOptimisationResult as jr. Each of the two adds the risk measures and the scalarisers of its own estimator, and ends in the fallback field fb.

The core holds no fb. The generic factory(res, fb) rebuilds a result from all of its fields but the last, and puts fb in the last field. Each of the two results ends in fb, so that method keeps the core unchanged. The core is not an OptimisationResult, so factory(res, fb) does not take it.

Fields

  • pr: Prior result.
  • clr: Clusters result.
  • wb: Weight bounds.
  • fees: Fees estimator or result.
  • retcode: Optimisation return code.
  • w: Portfolio weights vector assets × 1.
  • imsk: The Investable Mask the optimisation reduced on: true at every asset whose prior moments were finite. It is nothing when every asset was investable, and that sentinel is what skips both the reduction and the expansion. investable_mask derives it once from the full-universe prior result, and the result carries it, because the reduced prior can no longer yield it.

Constructors

HierarchicalResult(;    pr::Option{<:AbstractPriorResult},    clr::Option{<:AbstractClusteringResult},    wb::Option{<:WeightBounds},    fees::Option{<:Fees},    retcode::OptimisationReturnCode,    w::Option{<:VecNum},    imsk::Option{<:BitVector} = nothing) -> HierarchicalResult

Keywords correspond to the struct's fields.

The keyword constructor expands w from the investable assets to the full universe, through expand_investable_weights, and then calls the positional constructor. HierarchicalRiskParity and HierarchicalEqualRiskContribution build their results through the keyword constructor, so the stored w has one entry per asset of the universe, and a non-investable asset holds zero. When imsk is nothing, w is stored as it is given. The positional constructor always stores w as it is given, so it is the constructor for weights that are already expanded.

Related

source
PortfolioOptimisers.HierarchicalRiskParityResult — Type
struct HierarchicalRiskParityResult{__T_hr, __T_fb} <: HierarchicalOptimisationResult

Holds the weights and the fitted state of a hierarchical risk parity optimisation.

HierarchicalRiskParity returns it. It holds the shared HierarchicalResult as hr, and the risk measure and the scalariser of its estimator, with the measure stored resolved.

Every property of hr reads through this type, so res.w, res.pr, res.clr, res.wb, res.fees, res.retcode and res.imsk read as if they were fields of res.

Fields

  • hr: The HierarchicalResult that holds the fields every hierarchical result shares. Each of its properties reads through this result.
  • r: The risk measure the optimisation ran under, or a vector of them, stored resolved. Every Deferred Quantity in it is fitted, and every unstated slot holds the prior's field. A resolved measure is fitted state, not configuration, so it belongs on the Result. Pass it back as expected_risk(res.r, res.w, res.pr; sca = res.sca). The type of r is not a type parameter of the Result, so the Results of two risk measures have one type, and a verb that reads them compiles one time.
  • sca: The scalariser the optimisation ran under, taken from opt.sca. Pass it back as expected_risk(res.r, res.w, res.pr; sca = res.sca) so the reported figure matches the optimised one. The type of sca is not a type parameter, so the Results of two scalarisers have one type, and a verb that reads them compiles one time.
  • fb: The fallback chain that answered this result: the (estimator, result) pair of every attempt optimise made before this one, in the order they ran, or nothing when the estimator it was asked of answered (see FbChain).

Constructors

HierarchicalRiskParityResult(;    hr::HierarchicalResult,    r::BaseRM_VecBaseRM,    sca::Option{<:Scalariser},    fb::Option{<:OptE_Opt_FbChain}) -> HierarchicalRiskParityResult

Keywords correspond to the struct's fields.

Related

source
PortfolioOptimisers.HierarchicalEqualRiskContributionResult — Type
struct HierarchicalEqualRiskContributionResult{__T_hr, __T_fb} <: HierarchicalOptimisationResult

Holds the weights and the fitted state of a hierarchical equal risk contribution optimisation.

HierarchicalEqualRiskContribution returns it. It holds the shared HierarchicalResult as hr, and the two risk measures and the two scalarisers of its estimator, one pair inside the clusters and one pair across them. Both measures are stored resolved. The types of the measures and of the scalarisers are not type parameters of the Result, so the Results of two such estimators have one type, and a verb that reads them compiles one time.

Every property of hr reads through this type, so res.w, res.pr, res.clr, res.wb, res.fees, res.retcode and res.imsk read as if they were fields of res.

Fields

  • hr: The HierarchicalResult that holds the fields every hierarchical result shares. Each of its properties reads through this result.
  • ri: The intra-cluster risk measure the optimisation ran under, or a vector of them, stored resolved.
  • ro: The inter-cluster risk measure the optimisation ran under, or a vector of them, stored resolved.
  • fb: The fallback chain that answered this result: the (estimator, result) pair of every attempt optimise made before this one, in the order they ran, or nothing when the estimator it was asked of answered (see FbChain).

Constructors

HierarchicalEqualRiskContributionResult(;    hr::HierarchicalResult,    ri::BaseRM_VecBaseRM,    ro::BaseRM_VecBaseRM,    scai::Option{<:Scalariser},    scao::Option{<:Scalariser},    fb::Option{<:OptE_Opt_FbChain}) -> HierarchicalEqualRiskContributionResult

Keywords correspond to the struct's fields.

Related

source
PortfolioOptimisers.HierarchicalOptimiser — Type
struct HierarchicalOptimiser{__T_pe, __T_cle, __T_slv, __T_wb, __T_fees, __T_sets, __T_wf, __T_strict, __T_cache} <: BaseClusteringOptimisationEstimator

Holds the prior, the clustering, the bounds and the fees that a hierarchical optimiser shares with its siblings.

HierarchicalRiskParity, HierarchicalEqualRiskContribution and SchurComplementHierarchicalRiskParity each hold one in their field opt. Each of them uses these fields to select the returns, fit the prior, cluster the assets, resolve the fees and the weight bounds, and finalise the weights.

Fields

  • pe: Prior estimator.
  • cle: Clusters estimator.
  • slv: Solver or vector of solvers.
  • wb: Weight bounds.
  • fees: Fees estimator or result.
  • sets: Sets used to map estimator values to assets.
  • wf: Weight finaliser.
  • strict: Whether a constraint that names an asset or a group that sets does not hold raises an error. When it is false, the constraint generator warns and skips the entry. Under true the fit also refuses an asset with a gap in the rows that a plain graph reader reads, which it otherwise drops with a warning, see graph_reader_mask. It also refuses an asset that a distance reader of the fit cannot estimate, which it otherwise drops with a warning, see distance_reader_refusal_mask.
  • cache: Optional ReturnsBufferState, the fold context of the online step. It is nothing until partial_fit! writes one, and optimise(opt) with no returns reads it. The returns themselves are carried by the prior, which owns the rows once; this holds every other column of the returns data and the context pinned at the first step. factory carries it unchanged and port_opt_view slices it to the selected assets.

Constructors

HierarchicalOptimiser(;    pe::Onl{<:TD{<:PrE_Pr}} = EmpiricalPrior(),    cle::TD{<:HClE_HCl} = ClustersEstimator(),    slv::Option{<:Slv_VecSlv} = nothing,    wb::TD_Option{<:WbE_Wb} = WeightBounds(),    fees::TD_Option{<:FeesE_Fees} = nothing,    sets::TD_Option{<:UniverseSets} = nothing,    wf::TD{<:WeightFinaliser} = IterativeWeightFinaliser(),    strict::Bool = false,    cache::Option{<:ReturnsBufferState} = nothing) -> HierarchicalOptimiser

Keywords correspond to the struct's fields. A field typed TD or TD_Option can hold a TimeDependent schedule, one value per fold, in place of a static value. A cross-validation fold loop resolves the schedule on each fold, and a fold-less optimise runs with the field at its static default. These fields are the problem definition: the prior estimator, the clustering estimator, the weight bounds, the fees, the asset sets and the weight finaliser. The fields slv and strict control the execution, and they are always static.

Validation

  • If wb is keyed by name (name_keyed): !isnothing(sets). A ScoreWeightBounds with a number or a vector of numbers in val names no asset and needs no sets.
  • If a field holds a TimeDependent schedule: the constructor runs again with each value of the schedule in that field, so a value that the constructor refuses raises here and not in a later fold.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • fees: Recursively updated through factory.
  • cache: Carried unchanged through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Observation weight parameters

obs_weights_view indexes these fields to the selected observations:

Examples

julia> HierarchicalOptimiser()HierarchicalOptimiser      pe ┼ EmpiricalPrior         │           ce ┼ PortfolioOptimisersCovariance         │              │   ce ┼ Covariance         │              │      │    me ┼ SimpleExpectedReturns         │              │      │       │   w ┴ nothing         │              │      │    ce ┼ GeneralCovariance         │              │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)         │              │      │       │    w ┴ nothing         │              │      │   alg ┼ FullMoment()         │              │      │     w ┴ nothing         │              │   mp ┼ MatrixProcessing         │              │      │     pdm ┼ Posdef         │              │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton         │              │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()         │              │      │      dn ┼ nothing         │              │      │      dt ┼ nothing         │              │      │     alg ┼ nothing         │              │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)         │           me ┼ SimpleExpectedReturns         │              │   w ┴ nothing         │      horizon ┼ nothing         │   fill_limit ┼ nothing         │        x_src ┴ nothing     cle ┼ ClustersEstimator         │      ce ┼ PortfolioOptimisersCovariance         │         │   ce ┼ Covariance         │         │      │    me ┼ SimpleExpectedReturns         │         │      │       │   w ┴ nothing         │         │      │    ce ┼ GeneralCovariance         │         │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)         │         │      │       │    w ┴ nothing         │         │      │   alg ┼ FullMoment()         │         │      │     w ┴ nothing         │         │   mp ┼ MatrixProcessing         │         │      │     pdm ┼ Posdef         │         │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton         │         │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()         │         │      │      dn ┼ nothing         │         │      │      dt ┼ nothing         │         │      │     alg ┼ nothing         │         │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)         │      de ┼ Distance         │         │   power ┼ nothing         │         │     alg ┴ CanonicalDistance()         │     alg ┼ HClustAlgorithm         │         │   linkage ┴ Symbol: :ward         │     onc ┼ OptimalNumberClusters         │         │   max_k ┼ nothing         │         │     alg ┼ SecondOrderDifference         │         │         │   alg ┼ StandardisedValue         │         │         │       │   mv ┼ MeanValue         │         │         │       │      │   w ┴ nothing         │         │         │       │   sv ┼ StdValue         │         │         │       │      │           w ┼ nothing         │         │         │       │      │   corrected ┴ Bool: true         │   x_src ┴ nothing     slv ┼ nothing      wb ┼ WeightBounds         │   lb ┼ Float64: 0.0         │   ub ┴ Float64: 1.0    fees ┼ nothing    sets ┼ nothing      wf ┼ IterativeWeightFinaliser         │   iter ┴ Int64: 100  strict ┴ Bool: false

Related

source
PortfolioOptimisers.factory — Method
factory(a, args...; kwargs...) -> a
factory(a::AbstractArray{<:Union{Nothing, AbstractEstimator, AbstractAlgorithm,
                                 AbstractResult, StatsBase.CovarianceEstimator}},
        args...; kwargs...) -> AbstractArray

Rebuild an estimator, an algorithm or a result with runtime values. These two methods are the leaf and the array step of the traversal.

factory and port_opt_view are the two propagation mechanisms of this library. factory passes runtime values, such as prior moments, observation weights and previous portfolio weights, down a tree of composed structs. port_opt_view passes an index selection, a subset of the assets, down the same tree.

factory is a traversal, so it owns the identity on every leaf, as fmap in Functors.jl and adapt_structure in Adapt.jl do. A value that has no method of its own comes back with no change: a data field, a scalar, nothing, a set of observation weights and a covariance estimator of StatsBase. So a @propagatable struct sends each @fprop field to factory directly, whatever the field holds.

The array method is the one forwarding contract for every propagation field that holds an array of estimators, algorithms, results or covariance estimators. It applies factory to each element, and it forwards args... and kwargs... with no change. A family that accepts an array of them needs no method of its own. A family that needs more than the forward defines its own, more specific method. A concrete element type, through concrete_typed_array_if_abstract, is an example. An array of data, such as a vector of numbers, is a leaf, and it comes back with no change and with no copy.

Algorithm

The leaf method:

  1. Return a with no change, and drop args... and kwargs.... This method is the leaf of the recursion. Because of it, a call of the verb on a type with no tags returns the value and raises no error.

The array method:

  1. For each element ai of a, call factory on ai, and forward args... and kwargs... with no change.
  2. Collect the results into a new array of the shape of a, in the order of a, and return it.

A @propagatable struct always has a generated method that is more specific than the leaf method. If the struct has at least one @fprop or @wprop field, that method builds the struct again with its keyword constructor. It sends each @fprop field through factory and each @wprop field through _wprop. If the struct has no such field, that method returns x.

The leaf method does not fill a nothing field. A weights field that must take the incoming weights when it holds nothing carries @wprop, and it goes to _wprop.

Arguments

  • a: Any value, which the leaf method returns with no change. For the array method, an array of estimators, algorithms, results, covariance estimators or nothing, which it rebuilds one element at a time.
  • args...: Positional arguments. The leaf method ignores them, and the array method forwards them.
  • kwargs...: Keyword arguments. The leaf method ignores them, and the array method forwards them.

Returns

  • a: The input with no change, for the leaf method.
  • v::AbstractArray: The rebuilt elements, in an array of the shape of a, for the array method.

Examples

julia> factory(nothing, 1, 2; x = 3)julia> factory([1.0, 2.0], 1, 2; x = 3)2-element Vector{Float64}: 1.0 2.0julia> factory(MeanValue())MeanValue  w ┴ nothing

Related

source
factory(res::NonFiniteAllocationOptimisationResult, fb::Option{<:OptE_Opt_FbChain})

Rebuilds a continuous optimisation result with a new fallback record fb.

A concrete result can add its own method when a rebuild needs more than a new fb. optimise is the one caller, and it gives the FbChain of the attempts that failed.

Algorithm

  1. Read every field of res, in the order of the struct.
  2. Call the constructor of the type of res with every field except the last, and with fb as the last field.

Validation

  • fb is the last field of the type of res. Every concrete optimisation result obeys this rule, so the method checks nothing.

Related

source
factory(
    opt::Union{NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult},
    _
) -> RelaxedRiskBudgeting{JuMPOptimiser{__T_pe, __T_slv, __T_cbe, __T_bgt, __T_fees, __T_ret, __T_sca, __T_sc, __T_so, __T_strict, __T_cache}, _A, _B, <:RelaxedRiskBudgetingAlgorithm} where {__T_pe, __T_slv, __T_cbe, __T_bgt, __T_fees, __T_ret, __T_sca, __T_sc, __T_so, __T_strict, __T_cache, _A, _B}

Returns opt unchanged.

This is the fallback factory of an optimisation estimator or result. An estimator whose parameters change at each step of an optimisation adds its own method. A precomputed result in a field of an estimator, for example in fb, reaches this method when factory gives the estimator the weights of the previous fold. A precomputed result does not change.

Related

source
PortfolioOptimisers.factory — Method
factory(a, args...; kwargs...) -> a
factory(a::AbstractArray{<:Union{Nothing, AbstractEstimator, AbstractAlgorithm,
                                 AbstractResult, StatsBase.CovarianceEstimator}},
        args...; kwargs...) -> AbstractArray

Rebuild an estimator, an algorithm or a result with runtime values. These two methods are the leaf and the array step of the traversal.

factory and port_opt_view are the two propagation mechanisms of this library. factory passes runtime values, such as prior moments, observation weights and previous portfolio weights, down a tree of composed structs. port_opt_view passes an index selection, a subset of the assets, down the same tree.

factory is a traversal, so it owns the identity on every leaf, as fmap in Functors.jl and adapt_structure in Adapt.jl do. A value that has no method of its own comes back with no change: a data field, a scalar, nothing, a set of observation weights and a covariance estimator of StatsBase. So a @propagatable struct sends each @fprop field to factory directly, whatever the field holds.

The array method is the one forwarding contract for every propagation field that holds an array of estimators, algorithms, results or covariance estimators. It applies factory to each element, and it forwards args... and kwargs... with no change. A family that accepts an array of them needs no method of its own. A family that needs more than the forward defines its own, more specific method. A concrete element type, through concrete_typed_array_if_abstract, is an example. An array of data, such as a vector of numbers, is a leaf, and it comes back with no change and with no copy.

Algorithm

The leaf method:

  1. Return a with no change, and drop args... and kwargs.... This method is the leaf of the recursion. Because of it, a call of the verb on a type with no tags returns the value and raises no error.

The array method:

  1. For each element ai of a, call factory on ai, and forward args... and kwargs... with no change.
  2. Collect the results into a new array of the shape of a, in the order of a, and return it.

A @propagatable struct always has a generated method that is more specific than the leaf method. If the struct has at least one @fprop or @wprop field, that method builds the struct again with its keyword constructor. It sends each @fprop field through factory and each @wprop field through _wprop. If the struct has no such field, that method returns x.

The leaf method does not fill a nothing field. A weights field that must take the incoming weights when it holds nothing carries @wprop, and it goes to _wprop.

Arguments

  • a: Any value, which the leaf method returns with no change. For the array method, an array of estimators, algorithms, results, covariance estimators or nothing, which it rebuilds one element at a time.
  • args...: Positional arguments. The leaf method ignores them, and the array method forwards them.
  • kwargs...: Keyword arguments. The leaf method ignores them, and the array method forwards them.

Returns

  • a: The input with no change, for the leaf method.
  • v::AbstractArray: The rebuilt elements, in an array of the shape of a, for the array method.

Examples

julia> factory(nothing, 1, 2; x = 3)julia> factory([1.0, 2.0], 1, 2; x = 3)2-element Vector{Float64}: 1.0 2.0julia> factory(MeanValue())MeanValue  w ┴ nothing

Related

source
factory(res::NonFiniteAllocationOptimisationResult, fb::Option{<:OptE_Opt_FbChain})

Rebuilds a continuous optimisation result with a new fallback record fb.

A concrete result can add its own method when a rebuild needs more than a new fb. optimise is the one caller, and it gives the FbChain of the attempts that failed.

Algorithm

  1. Read every field of res, in the order of the struct.
  2. Call the constructor of the type of res with every field except the last, and with fb as the last field.

Validation

  • fb is the last field of the type of res. Every concrete optimisation result obeys this rule, so the method checks nothing.

Related

source
factory(
    opt::Union{NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult},
    _
) -> RelaxedRiskBudgeting{JuMPOptimiser{__T_pe, __T_slv, __T_cbe, __T_bgt, __T_fees, __T_ret, __T_sca, __T_sc, __T_so, __T_strict, __T_cache}, _A, _B, <:RelaxedRiskBudgetingAlgorithm} where {__T_pe, __T_slv, __T_cbe, __T_bgt, __T_fees, __T_ret, __T_sca, __T_sc, __T_so, __T_strict, __T_cache, _A, _B}

Returns opt unchanged.

This is the fallback factory of an optimisation estimator or result. An estimator whose parameters change at each step of an optimisation adds its own method. A precomputed result in a field of an estimator, for example in fb, reaches this method when factory gives the estimator the weights of the previous fold. A precomputed result does not change.

Related

source
PortfolioOptimisers.BaseHierarchicalOptimisationResult — Type
abstract type BaseHierarchicalOptimisationResult <: AbstractResult

Abstract supertype for the core field block that two hierarchical optimisation results embed.

The type is not in the tree of optimisation results, as BaseJuMPOptimisationResult is not on the JuMP side. optimise does not return a core, so a core must not match a method whose bound is the family of results, for example factory(res::NonFiniteAllocationOptimisationResult, fb).

Its one subtype is HierarchicalResult, which each of the two results embeds as hr.

Interfaces

A subtype is a block of fields, not a result. It declares no method, and it must not get a method whose bound is the family of optimisation results.

Related

source
PortfolioOptimisers.HierarchicalOptimisationResult — Type
abstract type HierarchicalOptimisationResult <: NonJuMPOptimisationResult

Abstract supertype for the results of the estimators that hold a hierarchical optimiser in their field opt.

The field holds a HierarchicalOptimiser. The members are exactly the results of HierarchicalRiskParity, HierarchicalEqualRiskContribution and SchurComplementHierarchicalRiskParity. NestedClustered holds no HierarchicalOptimiser, and its result is not a member.

The family is not called ClusteringOptimisationResult. ClusteringOptimisationEstimator has four subtypes, and one of them is NestedClustered, so that name states a set that this family does not hold.

Two of the three members embed HierarchicalResult as hr. SchurComplementHierarchicalRiskParityResult holds its fields directly. So each member that embeds hr forwards its own properties, and this family forwards none.

Interfaces

The family adds no method to NonJuMPOptimisationResult. A member that embeds HierarchicalResult declares its own property forwarding, so that the properties w and retcode that OptimisationResult needs resolve through hr.

Related

source
PortfolioOptimisers.needs_previous_weights — Method
needs_previous_weights(opt::HierarchicalOptimiser) -> Any

Return true when a HierarchicalOptimiser reads the weights of the previous fold.

The optimiser reads them when its fees do, as a turnover fee does, or when one of its TimeDependent schedules does.

Arguments

  • opt: The shared configuration of a hierarchical optimiser.

Returns

  • flag::Bool: true when a fold loop must pass the previous weights through factory.

Related

source