Base clustering optimisation
PortfolioOptimisers.HierarchicalResult — Type
struct HierarchicalResult{__T_pr, __T_clr, __T_wb, __T_fees, __T_retcode, __T_w, __T_imsk} <: BaseHierarchicalOptimisationResultHolds the fields that the results of hierarchical risk parity and hierarchical equal risk contribution share.
HierarchicalRiskParityResult and HierarchicalEqualRiskContributionResult each hold it as their first field, hr, as a JuMP result holds a JuMPOptimisationResult as jr. Each of the two adds the risk measures and the scalarisers of its own estimator, and ends in the fallback field fb.
The core holds no fb. The generic factory(res, fb) rebuilds a result from all of its fields but the last, and puts fb in the last field. Each of the two results ends in fb, so that method keeps the core unchanged. The core is not an OptimisationResult, so factory(res, fb) does not take it.
Fields
pr: Prior result.
clr: Clusters result.
wb: Weight bounds.
fees: Fees estimator or result.
retcode: Optimisation return code.
w: Portfolio weights vectorassets × 1.
imsk: The Investable Mask the optimisation reduced on:trueat every asset whose prior moments were finite. It isnothingwhen every asset was investable, and that sentinel is what skips both the reduction and the expansion.investable_maskderives it once from the full-universe prior result, and the result carries it, because the reduced prior can no longer yield it.
Constructors
HierarchicalResult(; pr::Option{<:AbstractPriorResult}, clr::Option{<:AbstractClusteringResult}, wb::Option{<:WeightBounds}, fees::Option{<:Fees}, retcode::OptimisationReturnCode, w::Option{<:VecNum}, imsk::Option{<:BitVector} = nothing) -> HierarchicalResultKeywords correspond to the struct's fields.
The keyword constructor expands w from the investable assets to the full universe, through expand_investable_weights, and then calls the positional constructor. HierarchicalRiskParity and HierarchicalEqualRiskContribution build their results through the keyword constructor, so the stored w has one entry per asset of the universe, and a non-investable asset holds zero. When imsk is nothing, w is stored as it is given. The positional constructor always stores w as it is given, so it is the constructor for weights that are already expanded.
Related
PortfolioOptimisers.HierarchicalRiskParityResult — Type
struct HierarchicalRiskParityResult{__T_hr, __T_fb} <: HierarchicalOptimisationResultHolds the weights and the fitted state of a hierarchical risk parity optimisation.
HierarchicalRiskParity returns it. It holds the shared HierarchicalResult as hr, and the risk measure and the scalariser of its estimator, with the measure stored resolved.
Every property of hr reads through this type, so res.w, res.pr, res.clr, res.wb, res.fees, res.retcode and res.imsk read as if they were fields of res.
Fields
hr: TheHierarchicalResultthat holds the fields every hierarchical result shares. Each of its properties reads through this result.
r: The risk measure the optimisation ran under, or a vector of them, stored resolved. Every Deferred Quantity in it is fitted, and every unstated slot holds the prior's field. A resolved measure is fitted state, not configuration, so it belongs on the Result. Pass it back asexpected_risk(res.r, res.w, res.pr; sca = res.sca). The type ofris not a type parameter of the Result, so the Results of two risk measures have one type, and a verb that reads them compiles one time.
sca: The scalariser the optimisation ran under, taken fromopt.sca. Pass it back asexpected_risk(res.r, res.w, res.pr; sca = res.sca)so the reported figure matches the optimised one. The type ofscais not a type parameter, so the Results of two scalarisers have one type, and a verb that reads them compiles one time.
fb: The fallback chain that answered this result: the(estimator, result)pair of every attemptoptimisemade before this one, in the order they ran, ornothingwhen the estimator it was asked of answered (seeFbChain).
Constructors
HierarchicalRiskParityResult(; hr::HierarchicalResult, r::BaseRM_VecBaseRM, sca::Option{<:Scalariser}, fb::Option{<:OptE_Opt_FbChain}) -> HierarchicalRiskParityResultKeywords correspond to the struct's fields.
Related
PortfolioOptimisers.HierarchicalEqualRiskContributionResult — Type
struct HierarchicalEqualRiskContributionResult{__T_hr, __T_fb} <: HierarchicalOptimisationResultHolds the weights and the fitted state of a hierarchical equal risk contribution optimisation.
HierarchicalEqualRiskContribution returns it. It holds the shared HierarchicalResult as hr, and the two risk measures and the two scalarisers of its estimator, one pair inside the clusters and one pair across them. Both measures are stored resolved. The types of the measures and of the scalarisers are not type parameters of the Result, so the Results of two such estimators have one type, and a verb that reads them compiles one time.
Every property of hr reads through this type, so res.w, res.pr, res.clr, res.wb, res.fees, res.retcode and res.imsk read as if they were fields of res.
Fields
hr: TheHierarchicalResultthat holds the fields every hierarchical result shares. Each of its properties reads through this result.
ri: The intra-cluster risk measure the optimisation ran under, or a vector of them, stored resolved.
ro: The inter-cluster risk measure the optimisation ran under, or a vector of them, stored resolved.
scai: Inner scalariser.nothinggives the sum ofSumScalariser.
scao: Outer scalariser.nothinggives the sum ofSumScalariser.
fb: The fallback chain that answered this result: the(estimator, result)pair of every attemptoptimisemade before this one, in the order they ran, ornothingwhen the estimator it was asked of answered (seeFbChain).
Constructors
HierarchicalEqualRiskContributionResult(; hr::HierarchicalResult, ri::BaseRM_VecBaseRM, ro::BaseRM_VecBaseRM, scai::Option{<:Scalariser}, scao::Option{<:Scalariser}, fb::Option{<:OptE_Opt_FbChain}) -> HierarchicalEqualRiskContributionResultKeywords correspond to the struct's fields.
Related
PortfolioOptimisers.HierarchicalOptimiser — Type
struct HierarchicalOptimiser{__T_pe, __T_cle, __T_slv, __T_wb, __T_fees, __T_sets, __T_wf, __T_strict, __T_cache} <: BaseClusteringOptimisationEstimatorHolds the prior, the clustering, the bounds and the fees that a hierarchical optimiser shares with its siblings.
HierarchicalRiskParity, HierarchicalEqualRiskContribution and SchurComplementHierarchicalRiskParity each hold one in their field opt. Each of them uses these fields to select the returns, fit the prior, cluster the assets, resolve the fees and the weight bounds, and finalise the weights.
Fields
pe: Prior estimator.
cle: Clusters estimator.
slv: Solver or vector of solvers.
wb: Weight bounds.
fees: Fees estimator or result.
sets: Sets used to map estimator values to assets.
wf: Weight finaliser.
strict: Whether a constraint that names an asset or a group thatsetsdoes not hold raises an error. When it isfalse, the constraint generator warns and skips the entry. Undertruethe fit also refuses an asset with a gap in the rows that a plain graph reader reads, which it otherwise drops with a warning, seegraph_reader_mask. It also refuses an asset that a distance reader of the fit cannot estimate, which it otherwise drops with a warning, seedistance_reader_refusal_mask.
cache: OptionalReturnsBufferState, the fold context of the online step. It isnothinguntilpartial_fit!writes one, andoptimise(opt)with no returns reads it. The returns themselves are carried by the prior, which owns the rows once; this holds every other column of the returns data and the context pinned at the first step.factorycarries it unchanged andport_opt_viewslices it to the selected assets.
Constructors
HierarchicalOptimiser(; pe::Onl{<:TD{<:PrE_Pr}} = EmpiricalPrior(), cle::TD{<:HClE_HCl} = ClustersEstimator(), slv::Option{<:Slv_VecSlv} = nothing, wb::TD_Option{<:WbE_Wb} = WeightBounds(), fees::TD_Option{<:FeesE_Fees} = nothing, sets::TD_Option{<:UniverseSets} = nothing, wf::TD{<:WeightFinaliser} = IterativeWeightFinaliser(), strict::Bool = false, cache::Option{<:ReturnsBufferState} = nothing) -> HierarchicalOptimiserKeywords correspond to the struct's fields. A field typed TD or TD_Option can hold a TimeDependent schedule, one value per fold, in place of a static value. A cross-validation fold loop resolves the schedule on each fold, and a fold-less optimise runs with the field at its static default. These fields are the problem definition: the prior estimator, the clustering estimator, the weight bounds, the fees, the asset sets and the weight finaliser. The fields slv and strict control the execution, and they are always static.
Validation
- If
wbis keyed by name (name_keyed):!isnothing(sets). AScoreWeightBoundswith a number or a vector of numbers invalnames no asset and needs no sets. - If a field holds a
TimeDependentschedule: the constructor runs again with each value of the schedule in that field, so a value that the constructor refuses raises here and not in a later fold.
Propagated parameters
factory passes its arguments on to these @fprop fields:
View parameters
port_opt_view takes the view of these @vprop fields at the selected indices:
pe: Recursively viewed throughport_opt_view.wb: Recursively viewed throughport_opt_view.fees: Recursively viewed throughport_opt_view.sets: Sliced to the selected indices throughport_opt_view.cache: Sliced to the selected assets throughport_opt_view.
Observation weight parameters
obs_weights_view indexes these fields to the selected observations:
pe: Recursively indexed throughobs_weights_view.cle: Recursively indexed throughobs_weights_view.fees: Recursively indexed throughobs_weights_view.cache: Removed byobs_weights_view, because a state has no slice on the observation axis.
Examples
julia> HierarchicalOptimiser()HierarchicalOptimiser pe ┼ EmpiricalPrior │ ce ┼ PortfolioOptimisersCovariance │ │ ce ┼ Covariance │ │ │ me ┼ SimpleExpectedReturns │ │ │ │ w ┴ nothing │ │ │ ce ┼ GeneralCovariance │ │ │ │ ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true) │ │ │ │ w ┴ nothing │ │ │ alg ┼ FullMoment() │ │ │ w ┴ nothing │ │ mp ┼ MatrixProcessing │ │ │ pdm ┼ Posdef │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple() │ │ │ dn ┼ nothing │ │ │ dt ┼ nothing │ │ │ alg ┼ nothing │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg) │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ horizon ┼ nothing │ fill_limit ┼ nothing │ x_src ┴ nothing cle ┼ ClustersEstimator │ ce ┼ PortfolioOptimisersCovariance │ │ ce ┼ Covariance │ │ │ me ┼ SimpleExpectedReturns │ │ │ │ w ┴ nothing │ │ │ ce ┼ GeneralCovariance │ │ │ │ ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true) │ │ │ │ w ┴ nothing │ │ │ alg ┼ FullMoment() │ │ │ w ┴ nothing │ │ mp ┼ MatrixProcessing │ │ │ pdm ┼ Posdef │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple() │ │ │ dn ┼ nothing │ │ │ dt ┼ nothing │ │ │ alg ┼ nothing │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg) │ de ┼ Distance │ │ power ┼ nothing │ │ alg ┴ CanonicalDistance() │ alg ┼ HClustAlgorithm │ │ linkage ┴ Symbol: :ward │ onc ┼ OptimalNumberClusters │ │ max_k ┼ nothing │ │ alg ┼ SecondOrderDifference │ │ │ alg ┼ StandardisedValue │ │ │ │ mv ┼ MeanValue │ │ │ │ │ w ┴ nothing │ │ │ │ sv ┼ StdValue │ │ │ │ │ w ┼ nothing │ │ │ │ │ corrected ┴ Bool: true │ x_src ┴ nothing slv ┼ nothing wb ┼ WeightBounds │ lb ┼ Float64: 0.0 │ ub ┴ Float64: 1.0 fees ┼ nothing sets ┼ nothing wf ┼ IterativeWeightFinaliser │ iter ┴ Int64: 100 strict ┴ Bool: falseRelated
PortfolioOptimisers.factory — Method
factory(a, args...; kwargs...) -> a
factory(a::AbstractArray{<:Union{Nothing, AbstractEstimator, AbstractAlgorithm,
AbstractResult, StatsBase.CovarianceEstimator}},
args...; kwargs...) -> AbstractArrayRebuild an estimator, an algorithm or a result with runtime values. These two methods are the leaf and the array step of the traversal.
factory and port_opt_view are the two propagation mechanisms of this library. factory passes runtime values, such as prior moments, observation weights and previous portfolio weights, down a tree of composed structs. port_opt_view passes an index selection, a subset of the assets, down the same tree.
factory is a traversal, so it owns the identity on every leaf, as fmap in Functors.jl and adapt_structure in Adapt.jl do. A value that has no method of its own comes back with no change: a data field, a scalar, nothing, a set of observation weights and a covariance estimator of StatsBase. So a @propagatable struct sends each @fprop field to factory directly, whatever the field holds.
The array method is the one forwarding contract for every propagation field that holds an array of estimators, algorithms, results or covariance estimators. It applies factory to each element, and it forwards args... and kwargs... with no change. A family that accepts an array of them needs no method of its own. A family that needs more than the forward defines its own, more specific method. A concrete element type, through concrete_typed_array_if_abstract, is an example. An array of data, such as a vector of numbers, is a leaf, and it comes back with no change and with no copy.
Algorithm
The leaf method:
- Return
awith no change, and dropargs...andkwargs.... This method is the leaf of the recursion. Because of it, a call of the verb on a type with no tags returns the value and raises no error.
The array method:
- For each element
aiofa, callfactoryonai, and forwardargs...andkwargs...with no change. - Collect the results into a new array of the shape of
a, in the order ofa, and return it.
A @propagatable struct always has a generated method that is more specific than the leaf method. If the struct has at least one @fprop or @wprop field, that method builds the struct again with its keyword constructor. It sends each @fprop field through factory and each @wprop field through _wprop. If the struct has no such field, that method returns x.
The leaf method does not fill a nothing field. A weights field that must take the incoming weights when it holds nothing carries @wprop, and it goes to _wprop.
Arguments
a: Any value, which the leaf method returns with no change. For the array method, an array of estimators, algorithms, results, covariance estimators ornothing, which it rebuilds one element at a time.args...: Positional arguments. The leaf method ignores them, and the array method forwards them.kwargs...: Keyword arguments. The leaf method ignores them, and the array method forwards them.
Returns
a: The input with no change, for the leaf method.v::AbstractArray: The rebuilt elements, in an array of the shape ofa, for the array method.
Examples
julia> factory(nothing, 1, 2; x = 3)julia> factory([1.0, 2.0], 1, 2; x = 3)2-element Vector{Float64}: 1.0 2.0julia> factory(MeanValue())MeanValue w ┴ nothingRelated
factory(res::NonFiniteAllocationOptimisationResult, fb::Option{<:OptE_Opt_FbChain})Rebuilds a continuous optimisation result with a new fallback record fb.
A concrete result can add its own method when a rebuild needs more than a new fb. optimise is the one caller, and it gives the FbChain of the attempts that failed.
Algorithm
- Read every field of
res, in the order of the struct. - Call the constructor of the type of
reswith every field except the last, and withfbas the last field.
Validation
fbis the last field of the type ofres. Every concrete optimisation result obeys this rule, so the method checks nothing.
Related
factory(
opt::Union{NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult},
_
) -> EqualWeighted{_A, _B, _C, _D, _E, Bool} where {_A, _B, _C, _D, _E}
Returns opt unchanged.
This is the fallback factory of an optimisation estimator or result. An estimator whose parameters change at each step of an optimisation adds its own method. A precomputed result in a field of an estimator, for example in fb, reaches this method when factory gives the estimator the weights of the previous fold. A precomputed result does not change.
Related
PortfolioOptimisers.factory — Method
factory(a, args...; kwargs...) -> a
factory(a::AbstractArray{<:Union{Nothing, AbstractEstimator, AbstractAlgorithm,
AbstractResult, StatsBase.CovarianceEstimator}},
args...; kwargs...) -> AbstractArrayRebuild an estimator, an algorithm or a result with runtime values. These two methods are the leaf and the array step of the traversal.
factory and port_opt_view are the two propagation mechanisms of this library. factory passes runtime values, such as prior moments, observation weights and previous portfolio weights, down a tree of composed structs. port_opt_view passes an index selection, a subset of the assets, down the same tree.
factory is a traversal, so it owns the identity on every leaf, as fmap in Functors.jl and adapt_structure in Adapt.jl do. A value that has no method of its own comes back with no change: a data field, a scalar, nothing, a set of observation weights and a covariance estimator of StatsBase. So a @propagatable struct sends each @fprop field to factory directly, whatever the field holds.
The array method is the one forwarding contract for every propagation field that holds an array of estimators, algorithms, results or covariance estimators. It applies factory to each element, and it forwards args... and kwargs... with no change. A family that accepts an array of them needs no method of its own. A family that needs more than the forward defines its own, more specific method. A concrete element type, through concrete_typed_array_if_abstract, is an example. An array of data, such as a vector of numbers, is a leaf, and it comes back with no change and with no copy.
Algorithm
The leaf method:
- Return
awith no change, and dropargs...andkwargs.... This method is the leaf of the recursion. Because of it, a call of the verb on a type with no tags returns the value and raises no error.
The array method:
- For each element
aiofa, callfactoryonai, and forwardargs...andkwargs...with no change. - Collect the results into a new array of the shape of
a, in the order ofa, and return it.
A @propagatable struct always has a generated method that is more specific than the leaf method. If the struct has at least one @fprop or @wprop field, that method builds the struct again with its keyword constructor. It sends each @fprop field through factory and each @wprop field through _wprop. If the struct has no such field, that method returns x.
The leaf method does not fill a nothing field. A weights field that must take the incoming weights when it holds nothing carries @wprop, and it goes to _wprop.
Arguments
a: Any value, which the leaf method returns with no change. For the array method, an array of estimators, algorithms, results, covariance estimators ornothing, which it rebuilds one element at a time.args...: Positional arguments. The leaf method ignores them, and the array method forwards them.kwargs...: Keyword arguments. The leaf method ignores them, and the array method forwards them.
Returns
a: The input with no change, for the leaf method.v::AbstractArray: The rebuilt elements, in an array of the shape ofa, for the array method.
Examples
julia> factory(nothing, 1, 2; x = 3)julia> factory([1.0, 2.0], 1, 2; x = 3)2-element Vector{Float64}: 1.0 2.0julia> factory(MeanValue())MeanValue w ┴ nothingRelated
factory(res::NonFiniteAllocationOptimisationResult, fb::Option{<:OptE_Opt_FbChain})Rebuilds a continuous optimisation result with a new fallback record fb.
A concrete result can add its own method when a rebuild needs more than a new fb. optimise is the one caller, and it gives the FbChain of the attempts that failed.
Algorithm
- Read every field of
res, in the order of the struct. - Call the constructor of the type of
reswith every field except the last, and withfbas the last field.
Validation
fbis the last field of the type ofres. Every concrete optimisation result obeys this rule, so the method checks nothing.
Related
factory(
opt::Union{NonFiniteAllocationOptimisationEstimator, NonFiniteAllocationOptimisationResult},
_
) -> EqualWeighted{_A, _B, _C, _D, _E, Bool} where {_A, _B, _C, _D, _E}
Returns opt unchanged.
This is the fallback factory of an optimisation estimator or result. An estimator whose parameters change at each step of an optimisation adds its own method. A precomputed result in a field of an estimator, for example in fb, reaches this method when factory gives the estimator the weights of the previous fold. A precomputed result does not change.
Related
PortfolioOptimisers.BaseHierarchicalOptimisationResult — Type
abstract type BaseHierarchicalOptimisationResult <: AbstractResultAbstract supertype for the core field block that two hierarchical optimisation results embed.
The type is not in the tree of optimisation results, as BaseJuMPOptimisationResult is not on the JuMP side. optimise does not return a core, so a core must not match a method whose bound is the family of results, for example factory(res::NonFiniteAllocationOptimisationResult, fb).
Its one subtype is HierarchicalResult, which each of the two results embeds as hr.
Interfaces
A subtype is a block of fields, not a result. It declares no method, and it must not get a method whose bound is the family of optimisation results.
Related
PortfolioOptimisers.HierarchicalOptimisationResult — Type
abstract type HierarchicalOptimisationResult <: NonJuMPOptimisationResultAbstract supertype for the results of the estimators that hold a hierarchical optimiser in their field opt.
The field holds a HierarchicalOptimiser. The members are exactly the results of HierarchicalRiskParity, HierarchicalEqualRiskContribution and SchurComplementHierarchicalRiskParity. NestedClustered holds no HierarchicalOptimiser, and its result is not a member.
The family is not called ClusteringOptimisationResult. ClusteringOptimisationEstimator has four subtypes, and one of them is NestedClustered, so that name states a set that this family does not hold.
Two of the three members embed HierarchicalResult as hr. SchurComplementHierarchicalRiskParityResult holds its fields directly. So each member that embeds hr forwards its own properties, and this family forwards none.
Interfaces
The family adds no method to NonJuMPOptimisationResult. A member that embeds HierarchicalResult declares its own property forwarding, so that the properties w and retcode that OptimisationResult needs resolve through hr.
Related
PortfolioOptimisers.needs_previous_weights — Method
needs_previous_weights(opt::HierarchicalOptimiser) -> Any
Return true when a HierarchicalOptimiser reads the weights of the previous fold.
The optimiser reads them when its fees do, as a turnover fee does, or when one of its TimeDependent schedules does.
Arguments
opt: The shared configuration of a hierarchical optimiser.
Returns
flag::Bool:truewhen a fold loop must pass the previous weights throughfactory.
Related