Meucci Entropy Pooling

PortfolioOptimisers.MeucciEntropyPoolingPrior — Type

struct MeucciEntropyPoolingPrior{__T_pe, __T_mu_views, __T_var_views, __T_cvar_views, __T_sigma_views, __T_sk_views, __T_kt_views, __T_cov_views, __T_rho_views, __T_sets, __T_ds_opt, __T_dm_opt, __T_opt, __T_w, __T_alg, __T_cache} <: AbstractLowOrderPriorEstimator_AF

Reweights the observations of a prior so that its moments meet a set of views, and root-finds a CVaR view.

MeucciEntropyPoolingPrior is a low order prior estimator that computes the mean and covariance of asset returns using entropy pooling. It supports views on the mean, the value at risk, the conditional value at risk, the variance, the covariance, the correlation, the skewness and the kurtosis, and it takes custom prior weights and solver configuration.

This is the earlier of the library's two entropy pooling estimators, and it is kept because its CVaR route is a different algorithm, not a different formulation of the same problem: a CVaR view is a target hunted by the recursive algorithm of Meucci, Ardia and Keel, where ConditionalValueatRiskEntropyPooling root-finds the value at risk level and re-solves the whole entropy pooling problem at each candidate. That route takes equality CVaR views alone, one asset per view.

Reach for EntropyPoolingPrior instead where a tail view has to be an inequality, name two assets, or land on the entropic value at risk: there each tail view is a constraint of the single entropy pooling problem, so one solve answers every view.

The comparison operator a view accepts depends on the moment it constrains: var_views takes == and >=, cvar_views takes == alone, and every other view family takes ==, >= and <=. An unsupported operator raises a Meta.ParseError naming the operators that view accepts.

Warning

An infeasible view set is never detected. Neither the entropy pooling solve nor the CVaR search reads one: the dual of an infeasible set is unbounded, so the minimiser runs away, the posterior collapses onto one observation, and Optim reports the solve as converged. The residual the CVaR search minimises is small on such a posterior, so the search reports success too. Read the result rather than the flag: ens falls to a handful out of the number of observations, one weight sits near one, kld is large, and the posterior statistic the view named is far from its target. Views that pull one asset in two directions at once are the common way to reach it, such as a sigma_views row that shrinks an asset written beside a cvar_views row that fattens the same asset's tail. The same pair on two different assets is feasible and solves normally, so it is the direction and not the pairing. entropy_pooling states the mechanism.

A runaway dual sometimes overflows before it settles, and the view set then raises rather than answering. The staged route reaches the moment estimators with non-finite weights, which raise an ArgumentError naming Infs or NaNs. The single-shot route raises the CVaR search's own ErrorException. Neither raise detects the infeasibility. Both are the same runaway dual met further along, and which of the three a given view set gives is not stable: the search over an infeasible set is chaotic, so a change in the sequence of candidate value at risk levels moves the answer between them. Treat any of the three as the same finding, and read the views rather than the message.

A feasible view can be missed in silence

The CVaR search reports on its own variable, and the view rides on a solve it never reads. With one view Roots.find_zero root-finds the posterior tail mass minus alpha over the candidate value at risk, and it stops when that residual is small. The view itself is the constraint the inner entropy_pooling solve carries, and how closely that solve met it is read nowhere. With more than one view the guard is Optim.converged, which accepts a solve that stopped on the step in x rather than on stationarity. A feasible view set can therefore return a posterior that misses its target, with no raise. It is not the answer the warning above describes: ens is healthy, kld is small, and the statistic the view named is the only thing that is short.

The size of the miss is a property of the run and not of the estimator, and it is not reproducible: a case that meets its target closely when solved alone can miss it by orders of magnitude when solved after other cases in the same process. Issue #573 also records the residual moving for reasons as unrelated as an edit to a comment elsewhere in the file, so the cause is not the search alone and it is not settled.

Read the result rather than the flag. Measure the posterior with ConditionalValueatRisk at the view's own alpha, and compare it with the target the view names. Where the answer has to be repeatable, pass a tighter Optim.Options in the optimiser's args; OptimEntropyPooling gives the block. No tolerance of the library's own is read on the residual: a threshold that decides whether a solve really succeeded is a policy this library does not set.

Algorithm

The constructor derives the prior probabilities, and validates everything else.

When w is nothing, derive nothing. prior builds the uniform weights 1/T at solve time, one per observation.
When w is not nothing, normalise it to sum to one, giving the prior probabilities the pooling starts from. A mutable w.values is normalised in place with LinearAlgebra.normalize!, and an immutable one is replaced by a new StatsBase.pweights over the normalised values.

Fields

pe: Prior estimator.

mu_views: Expected returns views estimator or result.

var_views: Value-at-risk views estimator or result.

cvar_views: Conditional value-at-risk views estimator or result.

sigma_views: Variance views estimator or result.

sk_views: Skewness views estimator or result.

kt_views: Kurtosis views estimator or result.

cov_views: Covariance views estimator or result.

rho_views: Correlation views estimator or result.

sets: Sets used to map estimator values to assets.

ds_opt: Thin wrapper for arguments and keyword arguments used in Roots.findzero for use with a single conditional value-at-risk view.

dm_opt: Optimiser for multiple conditional value at risk views.

opt: Entropy pooling optimisation estimator.

w: Prior observation probability weights, on the observations the wrapped estimator answers, which a wrapped estimator that drops rows makes fewer than the observations it is given. If nothing, the wrapped result's own w is used, and the uniform weights where it carries none.

alg: Entropy pooling algorithm.

cache: Optional partial-fit state. It is nothing until partial_fit! writes one, and the estimator's read-out verb reads it when the caller gives no data matrix. Each propagation channel does one thing with it: factory carries it unchanged, because a factory call resolves configuration rather than the sample; port_opt_view slices it to the selected assets by index copy, so the viewed estimator answers over those assets alone; and obs_weights_view drops it, because no slice of a state exists on the observation axis. A family whose state has no exact asset slice drops it on both axes and names the reason.

Constructors

MeucciEntropyPoolingPrior(;    pe::AbstractLowOrderPriorEstimator_A_F_AF = EmpiricalPrior(),    mu_views::Option{<:LinearConstraintEstimator} = nothing,    var_views::Option{<:VV_VecVV} = nothing,    cvar_views::Option{<:CVV_VecCVV} = nothing,    sigma_views::Option{<:LinearConstraintEstimator} = nothing,    sk_views::Option{<:LinearConstraintEstimator} = nothing,    kt_views::Option{<:LinearConstraintEstimator} = nothing,    cov_views::Option{<:LinearConstraintEstimator} = nothing,    rho_views::Option{<:LinearConstraintEstimator} = nothing,    sets::Option{<:UniverseSets} = nothing,    ds_opt::Option{<:ConditionalValueatRiskEntropyPooling} = nothing,    dm_opt::Option{<:OptimEntropyPooling} = nothing,    opt::NonCVaREP = OptimEntropyPooling(),    w::Option{<:StatsBase.ProbabilityWeights} = nothing,    alg::AbstractEntropyPoolingAlgorithm = H1_EntropyPooling(),    cache::Option{<:AbstractPartialFitState} = nothing) -> MeucciEntropyPoolingPrior

Keywords correspond to the struct's fields.

Validation

If any view constraint is not nothing, sets must not be nothing.
If var_views is a vector, !isempty(var_views).
If w is not nothing, it must be non-empty and match the number of observations.

Propagated parameters

When factory is called on this type, the following @fprop-tagged fields are automatically propagated:

pe: Recursively updated via factory.
w: Replaced with the incoming ObsWeights.

View parameters

When port_opt_view is called on this type, the following @vprop-tagged fields are automatically subset to the selected indices:

pe: Recursively viewed via port_opt_view.
sets: Sliced to the selected indices via port_opt_view.

Observation weight parameters

When obs_weights_view is called on this type, the following fields are automatically indexed to the selected observations:

pe: Recursively indexed via obs_weights_view.
w: Indexed to the selected observations via obs_weights_view.

Examples

julia> MeucciEntropyPoolingPrior(;                                 sets = UniverseSets(; xkey = "nx",                                                     dict = Dict("nx" => ["A", "B", "C"])),                                 mu_views = LinearConstraintEstimator(;                                                                      val = ["A == 0.03",                                                                             "B + C == 0.04"]))MeucciEntropyPoolingPrior           pe ┼ EmpiricalPrior              │           ce ┼ PortfolioOptimisersCovariance              │              │   ce ┼ Covariance              │              │      │    me ┼ SimpleExpectedReturns              │              │      │       │   w ┴ nothing              │              │      │    ce ┼ GeneralCovariance              │              │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)              │              │      │       │    w ┴ nothing              │              │      │   alg ┼ FullMoment()              │              │      │     w ┴ nothing              │              │   mp ┼ MatrixProcessing              │              │      │     pdm ┼ Posdef              │              │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton              │              │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()              │              │      │      dn ┼ nothing              │              │      │      dt ┼ nothing              │              │      │     alg ┼ nothing              │              │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)              │           me ┼ SimpleExpectedReturns              │              │   w ┴ nothing              │      horizon ┼ nothing              │   fill_limit ┴ nothing     mu_views ┼ LinearConstraintEstimator              │   val ┼ Vector{String}: ["A == 0.03", "B + C == 0.04"]              │   key ┴ nothing    var_views ┼ nothing   cvar_views ┼ nothing  sigma_views ┼ nothing     sk_views ┼ nothing     kt_views ┼ nothing    cov_views ┼ nothing    rho_views ┼ nothing         sets ┼ UniverseSets              │     xkey ┼ String: "nx"              │    uxkey ┼ String: "ux"              │    tfkey ┼ String: "nf"              │   utfkey ┼ String: "uf"              │    cfkey ┼ String: "ncf"              │   ucfkey ┼ String: "ucf"              │    nikey ┼ String: "ni"              │     dict ┴ Dict{String, Vector{String}}: Dict("nx" => ["A", "B", "C"])       ds_opt ┼ nothing       dm_opt ┼ nothing          opt ┼ OptimEntropyPooling              │     args ┼ Tuple{}: ()              │   kwargs ┼ @NamedTuple{}: NamedTuple()              │      sc1 ┼ Int64: 1              │      sc2 ┼ Float64: 1000.0              │      alg ┼ ExpEntropyPooling()              │      err ┴ nothing            w ┼ nothing          alg ┴ H1_EntropyPooling()

The incremental fit

This prior has no exact incremental recursion, so it takes the online step by refitting from a sample buffer: Online seeds cache, partial_fit! appends each observation to it verbatim, and the one-argument prior runs this estimator's own batch verb over the rows the buffer kept. The answer is therefore exactly a batch fit over those rows, and a max_history on the wrapper windows the whole fit.

cache travels the three propagation channels as every partial-fit state does: factory carries it unchanged, port_opt_view slices it to the selected assets, and obs_weights_view drops it, because no slice of a state exists on the observation axis. It is not rendered, because a running buffer is not the configuration a reader looks the type up for.

Related

References

[78] A. Meucci. Fully flexible views: theory and practice. Risk 21, 97–102 (2008).
[80] A. Meucci, D. Ardia and S. Keel. Fully flexible extreme views. The Journal of Risk 14, 39–49 (2011).
[79] A. Vorobets. Sequential entropy pooling heuristics. Available at SSRN 3936392 (2021).

source

PortfolioOptimisers.prior — Function

prior(pe::MeucciEntropyPoolingPrior, X::MatNum, F::Option{<:MatNum} = nothing,
      pnl::Option{<:AssetPanel} = nothing; dims::Int = 1, strict::Bool = false,
      kwargs...)

Compute entropy pooling prior moments for asset returns.

prior orients the data with respect to dims and delegates to ep_prior, which dispatches on the entropy pooling algorithm pe.alg. H0_EntropyPooling enforces every view in a single optimisation. StagedEP, the union of H1_EntropyPooling and H2_EntropyPooling, enforces the views in stages, from lower to higher moments.

Algorithm

Orient X and F to observations by assets with dims_oriented, so every step below reads one observation per row.
Dispatch on pe.alg through ep_prior, and return the LowOrderPrior it produces.

Arguments

pe: Entropy pooling prior estimator.
X: Asset returns matrix (observations × assets).
F: Optional factor matrix.
pnl: Optional AssetPanel, the panel the carrier held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.
dims: Dimension along which to perform the computation.
strict: If true, throws error for missing assets; otherwise, issues warnings.
kwargs...: Additional keyword arguments passed to underlying estimators and solvers.

Validation

dims in (1, 2).

Returns

pr::LowOrderPrior: Result object containing asset returns, posterior mean vector, posterior covariance matrix, weights, effective number of scenarios, Kullback-Leibler divergence, and optional factor moments.

Related

source

References

[78]: A. Meucci. Fully flexible views: theory and practice. Risk 21, 97–102 (2008).
[79]: A. Vorobets. Sequential entropy pooling heuristics. Available at SSRN 3936392 (2021).
[80]: A. Meucci, D. Ardia and S. Keel. Fully flexible extreme views. The Journal of Risk 14, 39–49 (2011).