Meucci entropy pooling Prior

PortfolioOptimisers.MeucciEntropyPoolingPrior — Type
struct MeucciEntropyPoolingPrior{__T_pe, __T_mu_views, __T_var_views, __T_cvar_views, __T_sigma_views, __T_sk_views, __T_kt_views, __T_cov_views, __T_rho_views, __T_sets, __T_ds_opt, __T_dm_opt, __T_opt, __T_diagnose, __T_w, __T_alg, __T_cache, __T_x_src} <: AbstractLowOrderPriorEstimator_AF

Reweights the observations of a prior so that its moments meet a set of views, and root-finds a CVaR view.

MeucciEntropyPoolingPrior is a low order prior estimator. It calculates the mean and the covariance of the asset returns with entropy pooling. It takes views on the mean, the value at risk, the conditional value at risk, the variance, the covariance, the correlation, the skewness and the kurtosis. It also takes prior weights and the configuration of the solvers.

This estimator is the earlier of the two entropy pooling estimators of the library. The library keeps it because its CVaR route is a different algorithm, not a different formulation of the same problem. A CVaR view goes through the recursive algorithm of Meucci et al. [159]. The search of this library finds the value at risk, and it solves the whole entropy pooling problem again at each candidate. For one view, ConditionalValueatRiskEntropyPooling configures that search. This route takes only equality CVaR views, with one asset in each view.

[1] states that the recursive algorithm takes only equality views. The staged algorithms come from [158]. That work puts the skewness and the kurtosis in the class of parameters that need a fixed mean and a fixed variance.

Use EntropyPoolingPrior when a tail view must be an inequality, must name two assets, or must state the entropic or the relativistic value at risk. That estimator writes each tail view as a constraint of one entropy pooling problem, so one solve meets every view.

The comparison operators that a view accepts depend on the moment that it constrains. var_views takes == and >=. cvar_views takes only ==. Every other view family takes ==, >= and <=. An unsupported operator raises a Meta.ParseError that names the operators that the view accepts.

A skewness, kurtosis or correlation view divides by the variance of an asset. A staged algorithm holds that variance at its value from the stage before. Both read the variance from the diagonal of the sigma that pe fits, but the rows state the uncorrected moment of the scenarios. The default covariance estimator is corrected. So under H1_EntropyPooling or H2_EntropyPooling, a skewness or kurtosis view moves the variance of the scenarios up by the correction factor of the stage before. To hold the uncorrected variance and divide by it, give pe an uncorrected covariance estimator, as EntropyPoolingPrior shows.

A prior(...) reference in a skewness or kurtosis view reads the sample statistic under the observation weights of the prior of the stage.

Warning

The estimator does not detect an infeasible view set. The entropy pooling solve and the CVaR search both miss it. The dual of an infeasible set is unbounded, so the minimiser does not stop at a finite point. The posterior puts almost all of its mass on one observation, and Optim reports the solve as converged. The residual of the CVaR search is small on such a posterior, so the search also reports success. entropy_pooling states the mechanism.

Read the result, not the flag. On such a result, ens is a small number against the number of observations, and one weight is near one. kld is large, and the posterior statistic of the view is far from its target. A frequent cause is two views that move one asset in opposite directions. An example is a sigma_views row that decreases the variance of an asset, beside a cvar_views row that increases the tail loss of the same asset. The same pair on two different assets is feasible and solves correctly, so the cause is the direction and not the pair.

Sometimes the dual overflows before the search stops, and the view set raises an error instead of a result. On the staged route, the moment estimators get weights that are not finite, and they raise an ArgumentError that names Infs or NaNs. On the single-shot route, the CVaR search raises its own ErrorException. Neither error names the infeasibility, because both come from the same unbounded dual at a later point. The outcome of a view set is not stable, and a small change in the sequence of candidate value at risk levels can change it. Treat the three outcomes as one result, and read the views, not the message.

To refuse an infeasible set of moment views before the solve, give diagnose a solver of linear programmes. ep_diagnose_views then decides with one linear programme for each stage whether the mu_views, var_views, sigma_views, cov_views, sk_views, kt_views and rho_views rows can hold together, and its error names the view fields. It does not read cvar_views, because the CVaR search makes its rows only during the solve.

The search can miss a feasible view with no error

The CVaR search checks its own variable, not the view. The view is a constraint of an inner solve, and the search does not read how closely that solve meets it. With one view, Roots.find_zero finds the root of the posterior tail mass minus alpha over the candidate value at risk. It stops when that residual is small. With more than one view, the guard is Optim.converged, which accepts a solve that stopped on the step in x and not on stationarity. So a feasible view set can return a posterior that misses its target, with no error.

This result is different from the result of the warning above. ens is normal, kld is small, and only the statistic of the view misses its target. The size of the miss is a property of the run, not of the estimator, and the miss does not repeat. A case that meets its target closely in a process of its own can miss it by orders of magnitude after other cases in the same process. In one measurement, the residual also changed after an edit to a comment in this file. So the search is not the only cause, and the cause is not known.

Read the result, not the flag. Measure the posterior with ConditionalValueatRisk at the alpha of the view, and compare it with the target of the view. For a result that repeats, give a tighter Optim.Options in the args of the optimiser. OptimEntropyPooling shows the block. The library reads no tolerance on the residual, because a threshold that decides if a solve succeeded is a policy that this library does not set.

With more than one view, the default dm_opt stops when the candidate value at risk moves by less than 1e-4. A target whose value at risk is below zero, or a view that lowers the CVaR beside a view that raises it, can then miss by more than 1e-5 relative. In one measurement on five pairs of views, the largest miss was 4.2e-5 relative under the default. Under dm_opt = OptimEntropyPooling(; args = (Optim.Fminbox(), Optim.Options(; outer_x_abstol = 1e-6, x_abstol = 1e-6))), it was 1.1e-5, at 3 to 9 times the time. On ten scenarios, a pair with an asset whose every loss is above zero missed by 1.7e-3 relative under the default, and by 5.5e-6 under the tighter rule. A tighter rule than 1e-6 gave no smaller miss, because the inner solves set the floor. For a smaller miss than that floor, give opt a tighter rule too.

Algorithm

The constructor derives the prior probabilities and validates every other field.

  1. When w is nothing, derive nothing. At solve time, prior uses the w of the wrapped result, or the uniform weights 1/T, one for each observation, when that result has none.
  2. When w is not nothing, normalise it to sum to one. The normalised weights are the prior probabilities that the pooling starts from. The caller's w does not change, because new weights of the same kind hold the normalised values, through scenario_probabilities. The kind fixes the bias correction of the posterior covariance. ScenarioProbabilities states the rule.

Fields

  • pe: Prior estimator.
  • mu_views: Mean views, or nothing for no mean view.
  • var_views: Value at risk views, as one view group or a vector of groups, or nothing for no value at risk view.
  • cvar_views: Conditional value at risk views, as one view group or a vector of groups, or nothing for no conditional value at risk view.
  • sigma_views: Variance views, or nothing for no variance view.
  • sk_views: Skewness views, or nothing for no skewness view.
  • kt_views: Kurtosis views, or nothing for no kurtosis view.
  • cov_views: Covariance views, or nothing for no covariance view.
  • rho_views: Correlation views, or nothing for no correlation view.
  • sets: Sets used to map estimator values to assets.
  • ds_opt: Root search for one conditional value at risk view, which holds the arguments and the keyword arguments of Roots.find_zero. If nothing, the search uses a default ConditionalValueatRiskEntropyPooling.
  • dm_opt: Optimiser of the search for more than one conditional value at risk view. If nothing, the search uses a default OptimEntropyPooling over Optim.Fminbox.
  • opt: Optimiser of each entropy pooling solve.
  • diagnose: Solver or vector of solvers of the feasibility programme that ep_diagnose_views solves before the solve of each stage, or nothing to skip the check. The programme refuses a set of views that no reweighting of the scenarios meets together, with an error that names the view fields. It costs one more solve for each stage, so the default is nothing.
  • w: Prior observation probability weights, on the observations that the wrapped estimator returns. A wrapped estimator that drops rows returns fewer observations than it gets. If nothing, the fit uses the w of the wrapped result, or the uniform weights when that result has none. The posterior probabilities take the kind of these weights, StatsBase.AnalyticWeights or StatsBase.ProbabilityWeights, and the kind fixes the bias correction of the posterior covariance. Without a w from the caller, the kind is the kind of the w of the wrapped result when it is one of the two, and StatsBase.AnalyticWeights otherwise. ScenarioProbabilities states the rule.
  • alg: Entropy pooling algorithm, which applies the views in one solve or in stages.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it. factory keeps it unchanged, because a factory call resolves the configuration, not the sample. port_opt_view copies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone. obs_weights_view removes it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.
  • x_src: The source of the returns that the prior fits on, nothing or an AbstractReturnsSource. nothing and DataSource read the asset returns rd.X. ExcessSource reads the returns in excess of the benchmark, rd.X - rd.B, the Excess Frame, and refuses returns data with no benchmark. nothing means that the caller stated no source, so an optimiser that holds a reader of the Excess Frame warns about a prior at nothing, and not about a prior at DataSource(). A prior reads no prior, so it refuses PriorSource. A prior inside another estimator, such as the pe of a FactorPrior, keeps nothing, and the outer estimator states the source.

Constructors

MeucciEntropyPoolingPrior(;    pe::AbstractLowOrderPriorEstimator_A_F_AF = EmpiricalPrior(),    mu_views::Option{<:LinearConstraintEstimator} = nothing,    var_views::Option{<:VV_VecVV} = nothing,    cvar_views::Option{<:CVV_VecCVV} = nothing,    sigma_views::Option{<:LinearConstraintEstimator} = nothing,    sk_views::Option{<:LinearConstraintEstimator} = nothing,    kt_views::Option{<:LinearConstraintEstimator} = nothing,    cov_views::Option{<:LinearConstraintEstimator} = nothing,    rho_views::Option{<:LinearConstraintEstimator} = nothing,    sets::Option{<:UniverseSets} = nothing,    ds_opt::Option{<:ConditionalValueatRiskEntropyPooling} = nothing,    dm_opt::Option{<:OptimEntropyPooling} = nothing,    opt::NonCVaREP = OptimEntropyPooling(),    diagnose::Option{<:Slv_VecSlv} = nothing,    w::Option{<:ScenarioProbabilities} = nothing,    alg::AbstractEntropyPoolingAlgorithm = H1_EntropyPooling(),    cache::Option{<:AbstractPartialFitState} = nothing,    x_src::Option{<:AbstractReturnsSource} = nothing) -> MeucciEntropyPoolingPrior

Keywords correspond to the struct's fields.

Validation

  • If a view field is not nothing, sets must not be nothing.
  • If var_views is a vector, it must not be empty.
  • If diagnose is a vector, it must not be empty.
  • If w is not nothing, it must not be empty. At solve time, its length must equal the number of observations of the wrapped result.
  • assert_scenario_refit checks that the refit under the posterior weights can state the model of pe. It refuses an OpinionPoolingPrior whose pe1 is inside a factor prior or a Black-Litterman prior of pe.
  • x_src must be nothing, DataSource() or ExcessSource(). assert_returns_source checks it.
  • The x_src of pe must be nothing, because this prior selects the returns that it reads. assert_inner_reader checks it.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • pe: Recursively updated through factory.
  • w: Replaced with the incoming ObsWeights.
  • cache: Carried unchanged through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Observation weight parameters

obs_weights_view indexes these fields to the selected observations:

Examples

julia> MeucciEntropyPoolingPrior(;                                 sets = UniverseSets(; xkey = "nx",                                                     dict = Dict("nx" => ["A", "B", "C"])),                                 mu_views = LinearConstraintEstimator(;                                                                      val = ["A == 0.03",                                                                             "B + C == 0.04"]))MeucciEntropyPoolingPrior           pe ┼ EmpiricalPrior              │           ce ┼ PortfolioOptimisersCovariance              │              │   ce ┼ Covariance              │              │      │    me ┼ SimpleExpectedReturns              │              │      │       │   w ┴ nothing              │              │      │    ce ┼ GeneralCovariance              │              │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)              │              │      │       │    w ┴ nothing              │              │      │   alg ┼ FullMoment()              │              │      │     w ┴ nothing              │              │   mp ┼ MatrixProcessing              │              │      │     pdm ┼ Posdef              │              │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton              │              │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()              │              │      │      dn ┼ nothing              │              │      │      dt ┼ nothing              │              │      │     alg ┼ nothing              │              │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)              │           me ┼ SimpleExpectedReturns              │              │   w ┴ nothing              │      horizon ┼ nothing              │   fill_limit ┼ nothing              │        x_src ┴ nothing     mu_views ┼ LinearConstraintEstimator              │   val ┼ Vector{String}: ["A == 0.03", "B + C == 0.04"]              │   key ┴ nothing    var_views ┼ nothing   cvar_views ┼ nothing  sigma_views ┼ nothing     sk_views ┼ nothing     kt_views ┼ nothing    cov_views ┼ nothing    rho_views ┼ nothing         sets ┼ UniverseSets              │     xkey ┼ String: "nx"              │    uxkey ┼ String: "ux"              │    tfkey ┼ String: "nf"              │   utfkey ┼ String: "uf"              │    cfkey ┼ String: "ncf"              │   ucfkey ┼ String: "ucf"              │    nikey ┼ String: "ni"              │     dict ┴ Dict{String, Vector{String}}: Dict("nx" => ["A", "B", "C"])       ds_opt ┼ nothing       dm_opt ┼ nothing          opt ┼ OptimEntropyPooling              │     args ┼ Tuple{}: ()              │   kwargs ┼ @NamedTuple{}: NamedTuple()              │      sc1 ┼ Int64: 1              │      sc2 ┼ Float64: 1000.0              │      alg ┼ ExpEntropyPooling()              │      err ┴ nothing     diagnose ┼ nothing            w ┼ nothing          alg ┼ H1_EntropyPooling()        x_src ┴ nothing

The incremental fit

This prior has no exact incremental recursion, and it takes the online step by a refit from a sample buffer. Online seeds cache, and partial_fit! adds each observation to it with no change. The prior method with one argument then runs the batch verb of this estimator over the rows that the buffer kept. The result is the batch fit over those rows, and a max_history on the wrapper sets the window of the whole fit.

cache goes through the three propagation channels as every partial-fit state does. factory carries it with no change, and port_opt_view cuts it to the selected assets. obs_weights_view drops it, because a state has no cut on the observation axis. The show method does not print cache, because a buffer that changes at each step is not part of the configuration.

Related

References

  • [157] A. Meucci. Fully flexible views: theory and practice. Risk 21, 97–102 (2008). Section 4, Equations 25 to 27, and Appendix A.2, Equations 65 to 73.
  • [159] A. Meucci, D. Ardia and S. Keel. Fully flexible extreme views. The Journal of Risk 14, 39–49 (2011). Section 3, Equations 3.1 to 3.5.
  • [1] D. Cajas. Entropy Pooling with CVaR and EVaR Views. Available at SSRN 7120258 (2026). Section 2.2.
  • [158] A. Vorobets. Sequential entropy pooling heuristics. Available at SSRN 3936392 (2021). Section 3.
source
PortfolioOptimisers.prior — Function
prior(pe::MeucciEntropyPoolingPrior, X::MatNum, F::Option{<:MatNum} = nothing,
      pnl::Option{<:AssetPanel} = nothing; dims::Int = 1, strict::Bool = false,
      kwargs...)

Compute the entropy pooling prior moments of the asset returns.

prior orients the data along dims and calls ep_prior, which dispatches on the entropy pooling algorithm pe.alg. H0_EntropyPooling applies every view in one optimisation. StagedEP, the union of H1_EntropyPooling and H2_EntropyPooling, applies the views in stages, from the lower moments to the higher moments.

Algorithm

  1. Orient X and F to observations by assets with dims_oriented, so every step below reads one observation in each row.
  2. When pe.sets is set, check that the length of its asset universe is equal to the number of columns of X, with assert_ep_views_axis.
  3. Dispatch on pe.alg through ep_prior, and return the LowOrderPrior that it makes.

Arguments

  • pe: Entropy pooling prior estimator.
  • X: Asset returns matrix, with its observations along dims.
  • F: Optional factor returns matrix, with its observations along dims.
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.
  • dims: Dimension along which the computation runs.
  • strict: If true, raise an error for an asset that the universe does not hold. If false, warn.
  • kwargs...: More keyword arguments, which go to the wrapped prior estimator.

Validation

  • dims in (1, 2).
  • If pe.sets is not nothing, length(pe.sets.dict[pe.sets.xkey]) == size(X, 2). Raises a DimensionMismatch.

Returns

  • pr::LowOrderPrior: Prior result that holds the asset returns, the posterior mean and covariance, the posterior probability weights, the effective number of scenarios, the Kullback-Leibler divergence and the factor block of the last refit.

Related

source

References

[1]
D. Cajas. Entropy Pooling with CVaR and EVaR Views. Available at SSRN 7120258 (2026).
[157]
A. Meucci. Fully flexible views: theory and practice. Risk 21, 97–102 (2008).
[158]
A. Vorobets. Sequential entropy pooling heuristics. Available at SSRN 3936392 (2021).
[159]
A. Meucci, D. Ardia and S. Keel. Fully flexible extreme views. The Journal of Risk 14, 39–49 (2011).