The source files can be found in examples/.
Opinion pooling
Entropy pooling turns one set of views into a reweighted prior. But you often hold several sets of views from different sources, such as a fundamental analyst, a quantitative signal and a macro desk, and they can conflict. Opinion pooling combines several entropy pooling posteriors into one consensus prior, and weights each opinion by how much you trust it. This page is the last of three on priors built from views. Each opinion is an EntropyPoolingPrior, and the pool blends them.
OpinionPoolingPrior takes a vector of entropy pooling priors in pes, optional credibility weights in w, a pooling algorithm in alg, which is LinearOpinionPooling or LogarithmicOpinionPooling, and an optional penalty p for robust pooling. We build three opinions and pool them. Sections 5 to 7 then go through the parameters of the pool. Section 5 compares the two pooling algorithms. Section 6 shows the credibility weights and the uniform prior that takes the weight they leave. Section 7 shows robust pooling, which lowers the weight of an opinion far from the others.
Reach for opinion pooling when you hold several sets of views that can conflict, and want one consensus rather than a choice of one set or an average of forecasts by hand. Give each opinion a weight for its credibility. If you hold one consistent set of views, entropy pooling is enough, and if your views are on the mean alone and Gaussian, Black-Litterman is simpler still.
using PortfolioOptimisers, PrettyTablesmmtfmt = (v, i, j) -> begin if j == 1 return v else return isa(v, Number) ? "$(round(v*100, digits=4)) %" : v endend;resfmt = (v, i, j) -> begin if j == 1 return v else return isa(v, Number) ? "$(round(v*100, digits=3)) %" : v endend;1. The data
We load one year of daily returns of 20 assets, and name two groups of them for the views.
using CSV, TimeSeries, DataFramesX = TimeArray(CSV.File(joinpath(@__DIR__, "..", "SP500.csv.gz")); timestamp = :Date)[(end - 252):end]rd = prices_to_returns(X)sets = UniverseSets(; dict = Dict("nx" => rd.nx, "tech" => ["AAPL", "AMD", "MSFT"], "energy" => ["CVX"]))UniverseSets
xkey ┼ String: "nx"
uxkey ┼ String: "ux"
tfkey ┼ String: "nf"
utfkey ┼ String: "uf"
cfkey ┼ String: "ncf"
ucfkey ┼ String: "ucf"
nikey ┼ String: "ni"
dict ┴ Dict{String, Vector{String}}: Dict("tech" => ["AAPL", "AMD", "MSFT"], "energy" => ["CVX"], "nx" => ["AAPL", "AMD", "BAC", "BBY", "CVX", "GE", "HD", "JNJ", "JPM", "KO", "LLY", "MRK", "MSFT", "PEP", "PFE", "PG", "RRC", "UNH", "WMT", "XOM"])
2. Three opinions
Each opinion is an EntropyPoolingPrior with its own views.
- Opinion A is bullish on Apple,
AAPL == 0.0008. - Opinion B is a view on sectors. The three tech means added together are at least the energy mean, and Microsoft returns 6 bps a day.
- Opinion C is defensive. It sets the expected returns of two low-volatility stocks, Johnson & Johnson at 4 bps a day and Coca-Cola at 3 bps a day.
opinion_a = EntropyPoolingPrior(; sets = sets, mu_views = LinearConstraintEstimator(; val = ["AAPL == 0.0008"]))opinion_b = EntropyPoolingPrior(; sets = sets, mu_views = LinearConstraintEstimator(; val = ["tech >= energy", "MSFT == 0.0006"]))opinion_c = EntropyPoolingPrior(; sets = sets, mu_views = LinearConstraintEstimator(; val = ["JNJ == 0.0004", "KO == 0.0003"]))EntropyPoolingPrior
pe ┼ EmpiricalPrior
│ ce ┼ PortfolioOptimisersCovariance
│ │ ce ┼ Covariance
│ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ w ┴ nothing
│ │ │ ce ┼ GeneralCovariance
│ │ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ │ w ┴ nothing
│ │ │ alg ┼ FullMoment()
│ │ │ w ┴ nothing
│ │ mp ┼ MatrixProcessing
│ │ │ pdm ┼ Posdef
│ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ │ dn ┼ nothing
│ │ │ dt ┼ nothing
│ │ │ alg ┼ nothing
│ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
│ me ┼ SimpleExpectedReturns
│ │ w ┴ nothing
│ horizon ┼ nothing
│ fill_limit ┼ nothing
│ x_src ┴ nothing
mu_views ┼ LinearConstraintEstimator
│ val ┼ Vector{String}: ["JNJ == 0.0004", "KO == 0.0003"]
│ key ┴ nothing
var_views ┼ nothing
cvar_views ┼ nothing
evar_views ┼ nothing
rlvar_views ┼ nothing
sigma_views ┼ nothing
sk_views ┼ nothing
kt_views ┼ nothing
cov_views ┼ nothing
rho_views ┼ nothing
sets ┼ UniverseSets
│ xkey ┼ String: "nx"
│ uxkey ┼ String: "ux"
│ tfkey ┼ String: "nf"
│ utfkey ┼ String: "uf"
│ cfkey ┼ String: "ncf"
│ ucfkey ┼ String: "ucf"
│ nikey ┼ String: "ni"
│ dict ┴ Dict{String, Vector{String}}: Dict("tech" => ["AAPL", "AMD", "MSFT"], "energy" => ["CVX"], "nx" => ["AAPL", "AMD", "BAC", "BBY", "CVX", "GE", "HD", "JNJ", "JPM", "KO", "LLY", "MRK", "MSFT", "PEP", "PFE", "PG", "RRC", "UNH", "WMT", "XOM"])
opt ┼ OptimEntropyPooling
│ args ┼ Tuple{}: ()
│ kwargs ┼ @NamedTuple{}: NamedTuple()
│ sc1 ┼ Int64: 1
│ sc2 ┼ Float64: 1000.0
│ alg ┼ ExpEntropyPooling()
│ err ┴ nothing
w ┼ nothing
alg ┼ H1_EntropyPooling()
x_src ┴ nothing
3. Pooling the opinions
We pool the three with credibility weights. We trust the Apple view most, the sector view next and the defensive view least. The weights sum to one here, and section 6 shows what happens when they sum to less.
op = OpinionPoolingPrior(; pes = [opinion_a, opinion_b, opinion_c], w = [0.5, 0.3, 0.2])OpinionPoolingPrior
pes ┼ 3-element Vector{EntropyPoolingPrior}
│ EntropyPoolingPrior ⋯
│ EntropyPoolingPrior ⋯
│ EntropyPoolingPrior ⋯
pe1 ┼ nothing
pe2 ┼ EmpiricalPrior
│ ce ┼ PortfolioOptimisersCovariance
│ │ ce ┼ Covariance
│ │ │ me ┼ SimpleExpectedReturns
│ │ │ │ w ┴ nothing
│ │ │ ce ┼ GeneralCovariance
│ │ │ │ ce ┼ SimpleCovariance: SimpleCovariance(true)
│ │ │ │ w ┴ nothing
│ │ │ alg ┼ FullMoment()
│ │ │ w ┴ nothing
│ │ mp ┼ MatrixProcessing
│ │ │ pdm ┼ Posdef
│ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton
│ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple()
│ │ │ dn ┼ nothing
│ │ │ dt ┼ nothing
│ │ │ alg ┼ nothing
│ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)
│ me ┼ SimpleExpectedReturns
│ │ w ┴ nothing
│ horizon ┼ nothing
│ fill_limit ┼ nothing
│ x_src ┴ nothing
p ┼ nothing
w ┼ Vector{Float64}: [0.5, 0.3, 0.2]
alg ┼ LinearOpinionPooling()
ex ┼ Transducers.ThreadedEx{@NamedTuple{}}: Transducers.ThreadedEx()
x_src ┴ nothing
4. The consensus and the individual opinions
We compute the posterior of each opinion and the pooled consensus, and compare their expected returns with those of the plain empirical prior. With weights that sum to one, each pooled mean is the average of the opinions' means, weighted by w.
pr_emp = prior(EmpiricalPrior(), rd)pr_a = prior(opinion_a, rd)pr_b = prior(opinion_b, rd)pr_c = prior(opinion_c, rd)pr_op = prior(op, rd)pretty_table(DataFrame(["Assets" => rd.nx, "Empirical" => pr_emp.mu, "Opinion A" => pr_a.mu, "Opinion B" => pr_b.mu, "Opinion C" => pr_c.mu, "Pooled" => pr_op.mu]); formatters = [mmtfmt], title = "Expected returns of the empirical prior, each opinion and the pooled consensus")Expected returns of the empirical prior, each opinion and the pooled consensus
┌────────┬───────────┬───────────┬───────────┬───────────┬───────────┐
│ Assets │ Empirical │ Opinion A │ Opinion B │ Opinion C │ Pooled │
│ String │ Float64 │ Float64 │ Float64 │ Float64 │ Float64 │
├────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ AAPL │ -0.1126 % │ 0.08 % │ 0.0764 % │ -0.1273 % │ 0.0375 % │
│ AMD │ -0.2809 % │ -0.0392 % │ 0.0591 % │ -0.3038 % │ -0.0626 % │
│ BAC │ -0.0934 % │ 0.0062 % │ 0.0113 % │ -0.1048 % │ -0.0145 % │
│ BBY │ -0.0279 % │ 0.1019 % │ 0.1194 % │ -0.0428 % │ 0.0782 % │
│ CVX │ 0.1945 % │ 0.2456 % │ 0.1955 % │ 0.1892 % │ 0.2193 % │
│ GE │ -0.0339 % │ 0.0688 % │ 0.0751 % │ -0.0474 % │ 0.0475 % │
│ HD │ -0.0707 % │ 0.0303 % │ 0.0441 % │ -0.0823 % │ 0.0119 % │
│ JNJ │ 0.0307 % │ 0.0646 % │ 0.0589 % │ 0.04 % │ 0.058 % │
│ JPM │ -0.0417 % │ 0.0473 % │ 0.0523 % │ -0.0511 % │ 0.0291 % │
│ KO │ 0.0497 % │ 0.1054 % │ 0.1016 % │ 0.03 % │ 0.0892 % │
│ LLY │ 0.1305 % │ 0.1839 % │ 0.1809 % │ 0.1346 % │ 0.1732 % │
│ MRK │ 0.1669 % │ 0.1968 % │ 0.1836 % │ 0.1717 % │ 0.1878 % │
│ MSFT │ -0.1206 % │ 0.0355 % │ 0.06 % │ -0.1332 % │ 0.0091 % │
│ PEP │ 0.039 % │ 0.0961 % │ 0.0921 % │ 0.0263 % │ 0.0809 % │
│ PFE │ -0.0256 % │ 0.0231 % │ 0.0181 % │ -0.0226 % │ 0.0125 % │
│ PG │ -0.0081 % │ 0.0466 % │ 0.0497 % │ -0.019 % │ 0.0344 % │
│ RRC │ 0.1824 % │ 0.2913 % │ 0.2556 % │ 0.1622 % │ 0.2548 % │
│ UNH │ 0.0364 % │ 0.0995 % │ 0.0941 % │ 0.0327 % │ 0.0845 % │
│ WMT │ 0.0163 % │ 0.0652 % │ 0.0547 % │ 0.0082 % │ 0.0506 % │
│ XOM │ 0.2637 % │ 0.3155 % │ 0.2644 % │ 0.2567 % │ 0.2884 % │
└────────┴───────────┴───────────┴───────────┴───────────┴───────────┘The expected returns of the pooled consensus.
using StatsPlots, GraphRecipesplot_mu(pr_op, rd.nx)
5. Linear and logarithmic pooling
The pooling alg sets how the pool combines the distributions of the opinions.
LinearOpinionPooling, the default, takes a weighted arithmetic average of the probabilities of the opinions. It is the mixture of experts rule. The consensus is a blend, so one confident opinion can move the mean a long way.LogarithmicOpinionPoolingtakes a weighted geometric mean, the consensus that is optimal in Kullback-Leibler divergence. An asset moves far only when the opinions move it the same way, so a conflict between them has less effect.
The two pools give similar means when the opinions are compatible, and different means as the opinions conflict. The table prints the pooled mean of every asset under each.
pr_lin = prior(OpinionPoolingPrior(; pes = [opinion_a, opinion_b, opinion_c], w = [0.5, 0.3, 0.2], alg = LinearOpinionPooling()), rd)pr_log = prior(OpinionPoolingPrior(; pes = [opinion_a, opinion_b, opinion_c], w = [0.5, 0.3, 0.2], alg = LogarithmicOpinionPooling()), rd)pretty_table(DataFrame(["Assets" => rd.nx, "Linear pool" => pr_lin.mu, "Logarithmic pool" => pr_log.mu]); formatters = [mmtfmt], title = "Pooled expected returns, linear and logarithmic")Pooled expected returns, linear and logarithmic
┌────────┬─────────────┬──────────────────┐
│ Assets │ Linear pool │ Logarithmic pool │
│ String │ Float64 │ Float64 │
├────────┼─────────────┼──────────────────┤
│ AAPL │ 0.0375 % │ 0.0367 % │
│ AMD │ -0.0626 % │ -0.0635 % │
│ BAC │ -0.0145 % │ -0.0148 % │
│ BBY │ 0.0782 % │ 0.0781 % │
│ CVX │ 0.2193 % │ 0.2193 % │
│ GE │ 0.0475 % │ 0.0475 % │
│ HD │ 0.0119 % │ 0.0114 % │
│ JNJ │ 0.058 % │ 0.0579 % │
│ JPM │ 0.0291 % │ 0.0289 % │
│ KO │ 0.0892 % │ 0.0891 % │
│ LLY │ 0.1732 % │ 0.1731 % │
│ MRK │ 0.1878 % │ 0.1878 % │
│ MSFT │ 0.0091 % │ 0.0085 % │
│ PEP │ 0.0809 % │ 0.081 % │
│ PFE │ 0.0125 % │ 0.0124 % │
│ PG │ 0.0344 % │ 0.0344 % │
│ RRC │ 0.2548 % │ 0.2542 % │
│ UNH │ 0.0845 % │ 0.0846 % │
│ WMT │ 0.0506 % │ 0.0508 % │
│ XOM │ 0.2884 % │ 0.2884 % │
└────────┴─────────────┴──────────────────┘6. Credibility weights and the uniform-prior fallback
The weights w state how credible each opinion is. If they sum to less than one, the pool gives the remaining weight to the uniform prior, which is the empirical distribution before any reweighting. The total of w therefore acts as one confidence for all the views together, and a smaller total pulls the consensus back toward the data and away from the views. We hold the relative trust at 5:3:2 and scale the total from 1.0 down to 0.4. The table shows the pooled expected returns of Apple and Microsoft as they move back toward the empirical prior.
scales = [1.0, 0.7, 0.4]pr_scaled = [prior(OpinionPoolingPrior(; pes = [opinion_a, opinion_b, opinion_c], w = s .* [0.5, 0.3, 0.2]), rd) for s in scales]i_aapl = findfirst(==("AAPL"), rd.nx)i_msft = findfirst(==("MSFT"), rd.nx)pretty_table(DataFrame("weight total" => ["empirical (0.0)"; string.(scales)], "AAPL posterior" => [pr_emp.mu[i_aapl]; [p.mu[i_aapl] for p in pr_scaled]], "MSFT posterior" => [pr_emp.mu[i_msft]; [p.mu[i_msft] for p in pr_scaled]]); formatters = [mmtfmt], title = "AAPL and MSFT pooled expected returns by total credibility weight")AAPL and MSFT pooled expected returns by total credibility weight
┌─────────────────┬────────────────┬────────────────┐
│ weight total │ AAPL posterior │ MSFT posterior │
│ String │ Float64 │ Float64 │
├─────────────────┼────────────────┼────────────────┤
│ empirical (0.0) │ -0.1126 % │ -0.1206 % │
│ 1.0 │ 0.0375 % │ 0.0091 % │
│ 0.7 │ -0.0075 % │ -0.0298 % │
│ 0.4 │ -0.0526 % │ -0.0687 % │
└─────────────────┴────────────────┴────────────────┘7. Robust pooling with p
By default the pool uses the weight of each opinion as given. Set p, a penalty above zero, to turn on robust opinion pooling. The pool then multiplies the weight of opinion $k$ by $\exp(-p D_k)$, where $D_k$ is the Kullback-Leibler divergence between the distribution of that opinion and the linear consensus, and scales the weights to sum to one. A larger p moves more weight to the opinions nearest the consensus. The adjustment runs before either pooling algorithm.
We pool the three opinions again with p = 0.1 and with p = 100. The ow field of a pooled prior holds the opinion weights after the adjustment, and the table prints them next to the weights we gave.
pr_robust = prior(OpinionPoolingPrior(; pes = [opinion_a, opinion_b, opinion_c], w = [0.5, 0.3, 0.2], p = 0.1), rd)pr_robust_100 = prior(OpinionPoolingPrior(; pes = [opinion_a, opinion_b, opinion_c], w = [0.5, 0.3, 0.2], p = 100), rd)pretty_table(DataFrame(["Opinion" => ["A", "B", "C"], "Given" => pr_op.ow, "p = 0.1" => pr_robust.ow, "p = 100" => pr_robust_100.ow]); formatters = [mmtfmt], title = "Opinion weights after the robust adjustment") Opinion weights after the robust adjustment
┌─────────┬─────────┬───────────┬───────────┐
│ Opinion │ Given │ p = 0.1 │ p = 100 │
│ String │ Float64 │ Float64 │ Float64 │
├─────────┼─────────┼───────────┼───────────┤
│ A │ 50.0 % │ 50.0035 % │ 53.3009 % │
│ B │ 30.0 % │ 30.0006 % │ 30.4414 % │
│ C │ 20.0 % │ 19.996 % │ 16.2577 % │
└─────────┴─────────┴───────────┴───────────┘On this data every opinion is close to the consensus in Kullback-Leibler divergence, so p = 0.1 barely moves the weights. At p = 100 weight moves from opinion C, the one farthest from the consensus, to the other two. To move the consensus mean on one short window, change the total of the credibility weights, as section 6 does.
8. A consensus portfolio
We maximise the risk-adjusted ratio under the empirical prior and under the pooled consensus, and compare the weights.
using Clarabelslv = Solver(; name = :clarabel1, solver = Clarabel.Optimizer, settings = Dict("verbose" => false), check_sol = (; allow_local = true, allow_almost = true))rf = 4.2 / 100 / 252res_emp = optimise(MeanRisk(; obj = MaximumRatio(; rf = rf), opt = JuMPOptimiser(; pe = pr_emp, slv = slv)))res_op = optimise(MeanRisk(; obj = MaximumRatio(; rf = rf), opt = JuMPOptimiser(; pe = pr_op, slv = slv)))pretty_table(DataFrame(["Assets" => rd.nx, "Empirical" => res_emp.w, "Pooled consensus" => res_op.w]); formatters = [resfmt], title = "Maximum-ratio weights under the empirical prior and the pooled consensus")Maximum-ratio weights under the empirical prior and the pooled consensus
┌────────┬───────────┬──────────────────┐
│ Assets │ Empirical │ Pooled consensus │
│ String │ Float64 │ Float64 │
├────────┼───────────┼──────────────────┤
│ AAPL │ 0.0 % │ 0.0 % │
│ AMD │ 0.0 % │ 0.0 % │
│ BAC │ 0.0 % │ 0.0 % │
│ BBY │ 0.0 % │ 0.0 % │
│ CVX │ 0.0 % │ 0.0 % │
│ GE │ 0.0 % │ 0.0 % │
│ HD │ 0.0 % │ 0.0 % │
│ JNJ │ 0.0 % │ 0.0 % │
│ JPM │ 0.0 % │ 0.0 % │
│ KO │ 0.0 % │ 0.0 % │
│ LLY │ 0.002 % │ 2.992 % │
│ MRK │ 65.977 % │ 64.458 % │
│ MSFT │ 0.0 % │ 0.0 % │
│ PEP │ 0.0 % │ 0.0 % │
│ PFE │ 0.0 % │ 0.0 % │
│ PG │ 0.0 % │ 0.0 % │
│ RRC │ 0.0 % │ 0.0 % │
│ UNH │ 0.0 % │ 0.0 % │
│ WMT │ 0.0 % │ 0.0 % │
│ XOM │ 34.02 % │ 32.55 % │
└────────┴───────────┴──────────────────┘The composition plot stacks the weights of each portfolio into one bar, the empirical portfolio first.
plot_stacked_bar_composition([res_emp, res_op], rd; xticks = (1:2, ["Empirical", "Pooled"]))
This page was generated using Literate.jl.