Opinion pooling Prior
PortfolioOptimisers.OpinionPoolingAlgorithm — Type
abstract type OpinionPoolingAlgorithm <: AbstractAlgorithmAbstract supertype for opinion pooling algorithms.
A subtype of OpinionPoolingAlgorithm is the rule by which an OpinionPoolingPrior pools the scenario weights of its opinions into one consensus weight vector. compute_pooling dispatches on it.
Interfaces
To add a pooling rule, subtype OpinionPoolingAlgorithm and implement the following method:
Required method name
compute_pooling(alg::OpinionPoolingAlgorithm, ow::VecNum, pw::MatNum) -> StatsBase.AnalyticWeights: Aggregate the columns ofpwinto one consensus scenario-weight vector.
Arguments
alg: The concrete subtype instance.ow: $K \times 1$ vector of opinion probabilities, summing to 1.pw: $T \times K$ matrix whose columnkholds expertk's scenario weights.
Returns
w::StatsBase.AnalyticWeights: $T \times 1$ consensus scenario weights, summing to 1. The type of the weights sets the bias correction of the covariance of the refit, seeScenarioProbabilities.
Examples
julia> struct MedianOpinionPooling <: PortfolioOptimisers.OpinionPoolingAlgorithm endjulia> function PortfolioOptimisers.compute_pooling(::MedianOpinionPooling, ow, pw) w = vec(mapslices(PortfolioOptimisers.Statistics.median, pw; dims = 2)) return PortfolioOptimisers.StatsBase.aweights(w / sum(w)) endjulia> PortfolioOptimisers.compute_pooling(MedianOpinionPooling(), [0.5, 0.5], [0.5 0.25; 0.25 0.25; 0.25 0.5])3-element AnalyticWeights{Float64, Float64, Vector{Float64}}: 0.375 0.25 0.375Related
References
- [160] F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017). Section 2. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [161] F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016). Section 6. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [162] C. Martini and J. Sprenger. Opinion Aggregation and Individual Expertise. In: Scientific Collaboration and Collective Knowledge (Oxford University Press, 2017). Section 3, Theorem 2.
PortfolioOptimisers.LinearOpinionPooling — Type
struct LinearOpinionPooling <: OpinionPoolingAlgorithmPools the opinions as a weighted arithmetic mean of their scenario weights.
The consensus weight of a scenario is the opinion-weighted average of the weights that the experts give it. So the pool keeps every scenario to which one expert gives a positive weight. The rule suits opinions that are independent and additive.
Mathematical definition
\[\begin{align} p_t^{*} &= \sum_{k=1}^{K} \alpha_k\, p_{tk}\,. \end{align}\]
Where:
- $p_t^{*}$: Pooled weight of scenario $t$.
- $\alpha_k$: Opinion probability of expert $k$.
- $p_{tk}$: Scenario weight for scenario $t$ from expert $k$.
- $K$: Number of opinions.
A sum of non-negative terms is zero only when every term is zero. So $p_t^{*} = 0$ only when every opinion of positive probability gives scenario $t$ a zero weight. The pool is a mixture of the opinions, and the entropy is concave, so the entropy of the pool is at least the opinion-weighted mean of the entropies of the opinions. It can be less than the entropy of the most dispersed opinion.
Related
OpinionPoolingAlgorithmLogarithmicOpinionPooling: the sibling rule, under which one opinion that gives a scenario a zero weight sets its pooled weight to zero.OpinionPoolingPriorcompute_pooling
References
- [160] F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017). Section 2. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.LogarithmicOpinionPooling — Type
struct LogarithmicOpinionPooling <: OpinionPoolingAlgorithmPools the opinions as a weighted geometric mean of their scenario weights, renormalised.
The result is the distribution $\boldsymbol{q}$ that minimises $\sum_{k} \alpha_k \mathrm{KL}(\boldsymbol{q} \,\|\, \boldsymbol{p}_k)$, the opinion-weighted Kullback-Leibler divergence from the pool to each opinion. The first-order condition of that problem gives the formula below. One opinion that gives a scenario a small weight makes the pooled weight of that scenario small, whatever weight the other opinions give it.
Mathematical definition
\[\begin{align} p_t^{*} &= \frac{\exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{tk}\right)}{\sum_{s=1}^{T} \exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{sk}\right)}\,. \end{align}\]
Where:
- $p_t^{*}$: Pooled weight of scenario $t$.
- $\alpha_k$: Opinion probability of expert $k$.
- $p_{tk}$: Scenario weight for scenario $t$ from expert $k$.
- $K$: Number of opinions.
- $T$: Number of observations.
One $\log 0$ makes the exponent $-\infty$. So $p_t^{*} = 0$ when one opinion of positive probability gives scenario $t$ a zero weight. The product $\alpha_k \log p_{tk}$ has no value when both factors are zero, and the form above does not cover that case. The source defines the pool only for opinions that give every scenario a positive weight. The library also accepts a zero weight, with the result above.
Related
OpinionPoolingAlgorithmLinearOpinionPooling: the sibling rule, under which the pooled weight of a scenario is zero only when every opinion gives it a zero weight.OpinionPoolingPriorcompute_pooling: its# Validationsection states what the code does when both factors of the product are zero.
References
- [161] F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016). Section 6. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [163] I. J. Good. Rational decisions. Journal of the Royal Statistical Society: Series B (Methodological) 14, 107–114 (1952).
PortfolioOptimisers.OpinionPoolingPrior — Type
struct OpinionPoolingPrior{__T_pes, __T_pe1, __T_pe2, __T_p, __T_w, __T_alg, __T_ex, __T_cache, __T_x_src} <: AbstractLowOrderPriorEstimator_AFOpinion pooling prior estimator for asset returns.
OpinionPoolingPrior is a low order prior estimator. Each estimator in pes is one opinion, and each opinion gives a vector of observation weights. The estimator pools these vectors into one consensus vector under alg, and fits pe2 under it.
The opinions contribute observation weights alone. Every moment of the result comes from the refit of pe2 under the pooled weights. So pes holds entropy pooling estimators, because their results carry a w. A w of nothing gives every opinion the probability 1/length(pes). A p of nothing pools the opinion probabilities as given, and robust_probabilities does not penalise them.
Fields
pes: Vector of entropy pooling prior estimators, one for each opinion. Each one gives the observation weights of its opinion.
pe1: Prior estimator whose returnsXthe opinions andpe2read, ornothingto read the returns as given.
pe2: Prior estimator that the pool fits under the pooled weights. Every moment of the result comes from it.
p: Penalty of robust opinion pooling, above zero, ornothingto pool the opinion weights as given. A larger value moves more weight to the opinions nearest the consensus.
w: Opinion probabilities, one for each estimator inpes, each in[0, 1]and with a sum of at most one, ornothingto give every opinion the same probability. A uniform prior over the observations takes the remainder1 - sum(w).
alg: The rule that pools the scenario weights of the opinions, seeOpinionPoolingAlgorithm.
ex: Parallel execution strategy.
cache: Optional partial-fit state. It isnothinguntilpartial_fit!writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it.factorykeeps it unchanged, because a factory call resolves the configuration, not the sample.port_opt_viewcopies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone.obs_weights_viewremoves it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.
x_src: The source of the returns that the prior fits on,nothingor anAbstractReturnsSource.nothingandDataSourceread the asset returnsrd.X.ExcessSourcereads the returns in excess of the benchmark,rd.X - rd.B, the Excess Frame, and refuses returns data with no benchmark.nothingmeans that the caller stated no source, so an optimiser that holds a reader of the Excess Frame warns about a prior atnothing, and not about a prior atDataSource(). A prior reads no prior, so it refusesPriorSource. A prior inside another estimator, such as thepeof aFactorPrior, keepsnothing, and the outer estimator states the source.
Constructors
OpinionPoolingPrior(; pes::VecEP, pe1::Option{<:AbstractLowOrderPriorEstimator_A_F_AF} = nothing, pe2::AbstractLowOrderPriorEstimator_A_F_AF = EmpiricalPrior(), p::Option{<:Number} = nothing, w::Option{<:VecNum} = nothing, alg::OpinionPoolingAlgorithm = LinearOpinionPooling(), ex::FLoops.Transducers.Executor = FLoops.Transducers.ThreadedEx(), cache::Option{<:AbstractPartialFitState} = nothing, x_src::Option{<:AbstractReturnsSource} = nothing) -> OpinionPoolingPriorKeywords correspond to the struct's fields. All arguments are validated for type and value consistency.
Validation
pesmust be a non-empty vector of prior estimators.- If
wis notnothing,!isempty(w),length(w) == length(pes),all(x -> 0 <= x <= 1, w), andsum(w) <= 1. - The last is an inequality on purpose. When
sum(w) < 1,priorgives the remaining weight to a uniform prior over the observations. That prior is one more opinion. It takes a column ofpw, and the pool penalises and pools it with the other opinions. - If
pis notnothing,p > 0. The bound is strict, sop = 0raises an error. Setp = nothingfor no penalty. x_srcisnothing,DataSource()orExcessSource(), seeassert_returns_source.- The
x_srcofpes,pe1,pe2isnothing, because the prior picks the returns that it reads, seeassert_inner_reader.
Propagated parameters
factory passes its arguments on to these @fprop fields:
pes: Recursively updated throughfactory.pe1: Recursively updated throughfactory.pe2: Recursively updated throughfactory.cache: Carried unchanged throughfactory.
View parameters
port_opt_view takes the view of these @vprop fields at the selected indices:
pes: Recursively viewed throughport_opt_view.pe1: Recursively viewed throughport_opt_view.pe2: Recursively viewed throughport_opt_view.cache: Sliced to the selected assets throughport_opt_view.
Examples
julia> sets = UniverseSets(; xkey = "nx", dict = Dict("nx" => ["A", "B", "C"]));julia> OpinionPoolingPrior(; pes = [EntropyPoolingPrior(; sets = sets, mu_views = LinearConstraintEstimator(; val = ["A == 0.03", "B + C == 0.04"])), EntropyPoolingPrior(; sets = sets, mu_views = LinearConstraintEstimator(; val = ["A == 0.05", "B + C >= 0.06"]))])OpinionPoolingPrior pes ┼ 2-element Vector{EntropyPoolingPrior} │ EntropyPoolingPrior ⋯ │ EntropyPoolingPrior ⋯ pe1 ┼ nothing pe2 ┼ EmpiricalPrior │ ce ┼ PortfolioOptimisersCovariance │ │ ce ┼ Covariance │ │ │ me ┼ SimpleExpectedReturns │ │ │ │ w ┴ nothing │ │ │ ce ┼ GeneralCovariance │ │ │ │ ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true) │ │ │ │ w ┴ nothing │ │ │ alg ┼ FullMoment() │ │ │ w ┴ nothing │ │ mp ┼ MatrixProcessing │ │ │ pdm ┼ Posdef │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple() │ │ │ dn ┼ nothing │ │ │ dt ┼ nothing │ │ │ alg ┼ nothing │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg) │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ horizon ┼ nothing │ fill_limit ┼ nothing │ x_src ┴ nothing p ┼ nothing w ┼ nothing alg ┼ LinearOpinionPooling() ex ┼ Transducers.ThreadedEx{@NamedTuple{}}: Transducers.ThreadedEx() x_src ┴ nothingThe incremental fit
This prior has no exact incremental recursion, so it takes the online step by refitting from a sample buffer: Online seeds cache, partial_fit! appends each observation to it verbatim, and the one-argument prior runs this estimator's own batch verb over the rows the buffer kept. The answer is therefore exactly a batch fit over those rows, and a max_history on the wrapper windows the whole fit.
cache travels the three propagation channels as every partial-fit state does: factory carries it unchanged, port_opt_view slices it to the selected assets, and obs_weights_view drops it, because no slice of a state exists on the observation axis. It is not rendered, because a running buffer is not the configuration a reader looks the type up for.
Related
OpinionPoolingAlgorithmLinearOpinionPoolingLogarithmicOpinionPoolingpriorrobust_probabilitiescompute_poolingfactoryport_opt_viewobs_weights_view
References
- [160] F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017). Section 2. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [161] F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016). Section 6. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
- [162] C. Martini and J. Sprenger. Opinion Aggregation and Individual Expertise. In: Scientific Collaboration and Collective Knowledge (Oxford University Press, 2017). Section 3, Theorem 2.
PortfolioOptimisers.compute_pooling — Function
compute_pooling(::LinearOpinionPooling, ow::VecNum, pw::MatNum)
compute_pooling(::LogarithmicOpinionPooling, ow::VecNum, pw::MatNum)Pool the scenario weights of several opinions into one consensus weight vector.
Each column of pw holds the scenario weights of one opinion, and ow holds the probability of each opinion. The type of the first argument selects the pooling rule.
Mathematical definition
Let $\boldsymbol{\alpha}$ be the opinion probabilities and $\mathbf{P}$ the $T \times K$ matrix of scenario weights for $K$ experts.
Linear (weighted arithmetic mean):
\[\begin{align} \boldsymbol{p}^* &= \mathbf{P} \boldsymbol{\alpha}\,. \end{align}\]
Logarithmic (weighted geometric mean, normalised):
\[\begin{align} p_t^* &= \frac{\exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{tk}\right)}{\sum_{s=1}^{T} \exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{sk}\right)}\,. \end{align}\]
Where:
- $\boldsymbol{p}^*$: $T \times 1$ pooled posterior weight vector.
- $\mathbf{P}$: $T \times K$ matrix of scenario weights for $K$ experts.
- $\boldsymbol{\alpha}$: $K \times 1$ opinion probability vector, summing to 1.
- $p_{tk}$: Scenario weight for scenario $t$ from expert $k$.
- $T$: Number of observations.
Algorithm
Under LinearOpinionPooling:
- Multiply
pwbyowto get the consensus weightsw.
Under LogarithmicOpinionPooling:
- Multiply the elementwise logarithm of
pwbyowto get the exponent vectoru. - Compute
lse = LogExpFunctions.logsumexp(u). The function subtracts the maximum ofubefore it exponentiates, so a very negative exponent does not underflow to a vector of zeros. - Exponentiate
u .- lseto get the consensus weightsw.
Arguments
alg: Opinion pooling algorithm,LinearOpinionPoolingorLogarithmicOpinionPooling.ow: Opinion probabilities, one entry per opinion.pw: Scenario weights of the opinions,observations × opinions, one column per opinion.
Validation
StatsBase.aweightschecks that the result carries noInfand noNaN. UnderLogarithmicOpinionPooling, an opinion probability of exactly0against a scenario weight of exactly0makes the product0 * log(0), which isNaN, and the call raisesArgumentError. A zero scenario weight alone is safe. It gives-Inf, which exponentiates to a consensus weight of0.
Returns
w::StatsBase.AnalyticWeights: Consensus scenario weights. The logarithmic rule renormalises them to sum to 1, and the linear rule gives a sum of 1 whenowand every column ofpwsum to 1. Their bias correction is the one of reliability weights, seeScenarioProbabilities.
Related
OpinionPoolingPrior: the estimator that calls this function, throughopinion_pooling_weights.LinearOpinionPoolingLogarithmicOpinionPooling
PortfolioOptimisers.prior — Function
prior(pe::OpinionPoolingPrior, X::MatNum, F::Option{<:MatNum} = nothing,
pnl::Option{<:AssetPanel} = nothing;
dims::Int = 1, strict::Bool = false, cache_reg::Bool = false, kwargs...)Compute opinion pooling prior moments for asset returns.
prior fits every opinion of pe.pes, pools their observation weights into one consensus vector, and fits pe.pe2 under that vector.
prior changes no field of pe, so two calls on one estimator give the same answer. Every moment of the result comes from pe.pe2. The opinions contribute observation weights alone.
Algorithm
- Orient
XandFbydims. - When
pe.pe1is notnothing, replaceXwith the returns of that estimator's prior. - Pool the opinions of
pe.pesoverXwithopinion_pooling_weights, which returns the consensus weightsw, the penalised opinion probabilitiesowand the weightspwof each opinion. - Refit
pe.pe2underwthroughfactory. The moments of the result come from this refit. Check that the refit answered on the observation axis of the pool, see# Validation. - Compute
ensasexpof the entropy ofw, andkldas the divergence fromwto each column ofpw. - When
cache_registrue, storepc = (; X, w, pe2).Xholds the rows that the opinions read, ornothingwhen they read theXargument.pe2holds the cache of the result ofpe.pe2. Whencache_regisfalse,pcisnothing.
The fit of pe.pe2 receives cache_reg too, so a prior there stores its own cache. The refit of an entropy pooling prior around the pool reads both caches, and runs no opinion and no pe.pe1.
Arguments
pe: Opinion pooling prior estimator.X: Asset returns matrix,observations × assets.F: Optional factor returns matrix.pnl: OptionalAssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it. It goes tope.pe1, to every opinion inpe.pes, and to the refitpe.pe2.dims: Dimension along which the computation runs.strict: Passed tope.pe1, to every opinion and tope.pe2. Iftrue, an opinion with a view that names an asset that itssetsdoes not hold raises an error. Iffalse, the opinion drops that view with a warning. Default isfalse.cache_reg: Iftrue, a prior stores on its result the cache that its refit under new scenario weights reads: aCrossSectionalFactorPriorthe active mask and the Return Forecast history on its block, and anOpinionPoolingPriorthe rows ofpe1, its pooled weights and the cache ofpe2inpc. The refit of an entropy pooling prior around it then runs no regression and no opinion. Every other prior forwards it to the prior that it wraps. Default isfalse.kwargs...: More keyword arguments, passed tope.pe1, to every opinion and tope.pe2.
Validation
dims in (1, 2).- Every opinion answers on the observation axis of the pool. That axis is the rows of the result of
pe.pe1whenpe.pe1is set, andXwhen it isnothing. An opinion whose own wrapped estimator drops rows answers on a shorter axis, so its probabilities are over other scenarios.opinion_pooling_weightsthen raises aDimensionMismatch. The message names the opinion, both counts and the fix, which is to move the estimator that drops rows intope1. - The refit
pe.pe2answers on the observation axis of the pool too, because the pooled weights are over that axis. Ape2that drops rows, such as one that holds a Scenario Cap, answers on a shorter axis.priorthen raises aDimensionMismatchthat namespe2, both counts and the same fix.
Returns
pr::LowOrderPrior: The moments of the refitpe.pe2, with the consensus weightsw, their diagnosticsens,kldandow, and the cachepc.pr.ens:expof the entropy of the consensus weights. It is 1 when all the mass is on one observation, andTwhen the weights are uniform.pr.kld[i]: $\mathrm{KL}(\boldsymbol{w} \,\|\, \boldsymbol{p}_i)$, from the consensus to opinioni.robust_probabilitiesreads the divergence in the other direction, so the two vectors are different numbers.pr.ow: The penalised opinion probabilities, one entry per column ofpw. It is one entry longer thanpe.wwhen the uniform-prior remainder took a column.
Related
References
- [160]
- F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017).
- [161]
- F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016).
- [162]
- C. Martini and J. Sprenger. Opinion Aggregation and Individual Expertise. In: Scientific Collaboration and Collective Knowledge (Oxford University Press, 2017).
- [163]
- I. J. Good. Rational decisions. Journal of the Royal Statistical Society: Series B (Methodological) 14, 107–114 (1952).