Opinion pooling Prior

PortfolioOptimisers.OpinionPoolingAlgorithm — Type
abstract type OpinionPoolingAlgorithm <: AbstractAlgorithm

Abstract supertype for opinion pooling algorithms.

A subtype of OpinionPoolingAlgorithm is the rule by which an OpinionPoolingPrior pools the scenario weights of its opinions into one consensus weight vector. compute_pooling dispatches on it.

Interfaces

To add a pooling rule, subtype OpinionPoolingAlgorithm and implement the following method:

Required method name

  • compute_pooling(alg::OpinionPoolingAlgorithm, ow::VecNum, pw::MatNum) -> StatsBase.AnalyticWeights: Aggregate the columns of pw into one consensus scenario-weight vector.

Arguments

  • alg: The concrete subtype instance.
  • ow: $K \times 1$ vector of opinion probabilities, summing to 1.
  • pw: $T \times K$ matrix whose column k holds expert k's scenario weights.

Returns

  • w::StatsBase.AnalyticWeights: $T \times 1$ consensus scenario weights, summing to 1. The type of the weights sets the bias correction of the covariance of the refit, see ScenarioProbabilities.

Examples

julia> struct MedianOpinionPooling <: PortfolioOptimisers.OpinionPoolingAlgorithm endjulia> function PortfolioOptimisers.compute_pooling(::MedianOpinionPooling, ow, pw)           w = vec(mapslices(PortfolioOptimisers.Statistics.median, pw; dims = 2))           return PortfolioOptimisers.StatsBase.aweights(w / sum(w))       endjulia> PortfolioOptimisers.compute_pooling(MedianOpinionPooling(), [0.5, 0.5],                                           [0.5 0.25; 0.25 0.25; 0.25 0.5])3-element AnalyticWeights{Float64, Float64, Vector{Float64}}: 0.375 0.25 0.375

Related

References

  • [160] F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017). Section 2. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [161] F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016). Section 6. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [162] C. Martini and J. Sprenger. Opinion Aggregation and Individual Expertise. In: Scientific Collaboration and Collective Knowledge (Oxford University Press, 2017). Section 3, Theorem 2.
source
PortfolioOptimisers.LinearOpinionPooling — Type
struct LinearOpinionPooling <: OpinionPoolingAlgorithm

Pools the opinions as a weighted arithmetic mean of their scenario weights.

The consensus weight of a scenario is the opinion-weighted average of the weights that the experts give it. So the pool keeps every scenario to which one expert gives a positive weight. The rule suits opinions that are independent and additive.

Mathematical definition

\[\begin{align} p_t^{*} &= \sum_{k=1}^{K} \alpha_k\, p_{tk}\,. \end{align}\]

Where:

  • $p_t^{*}$: Pooled weight of scenario $t$.
  • $\alpha_k$: Opinion probability of expert $k$.
  • $p_{tk}$: Scenario weight for scenario $t$ from expert $k$.
  • $K$: Number of opinions.

A sum of non-negative terms is zero only when every term is zero. So $p_t^{*} = 0$ only when every opinion of positive probability gives scenario $t$ a zero weight. The pool is a mixture of the opinions, and the entropy is concave, so the entropy of the pool is at least the opinion-weighted mean of the entropies of the opinions. It can be less than the entropy of the most dispersed opinion.

Related

References

  • [160] F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017). Section 2. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
source
PortfolioOptimisers.LogarithmicOpinionPooling — Type
struct LogarithmicOpinionPooling <: OpinionPoolingAlgorithm

Pools the opinions as a weighted geometric mean of their scenario weights, renormalised.

The result is the distribution $\boldsymbol{q}$ that minimises $\sum_{k} \alpha_k \mathrm{KL}(\boldsymbol{q} \,\|\, \boldsymbol{p}_k)$, the opinion-weighted Kullback-Leibler divergence from the pool to each opinion. The first-order condition of that problem gives the formula below. One opinion that gives a scenario a small weight makes the pooled weight of that scenario small, whatever weight the other opinions give it.

Mathematical definition

\[\begin{align} p_t^{*} &= \frac{\exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{tk}\right)}{\sum_{s=1}^{T} \exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{sk}\right)}\,. \end{align}\]

Where:

  • $p_t^{*}$: Pooled weight of scenario $t$.
  • $\alpha_k$: Opinion probability of expert $k$.
  • $p_{tk}$: Scenario weight for scenario $t$ from expert $k$.
  • $K$: Number of opinions.
  • $T$: Number of observations.

One $\log 0$ makes the exponent $-\infty$. So $p_t^{*} = 0$ when one opinion of positive probability gives scenario $t$ a zero weight. The product $\alpha_k \log p_{tk}$ has no value when both factors are zero, and the form above does not cover that case. The source defines the pool only for opinions that give every scenario a positive weight. The library also accepts a zero weight, with the result above.

Related

References

  • [161] F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016). Section 6. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [163] I. J. Good. Rational decisions. Journal of the Royal Statistical Society: Series B (Methodological) 14, 107–114 (1952).
source
PortfolioOptimisers.OpinionPoolingPrior — Type
struct OpinionPoolingPrior{__T_pes, __T_pe1, __T_pe2, __T_p, __T_w, __T_alg, __T_ex, __T_cache, __T_x_src} <: AbstractLowOrderPriorEstimator_AF

Opinion pooling prior estimator for asset returns.

OpinionPoolingPrior is a low order prior estimator. Each estimator in pes is one opinion, and each opinion gives a vector of observation weights. The estimator pools these vectors into one consensus vector under alg, and fits pe2 under it.

The opinions contribute observation weights alone. Every moment of the result comes from the refit of pe2 under the pooled weights. So pes holds entropy pooling estimators, because their results carry a w. A w of nothing gives every opinion the probability 1/length(pes). A p of nothing pools the opinion probabilities as given, and robust_probabilities does not penalise them.

Fields

  • pes: Vector of entropy pooling prior estimators, one for each opinion. Each one gives the observation weights of its opinion.
  • pe1: Prior estimator whose returns X the opinions and pe2 read, or nothing to read the returns as given.
  • pe2: Prior estimator that the pool fits under the pooled weights. Every moment of the result comes from it.
  • p: Penalty of robust opinion pooling, above zero, or nothing to pool the opinion weights as given. A larger value moves more weight to the opinions nearest the consensus.
  • w: Opinion probabilities, one for each estimator in pes, each in [0, 1] and with a sum of at most one, or nothing to give every opinion the same probability. A uniform prior over the observations takes the remainder 1 - sum(w).
  • ex: Parallel execution strategy.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it. factory keeps it unchanged, because a factory call resolves the configuration, not the sample. port_opt_view copies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone. obs_weights_view removes it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.
  • x_src: The source of the returns that the prior fits on, nothing or an AbstractReturnsSource. nothing and DataSource read the asset returns rd.X. ExcessSource reads the returns in excess of the benchmark, rd.X - rd.B, the Excess Frame, and refuses returns data with no benchmark. nothing means that the caller stated no source, so an optimiser that holds a reader of the Excess Frame warns about a prior at nothing, and not about a prior at DataSource(). A prior reads no prior, so it refuses PriorSource. A prior inside another estimator, such as the pe of a FactorPrior, keeps nothing, and the outer estimator states the source.

Constructors

OpinionPoolingPrior(;    pes::VecEP,    pe1::Option{<:AbstractLowOrderPriorEstimator_A_F_AF} = nothing,    pe2::AbstractLowOrderPriorEstimator_A_F_AF = EmpiricalPrior(),    p::Option{<:Number} = nothing,    w::Option{<:VecNum} = nothing,    alg::OpinionPoolingAlgorithm = LinearOpinionPooling(),    ex::FLoops.Transducers.Executor = FLoops.Transducers.ThreadedEx(),    cache::Option{<:AbstractPartialFitState} = nothing,    x_src::Option{<:AbstractReturnsSource} = nothing) -> OpinionPoolingPrior

Keywords correspond to the struct's fields. All arguments are validated for type and value consistency.

Validation

  • pes must be a non-empty vector of prior estimators.
  • If w is not nothing, !isempty(w), length(w) == length(pes), all(x -> 0 <= x <= 1, w), and sum(w) <= 1.
  • The last is an inequality on purpose. When sum(w) < 1, prior gives the remaining weight to a uniform prior over the observations. That prior is one more opinion. It takes a column of pw, and the pool penalises and pools it with the other opinions.
  • If p is not nothing, p > 0. The bound is strict, so p = 0 raises an error. Set p = nothing for no penalty.
  • x_src is nothing, DataSource() or ExcessSource(), see assert_returns_source.
  • The x_src of pes, pe1, pe2 is nothing, because the prior picks the returns that it reads, see assert_inner_reader.

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • pes: Recursively updated through factory.
  • pe1: Recursively updated through factory.
  • pe2: Recursively updated through factory.
  • cache: Carried unchanged through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Examples

julia> sets = UniverseSets(; xkey = "nx", dict = Dict("nx" => ["A", "B", "C"]));julia> OpinionPoolingPrior(;                           pes = [EntropyPoolingPrior(; sets = sets,                                                      mu_views = LinearConstraintEstimator(;                                                                                           val = ["A == 0.03",                                                                                                  "B + C == 0.04"])),                                  EntropyPoolingPrior(; sets = sets,                                                      mu_views = LinearConstraintEstimator(;                                                                                           val = ["A == 0.05",                                                                                                  "B + C >= 0.06"]))])OpinionPoolingPrior    pes ┼ 2-element Vector{EntropyPoolingPrior}        │ EntropyPoolingPrior ⋯        │ EntropyPoolingPrior ⋯    pe1 ┼ nothing    pe2 ┼ EmpiricalPrior        │           ce ┼ PortfolioOptimisersCovariance        │              │   ce ┼ Covariance        │              │      │    me ┼ SimpleExpectedReturns        │              │      │       │   w ┴ nothing        │              │      │    ce ┼ GeneralCovariance        │              │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)        │              │      │       │    w ┴ nothing        │              │      │   alg ┼ FullMoment()        │              │      │     w ┴ nothing        │              │   mp ┼ MatrixProcessing        │              │      │     pdm ┼ Posdef        │              │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton        │              │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()        │              │      │      dn ┼ nothing        │              │      │      dt ┼ nothing        │              │      │     alg ┼ nothing        │              │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)        │           me ┼ SimpleExpectedReturns        │              │   w ┴ nothing        │      horizon ┼ nothing        │   fill_limit ┼ nothing        │        x_src ┴ nothing      p ┼ nothing      w ┼ nothing    alg ┼ LinearOpinionPooling()     ex ┼ Transducers.ThreadedEx{@NamedTuple{}}: Transducers.ThreadedEx()  x_src ┴ nothing

The incremental fit

This prior has no exact incremental recursion, so it takes the online step by refitting from a sample buffer: Online seeds cache, partial_fit! appends each observation to it verbatim, and the one-argument prior runs this estimator's own batch verb over the rows the buffer kept. The answer is therefore exactly a batch fit over those rows, and a max_history on the wrapper windows the whole fit.

cache travels the three propagation channels as every partial-fit state does: factory carries it unchanged, port_opt_view slices it to the selected assets, and obs_weights_view drops it, because no slice of a state exists on the observation axis. It is not rendered, because a running buffer is not the configuration a reader looks the type up for.

Related

References

  • [160] F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017). Section 2. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [161] F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016). Section 6. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
  • [162] C. Martini and J. Sprenger. Opinion Aggregation and Individual Expertise. In: Scientific Collaboration and Collective Knowledge (Oxford University Press, 2017). Section 3, Theorem 2.
source
PortfolioOptimisers.compute_pooling — Function
compute_pooling(::LinearOpinionPooling, ow::VecNum, pw::MatNum)
compute_pooling(::LogarithmicOpinionPooling, ow::VecNum, pw::MatNum)

Pool the scenario weights of several opinions into one consensus weight vector.

Each column of pw holds the scenario weights of one opinion, and ow holds the probability of each opinion. The type of the first argument selects the pooling rule.

Mathematical definition

Let $\boldsymbol{\alpha}$ be the opinion probabilities and $\mathbf{P}$ the $T \times K$ matrix of scenario weights for $K$ experts.

Linear (weighted arithmetic mean):

\[\begin{align} \boldsymbol{p}^* &= \mathbf{P} \boldsymbol{\alpha}\,. \end{align}\]

Logarithmic (weighted geometric mean, normalised):

\[\begin{align} p_t^* &= \frac{\exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{tk}\right)}{\sum_{s=1}^{T} \exp\!\left(\sum_{k=1}^{K} \alpha_k \log p_{sk}\right)}\,. \end{align}\]

Where:

  • $\boldsymbol{p}^*$: $T \times 1$ pooled posterior weight vector.
  • $\mathbf{P}$: $T \times K$ matrix of scenario weights for $K$ experts.
  • $\boldsymbol{\alpha}$: $K \times 1$ opinion probability vector, summing to 1.
  • $p_{tk}$: Scenario weight for scenario $t$ from expert $k$.
  • $T$: Number of observations.

Algorithm

Under LinearOpinionPooling:

  1. Multiply pw by ow to get the consensus weights w.

Under LogarithmicOpinionPooling:

  1. Multiply the elementwise logarithm of pw by ow to get the exponent vector u.
  2. Compute lse = LogExpFunctions.logsumexp(u). The function subtracts the maximum of u before it exponentiates, so a very negative exponent does not underflow to a vector of zeros.
  3. Exponentiate u .- lse to get the consensus weights w.

Arguments

Validation

  • StatsBase.aweights checks that the result carries no Inf and no NaN. Under LogarithmicOpinionPooling, an opinion probability of exactly 0 against a scenario weight of exactly 0 makes the product 0 * log(0), which is NaN, and the call raises ArgumentError. A zero scenario weight alone is safe. It gives -Inf, which exponentiates to a consensus weight of 0.

Returns

  • w::StatsBase.AnalyticWeights: Consensus scenario weights. The logarithmic rule renormalises them to sum to 1, and the linear rule gives a sum of 1 when ow and every column of pw sum to 1. Their bias correction is the one of reliability weights, see ScenarioProbabilities.

Related

source
PortfolioOptimisers.prior — Function
prior(pe::OpinionPoolingPrior, X::MatNum, F::Option{<:MatNum} = nothing,
      pnl::Option{<:AssetPanel} = nothing;
      dims::Int = 1, strict::Bool = false, cache_reg::Bool = false, kwargs...)

Compute opinion pooling prior moments for asset returns.

prior fits every opinion of pe.pes, pools their observation weights into one consensus vector, and fits pe.pe2 under that vector.

prior changes no field of pe, so two calls on one estimator give the same answer. Every moment of the result comes from pe.pe2. The opinions contribute observation weights alone.

Algorithm

  1. Orient X and F by dims.
  2. When pe.pe1 is not nothing, replace X with the returns of that estimator's prior.
  3. Pool the opinions of pe.pes over X with opinion_pooling_weights, which returns the consensus weights w, the penalised opinion probabilities ow and the weights pw of each opinion.
  4. Refit pe.pe2 under w through factory. The moments of the result come from this refit. Check that the refit answered on the observation axis of the pool, see # Validation.
  5. Compute ens as exp of the entropy of w, and kld as the divergence from w to each column of pw.
  6. When cache_reg is true, store pc = (; X, w, pe2). X holds the rows that the opinions read, or nothing when they read the X argument. pe2 holds the cache of the result of pe.pe2. When cache_reg is false, pc is nothing.

The fit of pe.pe2 receives cache_reg too, so a prior there stores its own cache. The refit of an entropy pooling prior around the pool reads both caches, and runs no opinion and no pe.pe1.

Arguments

  • pe: Opinion pooling prior estimator.
  • X: Asset returns matrix, observations × assets.
  • F: Optional factor returns matrix.
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it. It goes to pe.pe1, to every opinion in pe.pes, and to the refit pe.pe2.
  • dims: Dimension along which the computation runs.
  • strict: Passed to pe.pe1, to every opinion and to pe.pe2. If true, an opinion with a view that names an asset that its sets does not hold raises an error. If false, the opinion drops that view with a warning. Default is false.
  • cache_reg: If true, a prior stores on its result the cache that its refit under new scenario weights reads: a CrossSectionalFactorPrior the active mask and the Return Forecast history on its block, and an OpinionPoolingPrior the rows of pe1, its pooled weights and the cache of pe2 in pc. The refit of an entropy pooling prior around it then runs no regression and no opinion. Every other prior forwards it to the prior that it wraps. Default is false.
  • kwargs...: More keyword arguments, passed to pe.pe1, to every opinion and to pe.pe2.

Validation

  • dims in (1, 2).
  • Every opinion answers on the observation axis of the pool. That axis is the rows of the result of pe.pe1 when pe.pe1 is set, and X when it is nothing. An opinion whose own wrapped estimator drops rows answers on a shorter axis, so its probabilities are over other scenarios. opinion_pooling_weights then raises a DimensionMismatch. The message names the opinion, both counts and the fix, which is to move the estimator that drops rows into pe1.
  • The refit pe.pe2 answers on the observation axis of the pool too, because the pooled weights are over that axis. A pe2 that drops rows, such as one that holds a Scenario Cap, answers on a shorter axis. prior then raises a DimensionMismatch that names pe2, both counts and the same fix.

Returns

  • pr::LowOrderPrior: The moments of the refit pe.pe2, with the consensus weights w, their diagnostics ens, kld and ow, and the cache pc.
  • pr.ens: exp of the entropy of the consensus weights. It is 1 when all the mass is on one observation, and T when the weights are uniform.
  • pr.kld[i]: $\mathrm{KL}(\boldsymbol{w} \,\|\, \boldsymbol{p}_i)$, from the consensus to opinion i. robust_probabilities reads the divergence in the other direction, so the two vectors are different numbers.
  • pr.ow: The penalised opinion probabilities, one entry per column of pw. It is one entry longer than pe.w when the uniform-prior remainder took a column.

Related

source

References

[160]
F. Dietrich and C. List. Probabilistic opinion pooling generalized. Part one: general agendas. Social Choice and Welfare 48, 747–786 (2017).
[161]
F. Dietrich and C. List. Probabilistic Opinion Pooling. In: The Oxford Handbook of Probability and Philosophy, edited by A. Hájek and C. Hitchcock (Oxford University Press, 2016).
[162]
C. Martini and J. Sprenger. Opinion Aggregation and Individual Expertise. In: Scientific Collaboration and Collective Knowledge (Oxford University Press, 2017).
[163]