High Order Prior

PortfolioOptimisers.HighOrderPriorEstimator — Type
struct HighOrderPriorEstimator{__T_pe, __T_kte, __T_ske, __T_x_src} <: AbstractHighOrderPriorEstimator

High order prior estimator for asset returns.

HighOrderPriorEstimator holds a low order prior estimator, a cokurtosis estimator and a coskewness estimator. Its prior fits the mean and the covariance with the low order prior estimator. It then fits the square cokurtosis and the coskewness of the same returns, and returns all of them in a HighOrderPrior.

Fields

  • pe: Prior estimator.
  • kte: Cokurtosis estimator.
  • ske: Coskewness estimator.
  • x_src: The source of the returns that the prior fits on, nothing or an AbstractReturnsSource. nothing and DataSource read the asset returns rd.X. ExcessSource reads the returns in excess of the benchmark, rd.X - rd.B, the Excess Frame, and refuses returns data with no benchmark. nothing means that the caller stated no source, so an optimiser that holds a reader of the Excess Frame warns about a prior at nothing, and not about a prior at DataSource(). A prior reads no prior, so it refuses PriorSource. A prior inside another estimator, such as the pe of a FactorPrior, keeps nothing, and the outer estimator states the source.

Constructors

HighOrderPriorEstimator(;    pe::Onl{<:AbstractLowOrderPriorEstimator_A_F_AF} = EmpiricalPrior(),    kte::Option{<:CokurtosisEstimator} = Cokurtosis(;        alg = FullMoment()    ),    ske::Option{<:CoskewnessEstimator} = Coskewness(;        alg = FullMoment()    ),    x_src::Option{<:AbstractReturnsSource} = nothing) -> HighOrderPriorEstimator

Keywords correspond to the struct's fields.

Validation

Propagated parameters

factory passes its arguments on to these @fprop fields:

  • pe: Recursively updated through factory.
  • kte: Recursively updated through factory.
  • ske: Recursively updated through factory.

View parameters

port_opt_view takes the view of these @vprop fields at the selected indices:

Examples

julia> HighOrderPriorEstimator()HighOrderPriorEstimator     pe ┼ EmpiricalPrior        │           ce ┼ PortfolioOptimisersCovariance        │              │   ce ┼ Covariance        │              │      │    me ┼ SimpleExpectedReturns        │              │      │       │   w ┴ nothing        │              │      │    ce ┼ GeneralCovariance        │              │      │       │   ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true)        │              │      │       │    w ┴ nothing        │              │      │   alg ┼ FullMoment()        │              │      │     w ┴ nothing        │              │   mp ┼ MatrixProcessing        │              │      │     pdm ┼ Posdef        │              │      │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton        │              │      │         │   kwargs ┴ @NamedTuple{}: NamedTuple()        │              │      │      dn ┼ nothing        │              │      │      dt ┼ nothing        │              │      │     alg ┼ nothing        │              │      │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)        │           me ┼ SimpleExpectedReturns        │              │   w ┴ nothing        │      horizon ┼ nothing        │   fill_limit ┼ nothing        │        x_src ┴ nothing    kte ┼ Cokurtosis        │      me ┼ SimpleExpectedReturns        │         │   w ┴ nothing        │      mp ┼ MatrixProcessing        │         │     pdm ┼ Posdef        │         │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton        │         │         │   kwargs ┴ @NamedTuple{}: NamedTuple()        │         │      dn ┼ nothing        │         │      dt ┼ nothing        │         │     alg ┼ nothing        │         │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)        │     alg ┼ FullMoment()        │       w ┼ nothing        │   cache ┴ nothing    ske ┼ Coskewness        │      me ┼ SimpleExpectedReturns        │         │   w ┴ nothing        │      mp ┼ MatrixProcessing        │         │     pdm ┼ Posdef        │         │         │      alg ┼ UnionAll: NearestCorrelationMatrix.Newton        │         │         │   kwargs ┴ @NamedTuple{}: NamedTuple()        │         │      dn ┼ nothing        │         │      dt ┼ nothing        │         │     alg ┼ nothing        │         │   order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg)        │     alg ┼ FullMoment()        │       w ┼ nothing        │   cache ┴ nothing  x_src ┴ nothing

Related

References

  • [64] E. Jondeau and M. Rockinger. Optimal portfolio allocation under higher moments. Notes d'Études et de Recherche 108, Banque de France (2004). Section 2.3.
  • [66] D. Cajas. Convex Optimization of Portfolio Kurtosis. Available at SSRN 4202967 (2022). Appendix A.
source
PortfolioOptimisers.prior — Function
prior(pe::HighOrderPriorEstimator, X::MatNum, F::Option{<:MatNum} = nothing, pnl::Option{<:AssetPanel} = nothing; dims::Int = 1, kwargs...)

Compute the high order prior of the asset returns.

prior fits the mean and the covariance with the embedded prior estimator pe.pe. It then fits the square cokurtosis with pe.kte and the coskewness with pe.ske, and returns all of them in a HighOrderPrior. The method passes the factor returns F to pe.pe alone, and the co-moments read the asset returns.

Mathematical definition

The estimator computes the first and the second moments, the coskewness matrix $\mathbf{M}_3$ and the square cokurtosis matrix $\mathbf{\Sigma}_4$:

\[\begin{align} \mathbf{W} &= (\mathbf{Z} \otimes \boldsymbol{1}_N^\intercal) \odot (\boldsymbol{1}_N^\intercal \otimes \mathbf{Z})\,, \\ \mathbf{M}_3 &= \frac{1}{T} \mathbf{Z}^\intercal \mathbf{W}\,, \\ \mathbf{\Sigma}_4 &= \frac{1}{T} \mathbf{W}^\intercal \mathbf{W}\,. \end{align}\]

Where:

  • $\mathbf{Z}$: $T \times N$ matrix of returns, with the mean of each column removed.
  • $\mathbf{W}$: $T \times N^2$ matrix of the products of each pair of centred returns.
  • $\mathbf{M}_3$: $N \times N^2$ coskewness matrix, sk.
  • $\mathbf{\Sigma}_4$: $N^2 \times N^2$ square cokurtosis matrix, kt.
  • $\boldsymbol{1}_N$: $N \times 1$ vector of ones.
  • $T$: Number of observations.
  • $\otimes$: Kronecker product.
  • $\odot$: Hadamard product.

$\mathbf{\Sigma}_4$ is the square form, of size $N^2 \times N^2$. It is not the $N \times N^3$ cokurtosis matrix $\mathbf{M}_4$ of the same source. The library does not build $\mathbf{M}_4$.

pe.kte computes kt, and pe.ske computes sk. A different alg, such as a semi-moment or an exponentially weighted moment, gives a different formula in place of the displays above. V is the negative spectral coskewness of sk. When an estimator is nothing, the result holds nothing for its moment.

The low order block can give the probabilities $\mathbf{w}$ of its scenarios. An entropy pooling prior and an opinion pooling prior do this. Then the co-moments are those of the same distribution. $\mathbf{Z}$ is centred on the mean under $\mathbf{w}$, and each $\frac{1}{T}$ above becomes $\frac{1}{\boldsymbol{1}^\intercal \mathbf{w}} \mathrm{diag}(\mathbf{w})$ inside the product. $\mathbf{w}$ replaces the observation weights of pe.kte and pe.ske. When the low order block gives no $\mathbf{w}$, the observation weights do not change.

Algorithm

  1. Orient X and F to observations × variables with dims_oriented.
  2. Compute the low order block pr with pe.pe, and pass F to it.
  3. Read the probabilities w = pr.w of its scenarios. Cut X and pnl to the rows that w weighs, with scenario_rows.
  4. Compute the square cokurtosis kt with factory(pe.kte, w). A nothing estimator gives a nothing moment.
  5. Compute the coskewness sk and its negative spectral form V with factory(pe.ske, w). A nothing estimator gives nothing for both.
  6. Build the structure matrices at the asset count size(pr.X, 2) with dup_elim_sum_matrices. Take all three when steps 4 and 5 both gave a moment. Take L2 and S2 alone when step 4 gave a moment and step 5 did not. Take none in the other cases. D2 goes with sk, and L2 and S2 go with kt. The second case has no sk, and its D2 is nothing.
  7. Build the HighOrderPrior with its keyword constructor. When step 5 gave an sk, set skmp to the processor of pe.ske, from coskewness_processor. Each @argcheck of the constructor checks the shapes that steps 4 to 6 gave.
  8. Warn with assert_matched_coverage when the co-moments hold fewer assets than the low order block. Steps 6 to 8 are assemble_high_order_prior.

Arguments

  • pe: High order prior estimator.
  • X: Asset returns matrix (observations × assets).
  • F: Optional factor returns matrix (observations × factors).
  • pnl: Optional AssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.
  • dims: Dimension along which the computation runs.
  • kwargs...: Additional keyword arguments, passed to the estimators that pe holds.

Validation

Returns

  • pr::HighOrderPrior: Result that holds the low order prior result, the square cokurtosis, the coskewness, its negative spectral form and the structure matrices.

Related

source
PortfolioOptimisers.partial_fit! — Function
partial_fit!(pe::HighOrderPriorEstimator, X, F = nothing; kwargs...)

Fold observations into a HighOrderPriorEstimator, and pass them to its members.

The estimator keeps no buffer. It builds HighOrderPrior(; pr = pr, ...) around the result of its embedded prior. The rows that prior(pe) with no data needs are the rows that the embedded prior holds. A buffer in this estimator is a second copy of the sample.

The estimator always passes the fold to pe.pe, because that member holds the rows. An embedded prior that cannot fold refuses here, and names the wrapper that gives it a buffer. This refusal is correct, because the estimator has no rows of its own to give it. The factor observations go with the rows, as the batch verb passes F to the embedded prior. The tree of the embedded prior decides what to do with them. pe.kte and pe.ske go through fold_member. prior(pe) with no data has rows for them, and it refits a co-moment that cannot fold. So HighOrderPriorEstimator(; ske = Coskewness(; alg = SemiMoment())) needs no wrapper, and it keeps one copy of the sample.

Algorithm

  1. At the first step, when pe.pe holds no state, refuse a member that holds one, with assert_cold_carry_entry.
  2. Fold the observations and F into pe.pe with partial_fit!.
  3. Fold the observations into pe.kte and pe.ske with fold_member.
  4. Rebuild the estimator with the three folded members through rebuild_struct, and return it.

Arguments

  • pe: High order prior estimator.
  • X: Observations to fold, a matrix of rows or one observation.
  • F: The factor observations beside X, or nothing. The method forwards them to the embedded prior.
  • kwargs...: Additional keyword arguments, forwarded to the members.

Validation

  • pe.pe folds. Otherwise the embedded prior refuses the fold, with an error that names the wrapper that gives it a buffer.
  • At the first step, no member holds a partial-fit state. The embedded prior holds the rows, and it starts with no rows. So a co-moment that holds a state at this step reads rows that the embedded prior does not hold. An ArgumentError that names the path of the state is thrown otherwise, see assert_cold_carry_entry.

Returns

  • pe: The estimator, with its members folded.

Related

source
PortfolioOptimisers.prior — Method
prior(pe::HighOrderPriorEstimator; kwargs...)

Read a HighOrderPriorEstimator out of its fold, with no data matrix.

The method reads the embedded prior first. A co-moment that folded gives its value from its state. The method refits a co-moment that did not fold, over the rows that the embedded prior folded, which prior_returns_buffer reads. When the embedded result gives the probabilities w of its scenarios, the method refits both co-moments under w, folded or not, because a state cannot read w. assemble_high_order_prior then builds the result, as in the batch method.

The refit reads the folded rows, and not pr.X. pr.X holds the scenarios of the result of the embedded prior, and these are not always the folded rows. A Scenario Cap on the embedded prior keeps only the last rows. The scenario fill writes zeros into the first rows of an asset that starts to list late. The batch method fits both co-moments over every row that the caller passes, with the NaN entries. When the embedded result gives w, it fits them over the last length(pr.w) rows, with scenario_rows. So in both cases a refit over pr.X gives a different value. Under an Online window, the buffer holds the last max_history rows, and the equal batch fit of that route uses the same window. The estimator keeps no copy of the rows, because the embedded prior holds them.

Mathematical definition

\[\begin{align} \mathcal{P}_T &= \mathcal{P}(\mathbf{X})\,. \end{align}\]

Where:

  • $\mathcal{P}_T$: Folded prior, the result that the estimator's batch verb gives with no data after the estimator folds the observations $\boldsymbol{x}_1, \ldots, \boldsymbol{x}_T$ in order, one at a time or in blocks of any size.
  • $\mathcal{P}(\cdot)$: Batch prior, the result that the estimator's batch verb gives on the matrices in the parentheses.
  • $\mathbf{X}$: Returns matrix $T \times N$.

The equality holds for the embedded result and for every co-moment, both the co-moments that fold and the co-moments that the method refits. In floating point, each co-moment differs from the batch co-moment by rounding.

Algorithm

  1. Read the embedded result pr out of pe.pe with prior.
  2. Read the rows that the embedded prior folded, with prior_returns_buffer. Drop the rows of a Shared Series gap with shared_series_buffer, as the embedded prior and the members that fold drop them. A member that refits drops the fixed observation weights of the same rows, through buffer_refit_member. Read the rows that remain with sample_buffer, and call them X.
  3. Rebuild the Asset Panel of the rows from the same buffer with buffer_asset_panel, and call it pnl. The batch method gives rd.pnl to both co-moments in the same way.
  4. Read kt out of pe.kte with read_comoment under pr.w. It refits the member over X and pnl when the member did not fold, and under pr.w when pr.w is not nothing.
  5. Read sk and V out of pe.ske with read_comoment, in the same way.
  6. Build the result with assemble_high_order_prior.
  7. Record in the result the returns source that pe states, with stamp_returns_source, and return it.

Arguments

  • pe: High order prior estimator whose members hold states.
  • kwargs...: Additional keyword arguments, passed to the members.

Validation

Returns

  • hop::HighOrderPrior: Result that holds the low order prior result and the co-moments.

Related

source

References

[64]
E. Jondeau and M. Rockinger. Optimal portfolio allocation under higher moments. Notes d'Études et de Recherche 108 (Banque de France, 2004).
[66]
D. Cajas. Convex Optimization of Portfolio Kurtosis. Available at SSRN 4202967 (2022).