High Order Prior
PortfolioOptimisers.HighOrderPriorEstimator — Type
struct HighOrderPriorEstimator{__T_pe, __T_kte, __T_ske, __T_x_src} <: AbstractHighOrderPriorEstimatorHigh order prior estimator for asset returns.
HighOrderPriorEstimator holds a low order prior estimator, a cokurtosis estimator and a coskewness estimator. Its prior fits the mean and the covariance with the low order prior estimator. It then fits the square cokurtosis and the coskewness of the same returns, and returns all of them in a HighOrderPrior.
Fields
pe: Prior estimator.
kte: Cokurtosis estimator.
ske: Coskewness estimator.
x_src: The source of the returns that the prior fits on,nothingor anAbstractReturnsSource.nothingandDataSourceread the asset returnsrd.X.ExcessSourcereads the returns in excess of the benchmark,rd.X - rd.B, the Excess Frame, and refuses returns data with no benchmark.nothingmeans that the caller stated no source, so an optimiser that holds a reader of the Excess Frame warns about a prior atnothing, and not about a prior atDataSource(). A prior reads no prior, so it refusesPriorSource. A prior inside another estimator, such as thepeof aFactorPrior, keepsnothing, and the outer estimator states the source.
Constructors
HighOrderPriorEstimator(; pe::Onl{<:AbstractLowOrderPriorEstimator_A_F_AF} = EmpiricalPrior(), kte::Option{<:CokurtosisEstimator} = Cokurtosis(; alg = FullMoment() ), ske::Option{<:CoskewnessEstimator} = Coskewness(; alg = FullMoment() ), x_src::Option{<:AbstractReturnsSource} = nothing) -> HighOrderPriorEstimatorKeywords correspond to the struct's fields.
Validation
x_srcisnothing,DataSource()orExcessSource(), seeassert_returns_source.- The
x_srcofpeisnothing, because the prior picks the returns that it reads, seeassert_inner_reader.
Propagated parameters
factory passes its arguments on to these @fprop fields:
pe: Recursively updated throughfactory.kte: Recursively updated throughfactory.ske: Recursively updated throughfactory.
View parameters
port_opt_view takes the view of these @vprop fields at the selected indices:
pe: Recursively viewed throughport_opt_view.kte: Recursively viewed throughport_opt_view.ske: Recursively viewed throughport_opt_view.
Examples
julia> HighOrderPriorEstimator()HighOrderPriorEstimator pe ┼ EmpiricalPrior │ ce ┼ PortfolioOptimisersCovariance │ │ ce ┼ Covariance │ │ │ me ┼ SimpleExpectedReturns │ │ │ │ w ┴ nothing │ │ │ ce ┼ GeneralCovariance │ │ │ │ ce ┼ StatsBase.SimpleCovariance: StatsBase.SimpleCovariance(true) │ │ │ │ w ┴ nothing │ │ │ alg ┼ FullMoment() │ │ │ w ┴ nothing │ │ mp ┼ MatrixProcessing │ │ │ pdm ┼ Posdef │ │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple() │ │ │ dn ┼ nothing │ │ │ dt ┼ nothing │ │ │ alg ┼ nothing │ │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg) │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ horizon ┼ nothing │ fill_limit ┼ nothing │ x_src ┴ nothing kte ┼ Cokurtosis │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ mp ┼ MatrixProcessing │ │ pdm ┼ Posdef │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple() │ │ dn ┼ nothing │ │ dt ┼ nothing │ │ alg ┼ nothing │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg) │ alg ┼ FullMoment() │ w ┼ nothing │ cache ┴ nothing ske ┼ Coskewness │ me ┼ SimpleExpectedReturns │ │ w ┴ nothing │ mp ┼ MatrixProcessing │ │ pdm ┼ Posdef │ │ │ alg ┼ UnionAll: NearestCorrelationMatrix.Newton │ │ │ kwargs ┴ @NamedTuple{}: NamedTuple() │ │ dn ┼ nothing │ │ dt ┼ nothing │ │ alg ┼ nothing │ │ order ┴ NTuple{4, Symbol}: (:pdm, :dn, :dt, :alg) │ alg ┼ FullMoment() │ w ┼ nothing │ cache ┴ nothing x_src ┴ nothingRelated
AbstractHighOrderPriorEstimatorAbstractLowOrderPriorEstimator_A_F_AFHighOrderPriorEmpiricalPriorCokurtosisEstimatorCoskewnessEstimatorCokurtosisCoskewnesspriorfactoryport_opt_view
References
PortfolioOptimisers.prior — Function
prior(pe::HighOrderPriorEstimator, X::MatNum, F::Option{<:MatNum} = nothing, pnl::Option{<:AssetPanel} = nothing; dims::Int = 1, kwargs...)Compute the high order prior of the asset returns.
prior fits the mean and the covariance with the embedded prior estimator pe.pe. It then fits the square cokurtosis with pe.kte and the coskewness with pe.ske, and returns all of them in a HighOrderPrior. The method passes the factor returns F to pe.pe alone, and the co-moments read the asset returns.
Mathematical definition
The estimator computes the first and the second moments, the coskewness matrix $\mathbf{M}_3$ and the square cokurtosis matrix $\mathbf{\Sigma}_4$:
\[\begin{align} \mathbf{W} &= (\mathbf{Z} \otimes \boldsymbol{1}_N^\intercal) \odot (\boldsymbol{1}_N^\intercal \otimes \mathbf{Z})\,, \\ \mathbf{M}_3 &= \frac{1}{T} \mathbf{Z}^\intercal \mathbf{W}\,, \\ \mathbf{\Sigma}_4 &= \frac{1}{T} \mathbf{W}^\intercal \mathbf{W}\,. \end{align}\]
Where:
- $\mathbf{Z}$: $T \times N$ matrix of returns, with the mean of each column removed.
- $\mathbf{W}$: $T \times N^2$ matrix of the products of each pair of centred returns.
- $\mathbf{M}_3$: $N \times N^2$ coskewness matrix,
sk. - $\mathbf{\Sigma}_4$: $N^2 \times N^2$ square cokurtosis matrix,
kt. - $\boldsymbol{1}_N$: $N \times 1$ vector of ones.
- $T$: Number of observations.
- $\otimes$: Kronecker product.
- $\odot$: Hadamard product.
$\mathbf{\Sigma}_4$ is the square form, of size $N^2 \times N^2$. It is not the $N \times N^3$ cokurtosis matrix $\mathbf{M}_4$ of the same source. The library does not build $\mathbf{M}_4$.
pe.kte computes kt, and pe.ske computes sk. A different alg, such as a semi-moment or an exponentially weighted moment, gives a different formula in place of the displays above. V is the negative spectral coskewness of sk. When an estimator is nothing, the result holds nothing for its moment.
The low order block can give the probabilities $\mathbf{w}$ of its scenarios. An entropy pooling prior and an opinion pooling prior do this. Then the co-moments are those of the same distribution. $\mathbf{Z}$ is centred on the mean under $\mathbf{w}$, and each $\frac{1}{T}$ above becomes $\frac{1}{\boldsymbol{1}^\intercal \mathbf{w}} \mathrm{diag}(\mathbf{w})$ inside the product. $\mathbf{w}$ replaces the observation weights of pe.kte and pe.ske. When the low order block gives no $\mathbf{w}$, the observation weights do not change.
Algorithm
- Orient
XandFtoobservations × variableswithdims_oriented. - Compute the low order block
prwithpe.pe, and passFto it. - Read the probabilities
w = pr.wof its scenarios. CutXandpnlto the rows thatwweighs, withscenario_rows. - Compute the square cokurtosis
ktwithfactory(pe.kte, w). Anothingestimator gives anothingmoment. - Compute the coskewness
skand its negative spectral formVwithfactory(pe.ske, w). Anothingestimator givesnothingfor both. - Build the structure matrices at the asset count
size(pr.X, 2)withdup_elim_sum_matrices. Take all three when steps 4 and 5 both gave a moment. TakeL2andS2alone when step 4 gave a moment and step 5 did not. Take none in the other cases.D2goes withsk, andL2andS2go withkt. The second case has nosk, and itsD2isnothing. - Build the
HighOrderPriorwith its keyword constructor. When step 5 gave ansk, setskmpto the processor ofpe.ske, fromcoskewness_processor. Each@argcheckof the constructor checks the shapes that steps 4 to 6 gave. - Warn with
assert_matched_coveragewhen the co-moments hold fewer assets than the low order block. Steps 6 to 8 areassemble_high_order_prior.
Arguments
pe: High order prior estimator.X: Asset returns matrix (observations × assets).F: Optional factor returns matrix (observations × factors).pnl: OptionalAssetPanel, the panel that the returns data held. A wrapping prior forwards it unchanged, so that it can compose an estimator that is fitted on a panel. An estimator that reads no panel ignores it.dims: Dimension along which the computation runs.kwargs...: Additional keyword arguments, passed to the estimators thatpeholds.
Validation
dims in (1, 2).- Each condition that
scenario_rowsrefuses.
Returns
pr::HighOrderPrior: Result that holds the low order prior result, the square cokurtosis, the coskewness, its negative spectral form and the structure matrices.
Related
PortfolioOptimisers.partial_fit! — Function
partial_fit!(pe::HighOrderPriorEstimator, X, F = nothing; kwargs...)Fold observations into a HighOrderPriorEstimator, and pass them to its members.
The estimator keeps no buffer. It builds HighOrderPrior(; pr = pr, ...) around the result of its embedded prior. The rows that prior(pe) with no data needs are the rows that the embedded prior holds. A buffer in this estimator is a second copy of the sample.
The estimator always passes the fold to pe.pe, because that member holds the rows. An embedded prior that cannot fold refuses here, and names the wrapper that gives it a buffer. This refusal is correct, because the estimator has no rows of its own to give it. The factor observations go with the rows, as the batch verb passes F to the embedded prior. The tree of the embedded prior decides what to do with them. pe.kte and pe.ske go through fold_member. prior(pe) with no data has rows for them, and it refits a co-moment that cannot fold. So HighOrderPriorEstimator(; ske = Coskewness(; alg = SemiMoment())) needs no wrapper, and it keeps one copy of the sample.
Algorithm
- At the first step, when
pe.peholds no state, refuse a member that holds one, withassert_cold_carry_entry. - Fold the observations and
Fintope.pewithpartial_fit!. - Fold the observations into
pe.kteandpe.skewithfold_member. - Rebuild the estimator with the three folded members through
rebuild_struct, and return it.
Arguments
pe: High order prior estimator.X: Observations to fold, a matrix of rows or one observation.F: The factor observations besideX, ornothing. The method forwards them to the embedded prior.kwargs...: Additional keyword arguments, forwarded to the members.
Validation
pe.pefolds. Otherwise the embedded prior refuses the fold, with an error that names the wrapper that gives it a buffer.- At the first step, no member holds a partial-fit state. The embedded prior holds the rows, and it starts with no rows. So a co-moment that holds a state at this step reads rows that the embedded prior does not hold. An
ArgumentErrorthat names the path of the state is thrown otherwise, seeassert_cold_carry_entry.
Returns
pe: The estimator, with its members folded.
Related
PortfolioOptimisers.prior — Method
prior(pe::HighOrderPriorEstimator; kwargs...)Read a HighOrderPriorEstimator out of its fold, with no data matrix.
The method reads the embedded prior first. A co-moment that folded gives its value from its state. The method refits a co-moment that did not fold, over the rows that the embedded prior folded, which prior_returns_buffer reads. When the embedded result gives the probabilities w of its scenarios, the method refits both co-moments under w, folded or not, because a state cannot read w. assemble_high_order_prior then builds the result, as in the batch method.
The refit reads the folded rows, and not pr.X. pr.X holds the scenarios of the result of the embedded prior, and these are not always the folded rows. A Scenario Cap on the embedded prior keeps only the last rows. The scenario fill writes zeros into the first rows of an asset that starts to list late. The batch method fits both co-moments over every row that the caller passes, with the NaN entries. When the embedded result gives w, it fits them over the last length(pr.w) rows, with scenario_rows. So in both cases a refit over pr.X gives a different value. Under an Online window, the buffer holds the last max_history rows, and the equal batch fit of that route uses the same window. The estimator keeps no copy of the rows, because the embedded prior holds them.
Mathematical definition
\[\begin{align} \mathcal{P}_T &= \mathcal{P}(\mathbf{X})\,. \end{align}\]
Where:
- $\mathcal{P}_T$: Folded prior, the result that the estimator's batch verb gives with no data after the estimator folds the observations $\boldsymbol{x}_1, \ldots, \boldsymbol{x}_T$ in order, one at a time or in blocks of any size.
- $\mathcal{P}(\cdot)$: Batch prior, the result that the estimator's batch verb gives on the matrices in the parentheses.
- $\mathbf{X}$: Returns matrix $T \times N$.
The equality holds for the embedded result and for every co-moment, both the co-moments that fold and the co-moments that the method refits. In floating point, each co-moment differs from the batch co-moment by rounding.
Algorithm
- Read the embedded result
prout ofpe.pewithprior. - Read the rows that the embedded prior folded, with
prior_returns_buffer. Drop the rows of a Shared Series gap withshared_series_buffer, as the embedded prior and the members that fold drop them. A member that refits drops the fixed observation weights of the same rows, throughbuffer_refit_member. Read the rows that remain withsample_buffer, and call themX. - Rebuild the Asset Panel of the rows from the same buffer with
buffer_asset_panel, and call itpnl. The batch method givesrd.pnlto both co-moments in the same way. - Read
ktout ofpe.ktewithread_comomentunderpr.w. It refits the member overXandpnlwhen the member did not fold, and underpr.wwhenpr.wis notnothing. - Read
skandVout ofpe.skewithread_comoment, in the same way. - Build the result with
assemble_high_order_prior. - Record in the result the returns source that
pestates, withstamp_returns_source, and return it.
Arguments
pe: High order prior estimator whose members hold states.kwargs...: Additional keyword arguments, passed to the members.
Validation
- Each condition that
prior(pe.pe)with no data checks. - Each condition that
prior_returns_bufferrefuses. - Each condition that
scenario_rowsrefuses.
Returns
hop::HighOrderPrior: Result that holds the low order prior result and the co-moments.
Related
References
- [64]
- E. Jondeau and M. Rockinger. Optimal portfolio allocation under higher moments. Notes d'Études et de Recherche 108 (Banque de France, 2004).
- [66]
- D. Cajas. Convex Optimization of Portfolio Kurtosis. Available at SSRN 4202967 (2022).