Base Uncertainty Sets: private API
PortfolioOptimisers.UcSE_UcS — Type
const UcSE_UcS = Union{AbstractUncertaintySetResult, AbstractUncertaintySetEstimator}Alias for a union of the result types and the estimator types of an uncertainty set.
Related
PortfolioOptimisers.Num_UcSK — Type
const Num_UcSK = Union{AbstractUncertaintyKAlgorithm, Number}Alias for a union of the radius algorithms of an ellipsoid and the number types. A number is the radius.
Related
PortfolioOptimisers.Num_UcSEps — Type
const Num_UcSEps = Union{AbstractUncertaintyEpsAlgorithm, Number}Alias for a union of the radius algorithms of an $\ell_1$ set and the number types. A number is the radius. An algorithm computes the radius from the data.
Related
PortfolioOptimisers.Num_CptRad — Type
const Num_CptRad = Union{AbstractCompactRadiusAlgorithm, Number}Type bound of the field that holds the compact covariance radius.
The field holds the radius, or a rule of AbstractCompactRadiusAlgorithm that computes it. The union names one family only. So the constructor refuses a rule of a different family at construction, at the line where the caller writes it. The union does not accept a plain Function. A rule of this family reads seven arguments from the call site. Four of them are the metric, the loadings block, the diagonal metric and the span. A closure over the data of the caller must repeat all seven of them before it can run.
Related
PortfolioOptimisers.ucs_prior — Function
ucs_prior(pe::AbstractLowOrderPriorEstimator, X::MatNum,
F::Option{<:MatNum} = nothing; dims::Int = 1, kwargs...)
ucs_prior(::Nothing, X::MatNum, F::Option{<:MatNum} = nothing; kwargs...)Fits the prior that an uncertainty set calibrates on, or throws an error that names pe when the set holds no prior estimator.
Every returns-data verb of the four families fits its pe through this function. The four families are DeltaUncertaintySet, NormalUncertaintySet, ARCHUncertaintySet and CharacteristicUncertaintySet. So the library writes the refusal once, and not once for each verb of each family.
A set whose pe is nothing calibrates on a Prior Result that the caller gives it, and nothing has that meaning only. The fit does not replace nothing with an empirical prior over X. That replacement makes one estimator calibrate on two different priors, and the call site selects which one. So the returns-data form throws an error, and the error names the form ucs(ue, pr) and the field pe.
Arguments
pe: The prior estimator of the set, ornothing.X: Data matrix, for example the returns.F: Optional factor matrix, which the prior estimator reads.dims: Dimension along which the computation runs.kwargs...: More keyword arguments, which the method gives to the prior estimator.
Validation
peis notnothing, else anArgumentErrorthat names the form over a Prior Result.
Returns
pr::AbstractPriorResult: The fitted prior.
Related
PortfolioOptimisers.ucs_selector — Function
ucs_selector(risk_ucs::Nothing, prior_ucs::Nothing)
ucs_selector(risk_ucs::UcSE_UcS, prior_ucs::Any)
ucs_selector(risk_ucs::Nothing, prior_ucs::UcSE_UcS)Chooses between the uncertainty set of a risk measure and the uncertainty set of a prior, and the set of the risk measure has priority.
The function is a selector, so it states the table below and has no # Algorithm section. Each of its three methods returns one of its arguments and does no step. The three methods cover every pair of arguments that the callers make. Only the first row gives nothing.
risk_ucs | prior_ucs | Result |
|---|---|---|
nothing | nothing | nothing |
a UcSE_UcS | anything | risk_ucs |
nothing | a UcSE_UcS | prior_ucs |
Arguments
risk_ucs: Uncertainty set estimator or result of the risk measure, ornothing.prior_ucs: Uncertainty set estimator or result of the prior, ornothing.
Returns
ucs::Option{<:UcSE_UcS}: The selected set or estimator, as the table above states.
Related
PortfolioOptimisers.investable_ucs_reduction — Function
investable_ucs_reduction(pr::AbstractPriorResult, rd, imsk::Nothing) -> (nothing, pr, rd)
investable_ucs_reduction(pr::AbstractPriorResult, rd,
imsk::BitVector = investable_mask(pr)) -> (imsk, pr, rd)Reduces a Prior Result, and the returns data beside it, to the Investable Mask before an estimator fits a set on them outside an optimiser.
A Prior Result covers the full asset universe. An asset outside its Investable Mask has NaN in its moments and in every block that a factor fit wrote. So a set fitted on the whole result gets a NaN where its arithmetic needs a number. Every optimisation family reduces the result once at its entry, so inside an optimiser the result is already reduced and the mask is nothing. Outside an optimiser the result is whole. This function then takes the same view as the optimiser, and the two routes fit the same set.
expand_investable_ucs writes the fitted set back onto the full universe. Every Prior Result follows the same pattern of reduction and expansion.
The mask is the last positional slot, and its default reads it from the Prior Result. A mask of nothing is the path where all assets are investable, and the method on Nothing then returns its arguments unchanged.
Algorithm
- Take the Investable Mask
imsk. By default,investable_maskreads it from the Prior Result. - If the mask is
nothing, returnnothingand the two arguments unchanged. - Else, take a
port_opt_viewof the Prior Result atfindall(imsk). Take the same view of the returns data when it is notnothing. Return the two views and the mask.
Arguments
pr: Prior result.rd: Returns data beside which the estimator fits the set, ornothing.imsk: The Investable Mask ofpr, ornothingwhen all assets are investable.
Returns
(imsk, pr, rd): The Investable Mask and the two arguments reduced to it, ornothingand the two arguments unchanged.
Related
PortfolioOptimisers.expand_investable_ucs — Method
expand_investable_ucs(set::AbstractUncertaintySetResult, ::Nothing, pr::AbstractPriorResult) -> setWrites a set fitted on the Investable Mask back onto the full asset universe.
The nothing method is the path of investable_ucs_reduction where all assets are investable. It returns the set unchanged, so a set that an optimiser fits on a reduced result costs nothing here. Each built set that a route through a Prior Result makes adds its own BitVector method beside its port_opt_view. The expansion is the inverse of that view. It writes the blocks of the set on the asset axis into a frame of the full width. It sets val to the full pr.mu or pr.sigma, with its NaN entries.
So port_opt_view(expand_investable_ucs(set, imsk, pr), findall(imsk)) gives back set. This lets a caller fit a set outside an optimiser on a point-in-time prior, and give it to an optimiser on the full universe. The frame is NaN for a box and zero for an ellipsoid, and each method states the reason.
The method is a passthrough, so it has no # Algorithm section.
Arguments
set: Fitted uncertainty set.imsk: The Investable Mask, ornothing.pr: Prior result.
Returns
set::AbstractUncertaintySetResult: The set on the full asset universe.
Related
PortfolioOptimisers.expand_investable_ucs — Method
expand_investable_ucs(
set::BoxUncertaintySet{<:AbstractVector{<:Union{Number, AbstractJuMPScalar}}, <:AbstractVector{<:Union{Number, AbstractJuMPScalar}}},
imsk::BitVector,
pr::AbstractPriorResult
) -> Union{BoxUncertaintySet{var"#s185", <:AbstractArray{var"#s166", N}, Nothing} where {var"#s166"<:Union{Number, AbstractJuMPScalar}, N, var"#s185"<:AbstractArray{var"#s166", N}, var"#s166"<:Union{Number, AbstractJuMPScalar}, N}, BoxUncertaintySet{var"#s185", var"#s1851", <:AbstractArray{var"#s166", N}} where {var"#s166"<:Union{Number, AbstractJuMPScalar}, N, var"#s185"<:AbstractArray{var"#s166", N}, var"#s166"<:Union{Number, AbstractJuMPScalar}, N, var"#s1851"<:AbstractArray{var"#s166", N}, var"#s166"<:Union{Number, AbstractJuMPScalar}, N}}
Writes a vector BoxUncertaintySet fitted on the Investable Mask back onto the full asset universe.
The expansion is the inverse of port_opt_view. The view slices the two bounds and the centre by the asset index, and reads nothing more. So the view removes all of the frame outside the mask, and it gives back the fitted set. The frame is NaN, as in the moment. A bound on mu has the units and the positions of mu, and outside the mask the prior states NaN for mu. The centre is the full pr.mu.
Algorithm
- Write
set.lbandset.ubat the rows that the mask keeps into twoNaNframes oflength(imsk)entries, withexpand_moment. - Build a
BoxUncertaintySetfrom the two frames, withpr.muasval.
Arguments
set: Box uncertainty set with vector bounds, fitted on the reduced prior.imsk: The Investable Mask of the full prior.pr: Prior result.
Returns
set::BoxUncertaintySet: The set on the full asset universe.
Related
PortfolioOptimisers.expand_investable_ucs — Method
expand_investable_ucs(
set::BoxUncertaintySet{<:AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}, <:AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}},
imsk::BitVector,
pr::AbstractPriorResult
) -> Union{BoxUncertaintySet{var"#s185", <:AbstractArray{var"#s166", N}, Nothing} where {var"#s166"<:Union{Number, AbstractJuMPScalar}, N, var"#s185"<:AbstractArray{var"#s166", N}, var"#s166"<:Union{Number, AbstractJuMPScalar}, N}, BoxUncertaintySet{var"#s185", var"#s1851", <:AbstractArray{var"#s166", N}} where {var"#s166"<:Union{Number, AbstractJuMPScalar}, N, var"#s185"<:AbstractArray{var"#s166", N}, var"#s166"<:Union{Number, AbstractJuMPScalar}, N, var"#s1851"<:AbstractArray{var"#s166", N}, var"#s166"<:Union{Number, AbstractJuMPScalar}, N}}
Writes a matrix BoxUncertaintySet fitted on the Investable Mask back onto the full asset universe.
The expansion is the inverse of port_opt_view. The view slices the two bounds on the two axes, and the centre, by the asset index. So the view removes all of the frame outside the mask, and it gives back the fitted set. The frame is NaN, as in the moment. A bound on sigma has the positions of sigma, and outside the mask the prior states NaN for its row and its column. The centre is the full pr.sigma.
Algorithm
- Write
set.lbandset.ubat(imsk, imsk)into twoNaNframes oflength(imsk)rows and columns, with the covariance method ofexpand_moment. - Build a
BoxUncertaintySetfrom the two frames, withpr.sigmaasval.
Arguments
set: Box uncertainty set with matrix bounds, fitted on the reduced prior.imsk: The Investable Mask of the full prior.pr: Prior result.
Returns
set::BoxUncertaintySet: The set on the full asset universe.
Related
PortfolioOptimisers.expand_investable_ucs — Method
expand_investable_ucs(
set::EllipsoidalUncertaintySet{<:AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}, <:Any, <:SigmaUncertaintySetClass},
imsk::BitVector,
pr::AbstractPriorResult
) -> Union{EllipsoidalUncertaintySet{<:AbstractMatrix{var"#s166"}, <:Number, <:AbstractUncertaintySetClass, Nothing} where var"#s166"<:Union{Number, AbstractJuMPScalar}, EllipsoidalUncertaintySet{var"#s185", var"#s1851", var"#s1852", <:AbstractArray{var"#s166", N}} where {var"#s166"<:Union{Number, AbstractJuMPScalar}, var"#s185"<:AbstractMatrix{var"#s166"}, var"#s1851"<:Number, var"#s1852"<:AbstractUncertaintySetClass, var"#s166"<:Union{Number, AbstractJuMPScalar}, N}}
Writes a covariance EllipsoidalUncertaintySet fitted on the Investable Mask back onto the full asset universe.
The expansion is the inverse of port_opt_view. The view slices the shape matrix at the fourth-moment index of the selected assets, and the centre at the plain index. So the view removes all of the zero block outside the mask, and it gives back the fitted set. The centre is the full pr.sigma.
The frame is zero and not NaN, because the shape matrix is geometry. The same set converts into a NormBallUncertaintySet, whose map must be finite. A zero row of that map has no effect on an asset that the prior did not estimate.
Algorithm
- Find the positions of the investable pairs in the vectorised covariance with
coverage_pair_index, in the order offourth_moment_index_generator. - Make a zero frame of
length(imsk)^2rows and columns. Writeset.sigmaat those positions on the two axes. - Rebuild the set from the frame, with
pr.sigmaasval, withrebuild_struct.kandclassdo not change.
Arguments
set: Ellipsoidal uncertainty set on the covariance axis, fitted on the reduced prior.imsk: The Investable Mask of the full prior.pr: Prior result.
Returns
set::EllipsoidalUncertaintySet: The set on the full asset universe.
Related
PortfolioOptimisers.expand_investable_ucs — Method
expand_investable_ucs(
set::EllipsoidalUncertaintySet{<:AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}, <:Any, <:MuUncertaintySetClass},
imsk::BitVector,
pr::AbstractPriorResult
) -> Union{EllipsoidalUncertaintySet{<:AbstractMatrix{var"#s166"}, <:Number, <:AbstractUncertaintySetClass, Nothing} where var"#s166"<:Union{Number, AbstractJuMPScalar}, EllipsoidalUncertaintySet{var"#s185", var"#s1851", var"#s1852", <:AbstractArray{var"#s166", N}} where {var"#s166"<:Union{Number, AbstractJuMPScalar}, var"#s185"<:AbstractMatrix{var"#s166"}, var"#s1851"<:Number, var"#s1852"<:AbstractUncertaintySetClass, var"#s166"<:Union{Number, AbstractJuMPScalar}, N}}
Writes a mean EllipsoidalUncertaintySet fitted on the Investable Mask back onto the full asset universe.
The expansion is the inverse of port_opt_view. The view slices the shape matrix on the two axes, and the centre, by the asset index. So the view removes all of the zero rows and columns outside the mask, and it gives back the fitted set. The centre is the full pr.mu.
The frame is zero and not NaN, because the shape matrix is geometry. The same set converts into a NormBallUncertaintySet, whose map must be finite. A zero row of that map has no effect on an asset that the prior did not estimate.
Algorithm
- Make a zero frame of
length(imsk)rows and columns. Writeset.sigmaat(imsk, imsk). - Rebuild the set from the frame, with
pr.muasval, withrebuild_struct.kandclassdo not change.
Arguments
set: Ellipsoidal uncertainty set on the mean axis, fitted on the reduced prior.imsk: The Investable Mask of the full prior.pr: Prior result.
Returns
set::EllipsoidalUncertaintySet: The set on the full asset universe.
Related
PortfolioOptimisers.vec_quantile_bounds — Function
vec_quantile_bounds(mus::MatNum, q::Number, kwargs)Computes the element-wise lower and upper quantile bounds of a vector statistic.
The bootstrap box methods of ucs and mu_ucs use this function for the mean. The samples are on the second axis. The body reads mus[j, :], so mus is $N \times M$, with one row for each component and one column for each sample. A caller that gives the transpose gets bounds of length $M$, and the BoxUncertaintySet constructor accepts them. So the axis is a contract that this method cannot check.
The two bounds do not always contain the sample mean of a component. A sample with one large outlier can put its mean outside the two quantiles.
Algorithm
- Read
N = size(mus, 1), the number of components. Makelbandubof that length, with the element typeeltype(mus). - For each component
j, read the rowmu_j = mus[j, :], theMsampled values of that component. - Write the
qquantile ofmu_jintolb[j], and the1 - qquantile intoub[j]. - Return
lbandub. The caller givesq < 1/2, solb .<= ubentrywise.
Arguments
mus: Sampled values, $N \times M$, with one row for each component.q: Significance level, which the caller halved.kwargs: Keyword arguments, which the function splats intoStatistics.quantile.
Returns
lb::Vector: Element-wise lower bound, length $N$.ub::Vector: Element-wise upper bound, length $N$.
Related
PortfolioOptimisers.ucs_dimension — Function
ucs_dimension(::MuUncertaintySetClass, ::Bool, m::Integer)
ucs_dimension(::SigmaUncertaintySetClass, diagonal::Bool, m::Integer)
ucs_dimension(class::AbstractUncertaintySetClass, diagonal::Bool, m::Integer,
M::Integer)Returns the dimension of an uncertainty set on a shape of side m, which is the degrees of freedom of its chi-squared radius.
The dimension is the number of free coordinates of the estimation error. The squared Mahalanobis distance of a normal error has a chi-squared distribution at that number. On the mean axis, the dimension is m, the number of assets. On the covariance axis, the error is a vectorised symmetric matrix, so the entries $(i, j)$ and $(j, i)$ are one number. A full shape then has rank $N(N+1)/2$ of its $N^{2}$. The positive definite repair gives it a nominal full rank, and adds $N(N-1)/2$ flat directions that no error fills, the antisymmetric matrices.
A diagonal shape keeps $N^{2}$. Its statistic is a sum of $N^{2}$ terms with a mean of one, so its expected value is $N^{2}$.
The four-argument method is for a full shape that is the sample covariance of $M$ sampled errors. The sample is centred, so that shape has a rank of $M - 1$ or less, and the method limits the dimension to that value. The limit applies when $M - 1$ is less than $N$ on the mean axis, or less than $N(N+1)/2$ on the covariance axis. A diagonal shape of sample variances has full rank, so it has no limit. The limit assumes a plain sample covariance. A shrinkage estimator gives a shape of full rank, and the limit then makes its dimension too small.
Mathematical definition
\[\begin{align} d &= \begin{cases} m & \text{mean axis, or a diagonal shape}\,, \\ \dfrac{N(N+1)}{2}\,, \quad N = \sqrt{m} & \text{full covariance shape}\,, \end{cases} \\ d_{M} &= \begin{cases} d & \text{diagonal shape}\,, \\ \min(d, M - 1) & \text{full shape}\,. \end{cases} \end{align}\]
Where:
- $d$: Dimension of the set.
- $d_{M}$: Dimension of a set whose full shape is the sample covariance of $M$ errors.
- $M$: Number of sampled errors.
- $m$: Side of the shape matrix, or row count of the geometry map.
- $N$: Number of assets.
Arguments
class: Axis tag of the set.diagonal: Whether the shape is the diagonal of the asymptotic covariance.m: Side of the shape matrix, $N$ on the mean axis and $N^{2}$ on the covariance axis.M: Number of sampled errors, of which the shape is the sample covariance.
Returns
d::Integer: Dimension of the set.
Related
PortfolioOptimisers.ellipsoidal_set — Function
ellipsoidal_set(alg::EllipsoidalUncertaintySetAlgorithm, q::Number, samples,
cov::MatNum, class::AbstractUncertaintySetClass,
val::Option{<:ArrNum} = nothing;
df::Integer = ucs_dimension(class, alg.diagonal, size(cov, 1)))Builds an EllipsoidalUncertaintySet from an asymptotic covariance cov that the caller computed.
Every ellipsoidal method of ucs, mu_ucs and sigma_ucs in all estimator families uses this function. It is the ellipsoidal counterpart of norm_ball_set, and it takes the set algorithm in the same position. k_ucs ignores the last arguments that its algorithm does not read. So samples can be the deviation matrix, a 1:n_sim range or nothing, as the caller has it.
The order of the first two steps below is necessary. The function takes the diagonal before it computes the radius. So under the default diagonal = true, a radius from a sample is a quantile of Mahalanobis distances against the diagonal shape, not against the full shape. Neither shape always gives the larger radius. If the function takes the diagonal after the radius, the radius comes from one shape and the set uses a different shape. The set then does not have the coverage that its significance level states.
Algorithm
- If
alg.diagonalistrue, replacecovwithLinearAlgebra.Diagonal(cov). This removes the correlations of the estimation errors between entries. The result is aDiagonal, not a dense matrix. - Compute the radius
k = k_ucs(alg.method, q, samples, cov, df)against the shape that step 1 gives.dfis the dimension of the set, which is $N(N+1)/2$ on a full covariance shape, and not the side of the shape. - Build an
EllipsoidalUncertaintySetfromcov,k,classandval.
Arguments
alg: Ellipsoidal uncertainty set algorithm, which holds the radius algorithmalg.methodand the diagonal switchalg.diagonal.q: Significance level.samples: Sampled estimation errors, or the container thatalg.methodreads. An algorithm that runs no simulation ignores it.cov: Asymptotic covariance of the statistic, which becomes the shape matrix.class: Axis tag, which sets the size of the shape matrix and the index that a view applies.val: Quantity that the set is a neighbourhood of. It is the fitted mean vector on the mean axis, and the fitted covariance on the covariance axis. Every caller has it, because every caller fits a prior before it calls this function.df: Dimension of the set, which is the degrees of freedom of a chi-squared radius. The default is the dimension of the shape of the statistic. A caller whose shape is the sample covariance of $M$ errors gives the dimension limited to $M - 1$, the rank of that shape.
Returns
ucs::EllipsoidalUncertaintySet: The assembled set.
Related
PortfolioOptimisers.box_quantile_bounds — Function
box_quantile_bounds(::Type{TE}, get_ij, N::Integer, q::Number, kwargs) where {TE}Computes the element-wise lower and upper quantile bounds of a symmetric $N \times N$ statistic.
The box methods of ucs and sigma_ucs of all estimator families use this function. The accessor get_ij lets one body serve all of them. It reads the Wishart sample, which is a vector of matrices, and the bootstrap sample, which is a three-dimensional array. The caller does the projection to a positive definite matrix, if one is necessary.
The two bounds do not always contain the sample mean of the statistic. A sample with one large outlier can put its mean outside the two quantiles.
Algorithm
- Make
lbandub, bothN × Nwith the element typeTE. - For each pair with
j <= i, reads_ij = get_ij(i, j), the sampled values of that entry. - Write the
qquantile ofs_ijintolb[i, j]andlb[j, i]. Write the1 - qquantile intoub[i, j]andub[j, i]. Each quantile goes to two positions, so the two bounds are symmetric, and each pair costs one quantile and not two. - Return
lbandub. The caller givesq < 1/2, soqis the smaller quantile level of the same sample andlb .<= ubentrywise.
Arguments
TE: Element type of the two bounds.get_ij: Accessor function.get_ij(i, j)returns the vector of sampled values of the entry $(i, j)$.N: Number of rows and columns of the statistic.q: Significance level, which the caller halved.kwargs: Keyword arguments, which the function splats intoStatistics.quantile.
Returns
lb::Matrix{TE}: Element-wise lower bound, symmetric.ub::Matrix{TE}: Element-wise upper bound, symmetric.
Related
PortfolioOptimisers.reader_frames — Method
reader_frames(ue::AbstractUncertaintySetEstimator, path::AbstractString, refit::Bool)
reader_frames(ue::AbstractPriorUncertaintySetEstimator, path::AbstractString, refit::Bool)Returns the frames that the prior of an uncertainty set adds to the group of the optimiser that holds the set.
An uncertainty set holds no source of its own. It gives the returns to its pe, and that prior states the source. So the method calls reader_frames on pe at the path "<path>.pe". The warning of warn_raw_readers then names the prior of a set as, for example, "r.ucs.pe". A set with pe = nothing reads the Prior Result of the optimiser, so it adds no frames. An AbstractPriorUncertaintySetEstimator holds no prior, and a built set reads no returns, so they also add no frames.
Arguments
ue: The uncertainty set estimator.path: The path of the value from the head, for example"pe","cle"or"r[2].pe". A warning names a reader by its path.refit:truewhen a regression reader atx_src = nothingin the group fits its regression on the raw returns data.refits_regressionfinds the value.
Returns
fr::ReturnsFrames: The frames of the prior of the set.
Related