Calibration Rules: private API
PortfolioOptimisers.Num_SigCal — Type
const Num_SigCal = Union{AbstractSignificanceCalibrationAlgorithm,
AbstractCountOrFraction, Function, Number}Field bound for a significance slot: the tail probability, a part of the scenarios as a CountOf or a FractionOf, a rule of the family, or a plain function of the same five arguments.
A plain number is the tail probability itself, so the slot takes no plain shorthand of count_or_fraction. alpha = CountOf(15) keeps fifteen scenarios in the tail at every sample length, and resolve_calibration_slot resolves it to 15 / T. alpha = FractionOf(0.05) is the level 0.05.
The slot names the end of the distribution, so the occupant does not have to. alpha names the lower tail and beta names the upper tail, and one rule serves both ends. So one bound serves every significance slot, and a Range measure sets its head slot to the occupant of its tail slot by default.
resolve_calibration_slot calls a rule, so a function and a callable struct are the same to it. A struct holds parameters and has a name that a docstring can describe. A function is the shortest way to state a rule for one use, and a closure over the data of a caller has no type. A function has no family, so this bound cannot refuse a deformation rule that is a closure. Only the name of the slot states the quantity there.
Related
PortfolioOptimisers.Num_DefCal — Type
const Num_DefCal = Union{AbstractDeformationCalibrationAlgorithm, Function,
Number}Field bound for a deformation slot: the deformation parameter, a rule of the family, or a plain function of the same five arguments. It is the counterpart of Num_SigCal for the deformation family, and the description of that bound applies to it.
Related
PortfolioOptimisers.AbstractCalibrationSeries — Type
abstract type AbstractCalibrationSeries <: AbstractEstimatorNames the series that a calibration rule reads, so that the rule reads the quantity that its slot owner prices.
A rule gets no portfolio, as resolve_calibration_slot states. So it cannot make the loss series of a portfolio, which exists only after the solver returns. A rule can make the series of each column of pr.X. This family names that series: the column, or the drawdown series of the column. A slot owner gives the series through the trait calibration_series, and the CalibrationContext takes it to the rule.
The marker names a quantity and not an estimator, so each rule reads it in its own way:
HillTailDecaymakes the series of each column and estimates its tail index.RadialTailDecaycannot make a radial series of a path, because a Mahalanobis distance has no path. It whitens the rows of the drawdown sample instead.- The three radius rules read the per-asset dispersion of the series, which
calibration_series_dispersiongives. TailTermParityreads both terms of its ratio over the series.
Related
PortfolioOptimisers.AbstractDrawdownSeries — Type
abstract type AbstractDrawdownSeries <: AbstractCalibrationSeriesNames a drawdown series, the quantity that a drawdown measure prices.
A drawdown is a function of the path of one series, so a rule makes it per column and never per row. The two members use different compounding of the path. They are the series that absolute_drawdown_vec and relative_drawdown_vec make.
A drawdown series is not positive, so it has one end. So a rule that answers per end refuses a head key under this family. The refusal is a fact about the quantity and not about the sample.
The tail of a drawdown series depends on the record. A drawdown is a running function of the path, so its law over a finite record is not the law of one step. So a rule that reads the shape of a series reads a different shape here. A sample with a strong drift lets the drawdown process become stationary, and its drawdown series has a heavier tail than its returns. A sample with a weak drift gives the range of the path over the record, and that range has a thinner tail than the law of one step. Both are properties of the series that the measure prices over its record. So a rule describes the sample, and not drawdowns in general.
Related
PortfolioOptimisers.Num_AmbRadCal — Type
const Num_AmbRadCal = Union{AbstractAmbiguityRadiusCalibrationAlgorithm, Function,
Number}Field bound for an ambiguity-radius slot: the radius, a rule of the family, or a plain function of the same five arguments.
The union names one rule family. So a tail-weight rule in a radius slot does not match the signature of the constructor, and the constructor refuses it. This is all the validation of the family, and no separate check exists for it. A function is a member of the union for the reason that Num_SigCal states. It has no family, so only the name of the slot states the quantity.
Related
PortfolioOptimisers.Num_AmbTwtCal — Type
const Num_AmbTwtCal = Union{AbstractAmbiguityTailWeightCalibrationAlgorithm,
Function, Number}Field bound for an ambiguity-tail-weight slot: the tail weight, a rule of the family, or a plain function of the same five arguments. It is the counterpart of Num_AmbRadCal for the tail-weight family, and the description of that bound applies to it.
Related
PortfolioOptimisers.Num_NormCeilCal — Type
const Num_NormCeilCal = Union{AbstractNormCeilingCalibrationAlgorithm,
AbstractCountOrFraction, Function, Number}Field bound for a norm-ceiling slot: the ceiling, a floor of effective assets as a CountOf or a FractionOf, a rule of the family, or a plain function of the same five arguments.
A plain number is the ceiling itself, so the slot takes no plain shorthand of count_or_fraction. l2c = CountOf(10) keeps at least ten effective assets, and l2c = FractionOf(0.5) keeps half of the universe effective. resolve_calibration_slot resolves each one to the ceiling against the norm order of the site.
The union names one rule family. So a constructor refuses a radius rule in a ceiling slot, and it refuses a ceiling rule in a radius slot in the same way. AbstractNormCeilingCalibrationAlgorithm states the difference between the two quantities.
Related
PortfolioOptimisers.Num_AmbRadNormCeilCal — Type
const Num_AmbRadNormCeilCal = Union{AbstractAmbiguityRadiusCalibrationAlgorithm,
AbstractNormCeilingCalibrationAlgorithm,
AbstractCountOrFraction, Number}Field bound for the one slot that the library reads as two quantities, the val field of LpRegularisation.
That estimator is a penalty in the lp field of JuMPOptimiser and a norm constraint in its lpc field. So val is an ambiguity radius on one route and a norm ceiling on the other. One field cannot have two bounds, so this bound admits both rule families, and each route refuses the family that has no meaning on it. In this slot alone, the route and not the bound settles the quantity, after the construction. A CountOf or a FractionOf states a floor of effective assets, which is a ceiling, so the penalty route refuses it as it refuses a ceiling rule.
This bound does not admit a plain function, and it is the only slot that refuses one. Every other slot names one quantity, so a function in it needs no family. This slot names two quantities, and the two checks read the family to tell them apart. A function has no family. So a function in this slot can reach a route that has no meaning for it, and no check can find the error.
Related
PortfolioOptimisers.resolve_calibration_slot — Function
resolve_calibration_slot(slot, key::Symbol, pr::AbstractPriorResult, w, slv = nothing,
ctx::CalibrationContext = CalibrationContext())Resolves one calibration slot against the prior result pr, the effective observation weights w, the effective solver slv and the context ctx of the site, and returns a plain number.
The function calls a rule in a slot as alg(key, pr, w, slv, ctx). So a callable rule and a plain function are the same here. It returns every other occupant, such as a stated number, unchanged.
A rule gets no portfolio. A prior result holds no portfolio weight vector, so no rule can measure the loss series of a portfolio. A rule can measure the series of each column of the sample, and ctx.series tells it which series the slot owner prices. A rule gets the solver on both routes that resolve a measure, so it can call ERM or RRM:
- On the
factoryroute,@propagatableruns the@cpropselection before the resolution, so the struct holds the solver. - On the
JuMProute, no selection runs.set_risk_constraints!passes the solver toresolve_deferred_quantities, and the owner selects it assel(x.slv, slv).
The context holds all the facts of the site, and this function rebuilds nothing. key names the slot and not the quantity. So a rule reads from ctx the shape of a series, the significance level of a sibling slot, the norm order of its constraint, or the count of the scenarios that the site prices. No rule has a field for these facts, so the function rebuilds no occupant and overwrites nothing that a caller stated. CalibrationContext states which rule reads which field.
This function checks the value of a ceiling rule and of a radius rule, and of no other family. AbstractNormCeilingCalibrationAlgorithm and AbstractAmbiguityRadiusCalibrationAlgorithm share one method. It runs assert_nonempty_gt0_finite_val on the number that the rule returned, under the key of the slot. Both quantities reach a JuMP model from a field of JuMPOptimiser: the ceiling from l2c and linfc, and the radius from l1 and linf. No term is rebuilt on that route, so the check here is the first one, and it names the slot. The three other families are only in slots that their owner rebuilds. The rebuild runs the constructor of the owner on the calibrated number, and that constructor checks the range. Only the calibrated path pays for the check, because a stated number takes the first method.
The site checks a rule that is a plain function. A function has no family, so no method here can know the quantity that it computes. The stage 2 of the constraint block, resolve_constraint_block, is the one site that reads a calibrated number and rebuilds no term. After the resolution, it checks the range of each of its four norm slots, whatever occupant gave the number.
This function is the counterpart of resolve_slot. It is a second function, and not a wider method of the first, for two reasons. The body of resolve_slot is deferred_quantity(fit_deferred_quantity(dq, pr), key), a fit and then an extraction, and a rule fits nothing. Also, resolve_slot takes neither w nor slv, and a rule needs both. So the rule families are not members of the DeferredQuantity union.
The caller computes w as sel(r.w, pr.w), and passes it with the slv of the measure. A parent that has no observation weights passes pr.w. A parent that has no solver leaves slv at its default.
A TimeDependent reaches the estimator that holds the slot, and does not go further. A schedule changes a field of an estimator, and update_time_dependent_fields applies it before the fit of a prior. A rule is always in a slot of an estimator, so a schedule replaces that estimator. If the estimator is a JuMPOptimiser, its four norm fields can each have a schedule, and the schedule selects a rule for each fold. If the estimator is a risk measure, the bound of the slot admits no schedule. The caller then changes the whole measure through the risk-measure field of the optimiser, which can have a schedule. On both routes the selection runs first, and the rule then resolves against the prior of the selected period. So a rule needs no schedule of its own. A schedule in a rule names a fold that the rule cannot see, and it repeats the schedule of the estimator.
Algorithm
- Return
slotunchanged whenis_calibration_rulereturnsfalsefor it. A stated number takes this branch. - Call the rule as
slot(key, pr, w, slv, ctx), and return the number that it gives. - A norm-ceiling rule or an ambiguity-radius rule takes a more specific method. That method checks the number before it returns it.
Arguments
slot: The occupant of the slot: a number or a rule.key: Name of the slot that the call resolves.pr: Prior result that the rule reads.w: Effective observation weights, ornothing.slv: Effective solver, ornothingwhen the measure has none.ctx: The facts that the site knows andkeydoes not give, as aCalibrationContext. The default has no sibling significance level, no norm order, and the returns series.
Validation
- A norm-ceiling rule or an ambiguity-radius rule must return a number that is
> 0and finite.assert_nonempty_gt0_finite_valchecks it underkey. The function returns every other occupant with no check, and the owner of its slot checks the range when it rebuilds.
Returns
val::Number: The calibrated quantity, or the stated value unchanged.
Related
resolve_calibration_slot(part::CountOf, key::Symbol, pr::AbstractPriorResult, w, slv,
ctx::CalibrationContext{<:Any, <:Any, Nothing})
resolve_calibration_slot(part::FractionOf, key::Symbol, pr::AbstractPriorResult, w, slv,
ctx::CalibrationContext{<:Any, <:Any, Nothing})
resolve_calibration_slot(part::AbstractCountOrFraction, key::Symbol,
pr::AbstractPriorResult, w, slv,
ctx::CalibrationContext{<:Any, <:Any, <:Number})Computes the quantity of a calibration slot that holds a CountOf or a FractionOf.
The pair states the size of a part of a set, and the slot states the set. A significance slot names the scenarios of the tail, and a norm-ceiling slot names the assets that stay effective. The norm order ctx.p of the CalibrationContext tells the two apart, because every norm-ceiling site states it and no significance site does. So the methods dispatch on the type of ctx.p. A call with no slv or no ctx takes the defaults of the generic method, nothing and CalibrationContext(). The default context states no order, so such a call reads the pair as a part of the scenarios.
A significance level that keeps the count of the tail
A stated alpha fixes the probability of the tail. A fold of half the length then has half the scenarios in its tail, and the measure becomes less accurate as the sample becomes shorter. alpha = CountOf(n) fixes the count and returns n / T. So alpha * T is n at every sample length, and the tail of the measure holds n scenarios. alpha = FractionOf(f) is the level f itself.
T is the row count of pr.X when no observation weights are stated, because the measure prices those rows. Under a Scenario Cap, the rows are fewer than the observations behind the moments, and the count reads the rows. When weights are stated, T is the effective sample size of the weights, from scenario_sample_size. That is the number of equally weighted observations that gives the precision of the weighted sample. So a weighted tail holds n observations in that sense, and not n rows.
The function does not check the range of the level. It returns the quantity of its slot, so the constructor of the slot owner does all the validation when it rebuilds. A count of T or more gives a level of one or more, and that constructor refuses it.
A norm ceiling that keeps a floor of effective assets
A bound on the norm of the weights is a floor on the effective number of assets. CountOf(m) states the floor as a count, and FractionOf(f) states it as a fraction of the universe, m = f N. The asset count N comes from the prior result, so a subset view, a cluster and a cross-validation fold each get the floor of their own universe.
Mathematical definition
\[\begin{align} \alpha &= \begin{cases} \dfrac{n}{T_{n}} & \textrm{for } \texttt{CountOf}(n)\\ f & \textrm{for } \texttt{FractionOf}(f) \end{cases}\,,\\ T_{n} &= \begin{cases} \dfrac{\left(\sum\limits_{t=1}^{T} w_{t}\right)^{2}}{\sum\limits_{t=1}^{T} w_{t}^{2}} & \textrm{if } w \textrm{ is given}\\ T & \textrm{otherwise} \end{cases}\,,\\ m &= \begin{cases} n & \textrm{for } \texttt{CountOf}(n)\\ f N & \textrm{for } \texttt{FractionOf}(f) \end{cases}\,,\\ \mathrm{val} &= \begin{cases} m^{-1} & \textrm{if } p \textrm{ is infinite}\\ m^{1/p - 1} & \textrm{otherwise} \end{cases}\,. \end{align}\]
Where:
- $\alpha$: Significance level (left tail probability), $\alpha \in (0, 1)$.
- $n$: Number of scenarios that the tail must hold, or the number of effective assets that the ceiling keeps.
- $f$: Fraction, the
valof aFractionOf. - $T_{n}$: Scenario count of the measure, the number of scenarios that it prices.
- $T$: Number of observations. It is the row count of the sample that the result holds.
- $w_{t}$: Observation weight of observation $t$.
- $m$: Effective number of assets that the ceiling keeps as a floor.
- $N$: Number of assets.
- $\mathrm{val}$: Norm ceiling.
- $p$: Norm order of the constraint of the ceiling,
ctx.p.
The effective number of assets of order $p$ is $N_{\mathrm{eff},\, p}(\boldsymbol{w}) = \left(\sum_{i} \lvert w_{i} \rvert^{p}\right)^{1/(1 - p)}$. So for $p > 1$, $N_{\mathrm{eff},\, p} \geq m$ is $\lVert \boldsymbol{w} \rVert_{p} \leq m^{1/p - 1}$. At $p = 2$, on weights whose absolute values sum to one, this count is the number that number_effective_assets gives. At every order, a portfolio with equal weights over $m$ assets gives exactly $m$. The exponent is also $-1/q$ for the conjugate order $q$, because $1/p - 1 = -1/q$. At $p = 1$ the ceiling is one for every $m$.
The infinite branch is the limit of the finite branch. $m^{1/p - 1} \to m^{-1}$ as $p$ increases. A cap of $1/m$ on the largest weight of a fully invested portfolio spreads it over at least $m$ assets. So the two branches agree, and as a caller increases $p$, the ceiling moves towards $1/m$.
Algorithm
- When
ctx.pisnothing, returnpart.val / scenario_sample_size(pr, w)for aCountOf, andpart.valfor aFractionOf. - Otherwise, take
p = ctx.p, and check thatp >= 1. - Read
N, the column count ofpr.X, and the floormfromeffective_asset_floor. Check thatm <= N. - Return
inv(m)whenpis infinite, andm^(1/p - 1)otherwise.
Arguments
part: The occupant of the slot.key: Name of the slot that the call resolves. The refusals below name the site with it.pr: Prior result from which the function reads the sample length and the asset count.w: Effective observation weights, ornothing. A count of the universe is not a count of the sample, so the ceiling does not read them.slv: Effective solver, which the methods do not read.ctx: The facts of the site, as aCalibrationContext. Its norm orderctx.pisnothingfor a significance slot, and the order of the ceiling otherwise.
Validation
p >= 1for a ceiling. ADomainErroris thrown otherwise.m <= Nfor a ceiling of aCountOf, because a floor above the universe names a portfolio that no constraint admits. ADomainErroris thrown otherwise. AFractionOfis at most one, so its floor is at mostN.
Returns
val::Number: The significance level whenctx.pisnothing, and the norm ceiling otherwise.
Related
References
- [9] L. Kish. Survey Sampling (John Wiley & Sons, 1965). Whole book, for the effective sample size of the weights. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.calibration_slots — Function
calibration_slots(x)Declares the slots of x that can hold a Calibration Rule, as a NamedTuple from the name of each slot to its current value. The default is empty, so a type with no calibration slot needs no method.
It is the counterpart of deferred_slots. Three functions read it:
assert_calibrated_slotsrefuses a rule that reached a value-level entry point.assert_declared_calibration_resolverrefuses a rule that the library did not resolve.resolve_calibration_slotsresolves the slots.
For most types, the library derives the resolution from this declaration. A type whose slots must resolve in an order writes its own resolve_deferred_quantities method instead. In such an order, one slot reads the resolved number of a sibling slot from a CalibrationContext. A derivation cannot know which sibling to read. A slot that names its quantity under a key of its own also writes its own resolution. The three regularisation keys do this.
A slot that holds a child measure is declared here too. So a container names its children, and each child names its own slots. Such a slot is also declared in deferred_slots, and the deferred recursion resolves the child. So only the two assert_ walks read the declaration of a container. It names a child only when that child can hold a slot. A plain function in a calibration slot is a rule, so an occupant that has no slot is not named here. OWA_CalOccupant is the one bound in the library that states which occupants of a weight slot are children.
Related
PortfolioOptimisers.is_calibration_rule — Function
is_calibration_rule(x) -> BoolReturns true when the occupant x of a calibration slot is a Calibration Rule.
A slot can hold and call two kinds of rule: an AbstractCalibrationAlgorithm, and a plain function of the same five arguments. Both return true. A CountOf or a FractionOf in a slot whose bound admits it also returns true, because resolve_calibration_slot computes the quantity from it. All other occupants return false: a stated number, nothing, and a child measure that a container declares.
A callable is not always a rule. calibration_slots also declares the children of a container. One such child is an ordered-weights builder in a reversal, which is a Function and computes no quantity. So this trait marks a rule, and the Function type does not. A type that is callable for a different purpose adds one method here that returns false. OWA_CalOccupant names the type that does this.
Three functions read the trait:
resolve_calibration_slotcalls a rule and returns every other occupant unchanged.assert_calibrated_slotsrefuses a rule that reached a value-level entry point.assert_declared_calibration_resolverrefuses a rule that the library did not resolve.
Arguments
x: The occupant of the slot.
Returns
y::Bool: Whetherxis a rule.
Related
PortfolioOptimisers.assert_calibrated_slots — Function
assert_calibrated_slots(x)Refuses a Calibration Rule that reached a value-level entry point, which has no prior result to resolve it against.
expected_risk takes a prior result or a plain returns matrix. With a prior result, it first resolves the measure through factory. With a matrix, it cannot. A rule reads the sample size, the moments and the effective observation weights of the prior, and a returns matrix gives none of them. So the function refuses the rule, and the message names the slot, the rule in it and the remedy.
assert_resolved_slots does the same check for a Deferred Quantity. Both messages name a type with nameof, because a printed type has a module prefix when its name is not visible from Main, and the message must be the same in every process. A rule that is a plain function has no name of its own, so the message then names the slot alone.
The slots come from calibration_slots, and the check recurses into each occupant. So the declarations of its children cover a container.
Algorithm
- Walk the pairs that
calibration_slotsdeclares forx. Each pair gives the namekeyof a slot and its occupantslot. - Refuse an occupant for which
is_calibration_rulereturnstrue. - Recurse into the occupant, so the check also reads the slots of a child measure. Walk a slot that holds a vector of children element by element.
- Return
nothingat the end of the walk.
Validation
- Throws an
ArgumentErrorwhen a slot ofx, or of a child that the walk reaches, holds a Calibration Rule. The message names the slot, the rule in it and the two remedies.
Returns
nothing.
Related
PortfolioOptimisers.assert_declared_calibration_resolver — Function
assert_declared_calibration_resolver(x, slots::NamedTuple)
assert_declared_calibration_resolver(x)Refuses a type that declares a calibration slot and does not resolve it.
slots is the output of the resolution. If a Calibration Rule remains in it, the type declared the slot in calibration_slots and wrote no resolution for it. With no check, the rule goes to the model builders, and they multiply it as a number. This function checks that each declaration has a resolution.
assert_declared_slot_resolver does the same check for a Deferred Quantity, and the two-argument method here has its form: the resolver gives the slots that it made. The one-argument method reads the slots from x. A caller that holds a resolved value, and not the slots that made it, calls this method.
The walk is the walk of assert_calibrated_slots, and the two messages are different. That message names a caller who reached a value-level entry point. This message names a slot that the library did not resolve. So each message serves one failure.
The library derives no recursion for the calibration slots, so the pairing is a check and not a derivation. A rule that reads a sibling slot must resolve after that sibling, and a derivation cannot know the order. So each type writes its resolution next to its declaration, and this function refuses a declaration that has no resolution.
Algorithm
- Walk the pairs of
slots. Each pair gives the namekeyof a slot and its occupantslot. - Refuse an occupant for which
is_calibration_rulereturnstrue. - Return
nothingat the end of the walk.
The one-argument method reads slots from calibration_slots, and then recurses into each occupant, so it also checks the declarations of a child measure. It walks a slot that holds a vector of children element by element.
Arguments
x: The slot owner, whose type the message names.slots: The slots that the resolution made, or the declaration for the one-argument method.
Validation
- Throws an
ArgumentErrorwhen an entry ofslots, or of a child that the walk reaches, still holds a Calibration Rule. The message names the type, the slot and the resolution to write next to the declaration.
Returns
nothing.
Related
PortfolioOptimisers.calibration_series — Function
calibration_series(x)Declares the series that x prices, so that a rule in one of its calibration slots reads that quantity.
The default is ReturnsSeries, so a type that prices the return distribution needs no method. The two relativistic drawdown measures add a method, because they price a series that is not the sample that they hold.
This is a trait of the slot owner and not a field of the rule. The quantity belongs to the measure, and a rule cannot know which measure it reached. The CalibrationContext holds the norm order for the same reason. No rule holds a marker, so the answer of the owner overwrites nothing. A caller who runs a rule directly states the marker in the context.
Arguments
x: The slot owner, usually a risk measure.
Returns
series::AbstractCalibrationSeries: The series that the owner prices.
Related
PortfolioOptimisers.calibration_series_vec — Function
calibration_series_vec(series::AbstractCalibrationSeries, x::VecNum)Returns the univariate series that series names, made from one column of the sample.
A drawdown marker makes a new vector. ReturnsSeries returns the column, so the returns series allocates nothing.
The function takes one column and not the whole matrix, because a drawdown is a function of a path. It accumulates down a column and does not read across a row. A caller that needs the pool walks the columns and calls this function once for each column.
Arguments
series: The series marker.x: One column of the sample.
Returns
s::VecNum: The series that the marker names.
Related
PortfolioOptimisers.calibration_series_matrix — Function
calibration_series_matrix(series::AbstractCalibrationSeries, X::AbstractMatrix)Returns the sample that series names, one series for each column of X.
A drawdown marker calls the array function of its convention along dims = 1, the dimension of the observations. AbsoluteDrawdownSeries calls absolute_drawdown_arr, and RelativeDrawdownSeries calls relative_drawdown_arr. Those functions hold the convention of the running peak, so the matrix form and the vector form use one definition of a drawdown. The result has the shape of X. A drawdown series has one entry for each observation, so no observation is lost, and the count of a tail that the caller made on X stays correct.
ReturnsSeries returns X, so the returns sample allocates nothing. A rule that walks the columns one at a time calls calibration_series_vec instead, and allocates one column and not a matrix.
Arguments
series: The series marker.X: Returns matrix,T × N.
Returns
Y::AbstractMatrix: The sample that the marker names,T × N.
Related
PortfolioOptimisers.calibration_series_dispersion — Function
calibration_series_dispersion(series::AbstractCalibrationSeries, pr::AbstractPriorResult)Returns the per-asset dispersion of the series that series names, one entry for each column of the sample.
A ReturnsSeries reads it from pr.sigma, as the square root of the diagonal of that matrix. So the dispersion of a prior that states a shrunk or a robust covariance matrix comes from that matrix.
A drawdown series reads it from the drawdown sample, because a prior result states no moment of a drawdown. calibration_series_matrix makes the sample, and the dispersion is the sample standard deviation of each of its columns. radial_series_inputs reads its centre and its whitening factor from the drawdown sample for the same reason. pr.sigma is a moment of the returns, and no scaling of it gives a moment of a drawdown.
A radius is a distance in the space of the scenarios that the model prices, so this vector gives the radius its units. ConcentrationRadius and DimensionalRateRadius average it into one scale. DualNormRadius divides it by the square root of the count that the site prices and takes a norm of the result.
Mathematical definition
\[\begin{align} \hat{s}_{i} &= \begin{cases} \sqrt{\hat{\mathbf{\Sigma}}_{ii}} & \textrm{for the returns series}\\ \sqrt{\dfrac{1}{T - 1} \sum\limits_{t=1}^{T} \left(y_{ti} - \bar{y}_{i}\right)^{2}} & \textrm{for a drawdown series} \end{cases}\,. \end{align}\]
Where:
- $\hat{s}_{i}$: Sample dispersion of the series the slot owner prices, over column $i$. It is $\sqrt{\hat{\mathbf{\Sigma}}_{ii}}$ under a
ReturnsSeries, and the dispersion of column $i$ of the drawdown sample under a drawdown marker. - $\hat{\mathbf{\Sigma}}_{ii}$: $i$-th diagonal entry of $\hat{\mathbf{\Sigma}}$.
- $y_{ti}$: Entry $t$ of the drawdown series of column $i$.
- $\bar{y}_{i}$: Sample mean of the drawdown series of column $i$.
- $T$: Number of observations.
Arguments
series: The series that the slot owner prices.pr: Prior result. A returns series reads its covariance matrix, and a drawdown series reads its sample.
Returns
s::AbstractVector: The per-asset dispersion of that series,N × 1.
Related
PortfolioOptimisers.effective_sample_size — Function
effective_sample_size(pr::AbstractPriorResult, w::Option{<:ObsWeights})Returns the number of observations behind the moments of pr, weighted by w.
A calibration rule that prices the sample size of the moments divides by this count, so those rules use one definition of it. The function reads three sources in order, and the first that gives a count wins:
- With weights, it is the effective sample size of [9]. That is the number of equally weighted observations that gives the precision of the weighted sample.
- With no weights, it is the count that the result states in
eoc, when the result has no weights of its own. It is the smaller of the counts of the mean and of the covariance, because a rule does not know which moment it prices, and the sample is worth no more than its less informed moment. A Scenario Cap states there the number of observations over which it fitted the moments, and a weighted estimator of anEmpiricalPriorstates the count of its weights. An entry ofnothingis the row count ofpr.X. - Otherwise, it is the row count of
pr.X, which is then the sample.
This function does not read an eoc that is next to a w. The weights w bind that eoc, because a pooling prior refits its moments under them. So a rule that reads no weights reads no count of them either. The function never reads ens, which is a diagnostic of w.
A rule that prices a property of the scenarios of the measure does not read this count. A CountOf in a significance slot, RateSignificance and EntropyBudget read the rows that the measure holds when no weights are stated. A distributionally robust measure states the count of its rows, from scenario_sample_size, in the n of its CalibrationContext, and a radius rule reads that count in place of this one.
The function reads the weights through get_observation_weights, so it accepts all of the ObsWeights union that the w field of a rule admits. A DynamicAbstractWeights that has no method for a column of the sample raises ObservationWeightsError, which names the two signatures to write.
Mathematical definition
\[T_{e} = \begin{cases} \dfrac{\left(\sum\limits_{t=1}^{T} w_{t}\right)^{2}}{\sum\limits_{t=1}^{T} w_{t}^{2}} & \textrm{if } w \textrm{ is given}\\ \min\left(T_{\mu},\, T_{\Sigma}\right) & \textrm{if } w \textrm{ is } \texttt{nothing} \textrm{ and the result states no weighting}\\ T & \textrm{otherwise} \end{cases}\,.\]
Where:
- $T$: Number of observations.
- $T_{e}$: Effective sample size, which is Kish's when the observation weights are stated.
- $w_{t}$: Observation weight of observation $t$.
- $T_{\mu}$, $T_{\Sigma}$: The effective numbers of observations of the mean and of the covariance that the result states in
eoc. An entry ofnothingis $T$.
Algorithm
- Resolve
wagainst the first column ofpr.Xthroughget_observation_weights, which givesws. - When
wsis notnothing, return the count of Kish for it, throughweight_observation_count. - Otherwise, return
stated_sample_sizeof thew, theeocand theXof the result. That is the smaller of the counts of the mean and of the covariance when the result has nopr.w, andsize(pr.X, 1)otherwise.
Arguments
pr: Prior result from which the function reads the sample length.w: Effective observation weights, ornothing.
Validation
wresolves throughget_observation_weights. That function raisesObservationWeightsErrorfor aDynamicAbstractWeightsthat has no method for a column ofpr.X.
Returns
T_e::Number: The effective sample size.
Related
get_observation_weightsstated_sample_sizeObsWeightsscenario_sample_size: the count of the rows that a measure prices.resolve_calibration_slot: aCountOfin a significance slot reads this count only when observation weights are stated.ConcentrationRadius: reads this count when the site states non.RateRadiusDimensionalRateRadiusDualNormRadius
References
- [9] L. Kish. Survey Sampling (John Wiley & Sons, 1965). Whole book. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.stated_sample_size — Function
stated_sample_size(::Nothing, eoc::Tuple, X::MatNum)
stated_sample_size(w, eoc, X::MatNum)Returns the count that a Prior Result states in eoc next to no weights, and the row count of X otherwise.
It is the branch of effective_sample_size for no weights, and it dispatches on the w of the result. With no w, the count is the smaller of the counts of the mean and of the covariance, eoc[1] and eoc[2], and an entry of nothing is size(X, 1). A Scenario Cap writes there the number of observations behind its moments, and a weighted estimator the count of its weights. The weights w bind an eoc next to them, and a read with no weights does not take it. A result that states no count holds its sample in X.
Arguments
w: The observation weights of the result, ornothing.eoc: The counts that the result states. AHighOrderPriorstates four, and the function reads the first two.X: The returns matrix of the result,observations × assets.
Returns
T::Number: The smaller count of the mean and the covariance when the result has now, andsize(X, 1)otherwise.
Related
PortfolioOptimisers.scenario_sample_size — Function
scenario_sample_size(pr::AbstractPriorResult, w::Option{<:ObsWeights})Returns the count of the empirical measure of the rows that pr holds, weighted by w.
A risk measure prices the rows of pr.X, and not the observations behind the moments. With weights, the measure is the weighted empirical measure of those rows, so the count is the effective sample size of [9], as in effective_sample_size. With no weights, the measure gives each row the same mass, so the count is the row count of pr.X.
The two counts differ under a Scenario Cap, and under a weighted estimator of the moments. A cap fits the moments over every observation, states that number in eoc, and holds fewer rows. effective_sample_size reads eoc there, and this function reads the rows. The eoc of an entropy-pooling prior is next to a w, so neither count reads it.
The function resolves w through get_observation_weights, as the programme of a measure does. So a DynamicAbstractWeights that resolves to nothing gives the row count, which is the count of the measure that the programme then prices.
Mathematical definition
\[T_{n} = \begin{cases} \dfrac{\left(\sum\limits_{t=1}^{T} w_{t}\right)^{2}}{\sum\limits_{t=1}^{T} w_{t}^{2}} & \textrm{if } w \textrm{ is given}\\ T & \textrm{otherwise} \end{cases}\,.\]
Where:
- $T$: Number of observations. It is the row count of the sample that the result holds.
- $w_{t}$: Observation weight of observation $t$.
- $T_{n}$: Scenario count of the measure, the number of scenarios that it prices.
Algorithm
- Resolve
wagainst the first column ofpr.Xthroughget_observation_weights, which givesws. - When
wsisnothing, returnsize(pr.X, 1). - Otherwise, return the count of Kish for
ws, througheffective_sample_size.
Arguments
pr: Prior result whose rows the measure prices.w: Effective observation weights, ornothing.
Returns
T_n::Number: The count of the scenarios.
Related
effective_sample_sizeresolve_calibration_slotCalibrationContextDistributionallyRobustConditionalValueatRisk
References
- [9] L. Kish. Survey Sampling (John Wiley & Sons, 1965). Whole book. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.effective_asset_floor — Function
effective_asset_floor(part::CountOf, N::Integer)
effective_asset_floor(part::FractionOf, N::Integer)Returns the effective number of assets that a norm ceiling keeps as a floor, part.val for a CountOf and part.val * N for a FractionOf of a universe of N assets.
The floor is not rounded, because the effective number of assets is a real number.
Related
PortfolioOptimisers.hill_tail_index — Function
hill_tail_index(
series::AbstractCalibrationSeries,
X::AbstractArray{E<:Number, 2},
s::Integer,
k::Integer
) -> Any
Estimates the tail index of the pool of standardised values of X, over the k most extreme order statistics of the pool.
series names the series of each column, and calibration_series_vec makes it. The function centres each series and divides it by its sample standard deviation, and s names the end. The estimator of [180] reads the k of the T N standardised values that are furthest into that end. HillTailDecay states the assumptions of the pool. This function makes the estimate alone.
The pool holds T values for each column under every marker, because a drawdown series has one entry for each observation. So the function reads the count k that the caller made, for every series.
The signature binds the element type, so the pool and its sum have concrete types. A rule reads pr.X from an AbstractPriorResult, whose field types no signature states, and this function is the boundary where the type becomes concrete.
Algorithm
- Read the shape of
XintoTandN, and allocate the poolpoolofT * Nvalues. - Walk the columns of
X. Make the series of each column withcalibration_series_vec, take its sample meanmuand its sample standard deviationsd, and write-s * (col[t] - mu) / sdintopool. - Partially sort
pool, so that itsk + 1smallest values are in order at the start, and read the last of them intovkp1.partialsort!with one index puts only that index in place, so the range is necessary to put thekentries of the estimate beforevkp1. The sign of step 2 puts the end that the caller prices in the lower tail of the pool. - Return
kdivided by the sum oflog(pool[i] / vkp1)over thosekentries. Both terms of each ratio are negative, so the ratio is a ratio of magnitudes, and the sum needs no change of sign.
Arguments
series: The series of each column.X: Returns matrix,T × N.s: Sign of the end that the estimate reads.1reads the gain tail, and-1reads the loss tail.k: Number of order statistics that the estimate reads. The caller states it and checks its floor.
Validation
- The
k + 1-th largest pooled value must be positive.
Returns
a::Number: The Hill estimate of the tail index of the pool.
Related
References
- [180] B. M. Hill. A simple general approach to inference about the tail of a distribution. The Annals of Statistics 3, 1163–1174 (1975). Pages 1163–1174. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.series_end_sign — Function
series_end_sign(series::AbstractCalibrationSeries, key::Symbol)Returns the sign of the end that key prices on series, and refuses an end that the series does not have.
:kappa_b is the only head key, so on a returns series every other key prices the loss end. A drawdown series is not positive and has only one end. So the head key names no end on it, and the function refuses it. The library has no drawdown Range measure, so only a caller who runs a rule directly gets this refusal.
Arguments
series: The series of the estimate.key: Name of the slot that the call resolves.
Validation
keymust not be:kappa_bwhenseriesis anAbstractDrawdownSeries.
Returns
s::Int:1for the gain end and-1for the loss end.
Related
PortfolioOptimisers.whitening_factor — Function
whitening_factor(pr::AbstractPriorResult) -> Any
Returns an upper triangular square factor of the covariance matrix of pr, so that the transpose of the factor whitens a centred observation.
The factor $\mathbf{U}$ satisfies $\mathbf{U}^{\top} \mathbf{U} = \hat{\mathbf{\Sigma}}$. That is the convention of the chol field of a prior result, and the variance constraint layer uses it too. chol has precedence over sigma, as the chol field states. So the function whitens a prior that has a factor with that factor, and never with a factorisation of sigma. chol is optional, so usually the function factorises sigma.
The library checks chol only against the length of mu, so it can be $K \times N$ and not square, and a square chol need not be triangular. The function needs a triangular factor, because only the diagonal of a triangular factor gives its rank:
- A
cholof typeLinearAlgebra.UpperTriangularis the factor. - Every other
cholwith $K \geq N$ gives the $\mathbf{R}$ factor of its QR factorisation. $\mathbf{R}^{\top} \mathbf{R} = \mathbf{F}^{\top} \mathbf{F}$, so $\mathbf{R}$ is a triangular factor of the same covariance matrix. This includes a tall factor, and a square factor that is not triangular, such as a symmetric square root. - A wide
chol, with $K < N$, gives a singular covariance matrix, and no whitening exists for it.
RadialTailDecay calls this function. The function is separate because the precedence rule and its refusals are facts about a prior result and not about a tail index.
Algorithm
- Read
pr.cholintoF. - When
Fisnothing, factorisepr.sigmawith a Cholesky factorisation that reports a failure and does not raise it, and return its upper factor. - Otherwise, refuse a wide
F. TakeFwhen it is anUpperTriangular, and theRfactor of its QR factorisation otherwise. - Refuse a zero on the diagonal of that triangular factor, because it gives a singular covariance matrix, and return the factor.
Arguments
pr: Prior result from which the function reads the covariance matrix.
Validation
pr.sigmamust be positive definite, whenpr.cholisnothing.pr.cholmust have at least as many rows as columns.- The diagonal of the triangular factor must have no zero.
Returns
U::AbstractMatrix: Upper triangular square factor of the covariance matrix.
Related
PortfolioOptimisers.radial_tail_index — Function
radial_tail_index(
X::AbstractArray{E<:Number, 2},
mu::AbstractVector,
U::AbstractMatrix,
k::Integer
) -> Any
Estimates the tail index of the radial series of X, over the k largest order statistics of that series.
The function centres each row on mu and whitens it with U. The norm of the whitened row is one entry of the radial series. The estimator of [180] then reads the k entries that are furthest out. RadialTailDecay states what the series measures. This function makes the estimate alone.
The signature binds the element type, so the series and its sum have concrete types. A rule reads pr.X from an AbstractPriorResult, whose field types no signature states, and this function is the boundary where the type becomes concrete.
Algorithm
- Solve
transpose(U) \ transpose(X .- transpose(mu)). The columns of the resultZare the whitened observations. - Walk the columns of
Z, and write the negative Euclidean norm of each column into the seriesd. - Partially sort
d, so that itsk + 1smallest values are in order at the start, and read the last of them intovkp1.partialsort!with one index puts only that index in place, so the range is necessary to put thekentries of the estimate beforevkp1. The negative sign of step 2 puts the largest distances at the start. - Return
kdivided by the sum oflog(d[i] / vkp1)over thosekentries. Both terms of each ratio are negative, so the ratio is a ratio of magnitudes, and the sum needs no change of sign.
Arguments
X: Returns matrix,T × N.mu: Expected returns vector,N × 1. The function centres the rows on it.U: Upper triangular square factor of the covariance matrix, whichwhitening_factorreturns.k: Number of order statistics that the estimate reads. The caller states it and checks its floor.
Validation
- The
k + 1-th largest radial distance must be positive.
Returns
a::Number: The Hill estimate of the tail index of the radial series.
Related
References
- [180] B. M. Hill. A simple general approach to inference about the tail of a distribution. The Annals of Statistics 3, 1163–1174 (1975). Pages 1163–1174. The original source of this formulation was not found. The formulation follows this work and is not checked against the original.
PortfolioOptimisers.radial_series_inputs — Function
radial_series_inputs(series::AbstractCalibrationSeries, pr::AbstractPriorResult)Returns the sample, the centre and the whitening factor from which RadialTailDecay makes the radial series of series.
A ReturnsSeries reads the three from the prior result: pr.X, pr.mu and the factor that whitening_factor takes from pr.chol or pr.sigma.
A drawdown series reads the three from the drawdown sample, because a prior result states no moment of a drawdown. calibration_series_matrix makes the sample. The centre is the vector of column means of that sample, and the factor is the Cholesky factor of its covariance matrix. So all three come from the quantity that the measure prices. pr.mu and pr.sigma are not read, because they are the moments of the returns, and no scaling of them gives the moments of a drawdown.
Arguments
series: The series that the slot owner prices.pr: Prior result. A returns series reads its sample and its moments, and a drawdown series reads its sample.
Validation
- The covariance matrix of the drawdown sample must be positive definite. A column that does not move fails this check. Two columns with the same path also fail it.
Returns
Y::AbstractMatrix: The sample that the marker names,T × N.mu::AbstractVector: The centre of that sample,N × 1.U::AbstractMatrix: Upper triangular square factor of its covariance matrix.
Related
PortfolioOptimisers.dual_norm_radius_scale — Function
dual_norm_radius_scale(key::Symbol, p::Option{<:Number}, e::AbstractVector)Reduces the per-asset error vector e to one number, in the ground metric that key names.
This function holds all the meaning of the key, apart from the functor. Six of the seven keys name a fixed norm order, so they are a literal table and not a chain of branches. The seventh key, :lpreg_val, gets its order from the p of the context, because no key can name it. The function refuses any other key. A caller who writes a measure gets this refusal first, so the message names the keys that the rule serves.
Arguments
key: Name of the slot that the call resolves.p: Norm order that the site states in itsCalibrationContext. Only the:lpreg_valbranch reads it.e: Per-asset scale of the sampling error of the mean vector.
Validation
keymust be one of the seven keys thatDualNormRadiusserves, else anArgumentError.pmust be stated whenkeyis:lpreg_val, else anArgumentError. It must then be finite and more than one, so that the conjugate orderq = p / (p - 1)exists.
Returns
s::Number: The norm ofein the ground metric of the slot.
Related
References
- [9]
- L. Kish. Survey Sampling (John Wiley & Sons, New York, 1965).
- [180]
- B. M. Hill. A simple general approach to inference about the tail of a distribution. The Annals of Statistics 3, 1163–1174 (1975).