Coverage Policy: private API
PortfolioOptimisers.CoverageCounts — Type
struct CoverageCounts{__T_nu, __T_centre, __T_active, __T_stale}Carries the per-cell denominators and the per-asset bookkeeping of an available-case fold.
A partial-fit state holds it in its cvg field. On the plain path that field is nothing, so the plain state costs nothing. nu has the shape of the accumulator it serves, because the observation count of a covariance cell is a count per pair and not a count per asset. centre has that shape too wherever the centre of a cell differs from the per-asset mean of the state.
This type is an implementation detail and is not intended for direct use. partial_fit! writes it, the batch verbs called with no data divide by it, and merge_states folds two of them.
Fields
nu:nu: Per-cell surviving observation count, of the shape of the accumulator it serves.
centre:centre: Per-cell running mean, of the shape of the accumulator it serves, ornothingwhen the state's ownmuis already the cell's centre. Entry(i, j)is the running mean of assetiover the observations at which the pair(i, j)is valid, so entry(j, i)is the mean of assetjover those same observations.
active:active: Active mask of the last observation folded, one entry per asset.
stale:stale: Number of observations folded since each asset was last finite and active, one entry per asset.
Related
PortfolioOptimisers.coverage_counts_seed — Function
coverage_counts_seed(cvg::Nothing, counts, N::Integer, Tf::Type,
pairwise::Bool) -> Nothing
coverage_counts_seed(cvg::CoveragePolicy, counts::CoverageCounts, N::Integer, Tf::Type,
pairwise::Bool) -> CoverageCounts
coverage_counts_seed(cvg::CoveragePolicy, counts::Nothing, N::Integer, Tf::Type,
pairwise::Bool) -> CoverageCountsReturns the CoverageCounts an available-case fold writes into, seeding one of zeros when the state carries none.
partial_fit! calls it, so one function holds the branch on the cvg field of a state. The policy is the first argument because it selects the arm. nothing is the plain path and gives nothing, whatever the state carries.
Arguments
cvg: The policy the estimator carries.counts: The component the state carries, ornothing.N: Number of assets.Tf: Element type of the accumulators.pairwise: Whether the cell axis is a pair, so thatnuandcentreareN × Nrather thanN-long.
Returns
counts::Option{<:CoverageCounts}: The componentcountsholds, or a fresh one of zeros.
Related
Base.copy — Method
copy(x::CoverageCounts) -> CoverageCounts
Copies a CoverageCounts, so that the copy shares no array with the original.
Arguments
x: The component to copy.
Returns
counts::CoverageCounts: A fresh component, equal tox, whose arrays are fresh.
Related
PortfolioOptimisers.coverage_counts_view — Function
coverage_counts_view(x::Nothing, i) -> Nothing
coverage_counts_view(x::CoverageCounts, i) -> CoverageCountsSlices a CoverageCounts to the selected assets.
A per-cell count reads the assets of its own cell alone, so the slice of the component is the component of the sliced universe, entry for entry. The slice copies by index and does not view, as port_opt_view does for every partial-fit state.
Arguments
x: The component to slice, ornothing.i: Index or indices of the assets to keep.
Returns
counts::Option{<:CoverageCounts}: The component of the same sample over the selected assets.
Related
PortfolioOptimisers.coverage_valid — Function
coverage_valid(
x::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
active_mask::Union{Nothing, AbstractVector{<:Bool}},
counts::CoverageCounts
) -> Tuple{Any, Any}
Reads the assets that are valid at one observation, and the assets that have just gone inactive.
An asset is valid when its return is finite and the active mask admits it. The exponentially weighted family reads the same condition. With no mask every asset is active, so a non-finite return reads as a holiday and not as a delisting, and no asset ever goes inactive.
Arguments
x: One observation, one entry per asset.active_mask: The active mask of the Asset Panel at this observation, ornothing.counts: The component whoseactivefield holds the mask of the previous observation.
Validation
active_mask, when it is given, has one entry per asset. ADimensionMismatchis thrown otherwise.
Returns
(valid, ni)::Tuple: The valid assets, and the assets that were active at the previous observation and are inactive at this one.
Related
PortfolioOptimisers.coverage_valid_block — Function
coverage_valid_block(
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
active_mask::Union{Nothing, AbstractMatrix{<:Bool}},
alg::AbstractCoverageAlgorithm;
dims
) -> NTuple{5, Any}
Reads everything a two-pass available-case fit needs out of a block of observations.
It is the block counterpart of coverage_valid, fold_inactive! and coverage_step!, for the arms that have no incremental recursion and must see the whole window at once. It returns the oriented block beside the four quantities that a fold carries, so that a two-pass arm and a folding arm read the same universe and the same bookkeeping.
Algorithm
- Orient
X, and the mask when there is one, toobservations × assets. - Take the valid entries, finite and active, giving
F. - Take the active mask of the last observation, giving
active, and the number of observations after the last valid entry of each asset, givingstale. An asset with no valid entry has a staleness ofT, the number of observations. - Apply the fold-time rule of
algtoFwithcoverage_inactive_block!. - Take the available-case mean of each asset over its own entries of
F, givingmu. It is zero for an asset with none.
Arguments
X: The block of observations.active_mask: The active mask of the Asset Panel over the block, ornothing.alg: The coverage algorithm of the policy.dims: Whether the observations lie on the rows,1, or on the columns,2.
Validation
active_mask, when it is given, has the shape ofX. ADimensionMismatchis thrown otherwise.
Returns
(Xo, F, mu, active, stale)::Tuple: The oriented block, its valid entries after the fold-time rule, each asset's available-case mean, the active mask of the last observation, and each asset's staleness at it.
Related
PortfolioOptimisers.coverage_inactive_block! — Function
coverage_inactive_block!(alg::AbstractCoverageAlgorithm, F::AbstractMatrix{Bool},
amsk::Option{<:AbstractMatrix{<:Bool}}) -> Nothing
coverage_inactive_block!(alg::ResetCoverage, F::AbstractMatrix{Bool},
amsk::AbstractMatrix{<:Bool}) -> NothingApplies the fold-time rule of a coverage algorithm to the valid entries of a whole block.
It is the block form of fold_inactive!, for the two-pass arms that never fold. A fold applies the rule at each observation where an asset goes inactive. A block applies the same rule to the valid entries of the window at once, so that the two arms admit the same observations.
The first method passes the block through. It serves DecayCoverage and ExpireCoverage, which keep the history, and ResetCoverage with no active mask, where no asset ever goes inactive. It also serves a coverage algorithm of the caller's own, whose fold_inactive! method the two-pass arms cannot call, so such an algorithm keeps the history in those arms.
Algorithm
The ResetCoverage method runs these steps for each asset j:
- Find the last observation
t0at which the asset goes inactive, where the asset is inactive att0and active at the observation before it. The observation before the first one counts as active, as it does for the seed of a fold. - Clear the valid entries of the asset at every observation up to
t0. An asset that never goes inactive hast0 = 0, and keeps every entry.
Arguments
alg: The coverage algorithm.F: The valid entries of the block,observations × assets, mutated in place.amsk: The active mask of the block in the orientation ofF, ornothing.
Returns
nothing.
Related
PortfolioOptimisers.coverage_step! — Function
coverage_step!(
counts::CoverageCounts,
valid::AbstractVector{<:Bool},
active_mask::Union{Nothing, AbstractVector{<:Bool}}
)
Moves the per-asset bookkeeping of a CoverageCounts on by one observation.
Algorithm
- Set the staleness of every valid asset to zero, and add one to the staleness of every other.
- Overwrite
activein place with the mask of this observation, or withtruefor every asset when the caller gave no mask.
Arguments
counts: The component to move on, mutated in place.valid: The valid assets of this observation.active_mask: The active mask of this observation, ornothing.
Returns
nothing.
Related
PortfolioOptimisers.coverage_merge_stale — Function
coverage_merge_stale(
a::CoverageCounts,
b::CoverageCounts,
n_b::Integer
) -> Any
Folds the staleness counters of two CoverageCounts fitted on consecutive blocks.
An asset that was valid somewhere in the second block takes the count of that block. An asset that was valid nowhere in it was stale for the whole block, so its staleness is the sum of the two counts. The counter of b reaches the observation count of b exactly when the asset was valid nowhere in b, because only a valid observation sets the counter back to zero.
Mathematical definition
\[\begin{align} \tau_{i} &= \begin{cases} \tau_{i}^{a} + \tau_{i}^{b} & \text{if } \tau_{i}^{b} = n_{b}\,, \\ \tau_{i}^{b} & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $\tau_{i}$: Staleness of asset $i$, the number of observations folded since the asset was last finite and active.
- $\tau_{i}^{a}$, $\tau_{i}^{b}$: Staleness of asset $i$ in
aand inb. - $n_{b}$: Number of observations folded into
b.
Arguments
a: The component of the first block of observations.b: The component of the second block of observations.n_b: The number of observations folded intob.
Returns
stale::Vector{<:Integer}: The staleness of each asset over the concatenated block.
Related
PortfolioOptimisers.coverage_reset! — Function
coverage_reset!(A::Nothing, ni::AbstractVector{<:Bool}) -> Nothing
coverage_reset!(A::AbstractVector, ni::AbstractVector{<:Bool}) -> Nothing
coverage_reset!(A::AbstractMatrix, ni::AbstractVector{<:Bool}) -> NothingZeroes the entries of an accumulator that touch a set of assets.
The ResetCoverage methods of fold_inactive! share it, so the reset of a per-asset accumulator and the reset of a per-pair accumulator are each written once.
Arguments
A: The accumulator or count array to zero, mutated in place.nothingpasses through, which is thecentreof a per-asset state.ni: One entry per asset,truewhere the asset is to be zeroed.
Returns
nothing.
Related
PortfolioOptimisers.coverage_admission — Function
coverage_admission(
cvg::CoveragePolicy,
counts::CoverageCounts,
n::Integer
) -> Union{Nothing, BitVector}
Reads the universe of an available-case fit out of its per-cell counts.
It makes the mask of the estimate from the state, and it is the one place that calls admits. It collapses onto the nothing sentinel exactly as coverage_sentinel does, so an answer that admits every asset allocates no mask. Unlike coverage_sentinel it refuses no sample. An available-case fit whose window admits nothing answers all NaN and does not raise, because the fit is opt-in and its caller asked for the gaps.
Mathematical definition
\[\begin{align} s_{i} &= \frac{\nu_{i}}{\max(n, 1)}\,. \end{align}\]
Where:
- $s_{i}$: Coverage share of asset $i$, its own observation count over the number of observations folded.
- $\nu_{i}$: Observation count of asset $i$, the diagonal entry $\nu_{ii}$ when the counts are per pair.
- $n$: Number of observations folded. A state that folded none gives every asset a share of zero.
Algorithm
- Take the per-asset observation count
nu, which is the diagonal when the counts are per pair. - Take the coverage share of each asset.
- Call
admitsonce per asset, giving the maskcmsk. - Return
nothingwhen every asset is admitted, andcmskotherwise.
Arguments
cvg: The policy the estimator carries.counts: The per-cell counts the state carries.n: The number of observations folded, which is the share's denominator.
Returns
cmsk::Option{BitVector}: The admitted assets, ornothingwhen every asset is admitted.
Related
PortfolioOptimisers.coverage_peel — Function
coverage_peel(
cvg::CoveragePolicy,
cmsk::Union{Nothing, BitVector},
nu::AbstractMatrix{<:Integer},
corrected::Bool,
strict::Bool
) -> Any
Peels the assets of the undetermined pairs out of the universe of an available-case covariance.
It is the pairwise half of admission, and it runs after coverage_admission in every available-case covariance. A pair is undetermined when both of its assets are admitted and have a variance, and the pair shares too few observations for a covariance. coverage_divide would write NaN at such a cell inside the finite block, and the matrix repair would refuse it. The peel rule of the policy chooses the assets to remove, and they leave the mask, so every consumer of the result reads one universe.
Mathematical definition
\[\begin{align} \mathcal{V} &= \left\{ i \in \mathcal{A} : \nu_{ii} - \delta \geq 1 \right\}\,, \\ \mathcal{E} &= \left\{ (i, j) \in \mathcal{V}^{2} : i \neq j,\ \nu_{ij} - \delta < 1 \right\}\,. \end{align}\]
Where:
- $\mathcal{V}$: Vertices of the graph of undetermined pairs, the admitted assets that have a variance.
- $\mathcal{E}$: Edges of the graph of undetermined pairs, the pairs of vertices that share too few observations for a covariance.
- $\mathcal{A}$: Admitted assets, those that
admitslets into the answer. A cell is admitted when every asset of the cell is in $\mathcal{A}$. - $\nu_{ij}$: Number of observations at which assets $i$ and $j$ are both valid.
- $\delta$: The Bessel correction, $1$ when
correctedistrueand $0$ otherwise.
Algorithm
- Take the vertices
V, the admitted assets whose variance has a denominator of at least one. - Take the adjacency matrix
Uof the undetermined pairs among them. Returncmskwhen it holds no edge. - Take the assets to remove with
peel_assets. Returncmskwhen the rule removes none, aspeel = nothingdoes. - Report the peel through
strict_diagnostic, which warns, or refuses understrict. - Remove the peeled assets from the mask.
Arguments
cvg: The policy the estimator carries.cmsk: The admitted assets fromcoverage_admission, ornothingwhen every asset is admitted.nu: The per-pair observation count,assets × assets.corrected: The Bessel correction of the covariance.strict: Whether a peel refuses rather than warns.
Validation
- Under
strict, the rule peels no asset. AnArgumentErrorthat names the assets is thrown otherwise.
Returns
cmsk::Option{BitVector}: The admitted assets after the peel, ornothingwhen every asset is admitted.
Related
PortfolioOptimisers.peel_msg — Function
peel_msg(
idx::AbstractVector{<:Integer},
m::Integer,
peel::AbstractPeel
) -> String
Builds the text of the report of coverage_peel.
Arguments
idx: The peeled assets, as indices of the universe of the fit.m: The number of undetermined pairs before the peel.peel: The peel rule that chose the assets.
Returns
msg::String: The message, which names the assets, the count of pairs and the rule.
Related
PortfolioOptimisers.peel_max_degree — Function
peel_max_degree(
U::AbstractMatrix{Bool},
alive::AbstractVector{Bool}
) -> Tuple{Any, Any}
Finds the vertex of the largest degree among the vertices still in a graph, the smallest index on a tie.
Arguments
U: The adjacency matrix of the graph.alive: One entry per vertex,truewhere the vertex is still in the graph.
Returns
(v, d)::Tuple{Int, Int}: The vertex and its degree over the vertices still in the graph.vis0anddis0when no edge is left.
Related
PortfolioOptimisers.peel_lexicographic — Function
peel_lexicographic(
U::AbstractMatrix{Bool},
budget::Integer
) -> Vector{Int64}
Builds the lexicographically smallest vertex cover of a given size, when a cover of that size exists.
It is step 3 of MinimalPeel. It visits the vertices in index order, and removes each vertex for which a cover of the remaining size still exists. A smaller index taken first gives the smaller sorted index vector, so the cover that the visit completes is the lexicographically smallest one.
Algorithm
- Skip a vertex that is no longer in the graph, or that has no edge left.
- Remove the vertex when
peel_cover_withinfinds a cover of the remaining size without it, and lower the size by one. - Otherwise keep it. Every cover of the size that keeps it removes each of its neighbours, so remove them and lower the size by their number.
Arguments
U: The adjacency matrix of the graph.budget: The size of the cover. A cover of this size exists.
Returns
idx::Vector{Int}: The vertices of the cover, sorted.
Related
PortfolioOptimisers.peel_cover_within — Function
peel_cover_within(
U::AbstractMatrix{Bool},
alive::AbstractVector{Bool},
k::Integer
) -> Bool
Decides whether the vertices still in a graph have a vertex cover of at most k vertices.
It is the exact search of MinimalPeel, a bounded search tree on the vertex of the largest degree.
Algorithm
- Return
falsewhenkis negative. - While a vertex has degree one, put its neighbour in the cover and lower
kby one, withpeel_pendants!. - Take the vertex
vof the largest degreedwithpeel_max_degree. Returntruewhendis zero. - Return
falsewhen the graph has more thank dedges, because each vertex of a cover covers at mostdedges. - When
dis two, every vertex left with an edge has degree two, so each component is a cycle. Comparekwith the size of the cover of the cycles, frompeel_cycle_cover. - A cover either holds
v, or holds every neighbour ofv. Search the graph withoutvat the sizek - 1, then the graph withoutvand its neighbours at the sizek - d.
The branch of step 6 has d >= 3, so the tree has at most $1.47^{k}$ leaves.
Arguments
U: The adjacency matrix of the graph.alive: One entry per vertex,truewhere the vertex is still in the graph. It is not mutated.k: The largest size of the cover.
Returns
found::Bool: Whether a vertex cover of at mostkvertices exists.
Related
PortfolioOptimisers.peel_pendants! — Function
peel_pendants!(
U::AbstractMatrix{Bool},
alive::AbstractVector{Bool},
k::Integer
) -> Any
Puts the neighbour of each vertex of degree one into the cover, until no such vertex is left or the size is spent.
It is step 2 of peel_cover_within. Some minimum cover holds the neighbour, because the edge needs one of its two ends and the neighbour covers at least as many edges. Each removal can leave a new vertex of degree one, so the rule repeats.
Arguments
U: The adjacency matrix of the graph.alive: One entry per vertex,truewhere the vertex is still in the graph, mutated in place.k: The largest size of the cover.
Returns
k::Integer: The size left, which is negative when the rule spent more thank.
Related
PortfolioOptimisers.peel_cycle_cover — Function
peel_cycle_cover(
U::AbstractMatrix{Bool},
alive::AbstractVector{Bool}
) -> Int64
Returns the size of the minimum vertex cover of a graph whose components are cycles.
peel_cover_within calls it when every vertex left with an edge has degree two. A cycle of c vertices needs cld(c, 2) of them, and the cycles share no vertex, so the sizes add.
Arguments
U: The adjacency matrix of the graph.alive: One entry per vertex,truewhere the vertex is still in the graph. Every vertex still in the graph has degree zero or two.
Returns
k::Int: The size of the minimum vertex cover.
Related
PortfolioOptimisers.coverage_divide — Function
coverage_divide(
M::AbstractArray,
nu::AbstractArray,
corrected::Bool,
cmsk::Union{Nothing, BitVector}
) -> Any
Divides an available-case accumulator by its per-cell denominator, and frames the assets the policy refuses.
Every available-case estimate from a state ends with it. A cell whose denominator has not reached one is NaN, which is the per-cell half of the rule. An asset that coverage_admission refuses is NaN across its whole row and column, which is the per-asset half.
The element type of the answer is the type of the division itself, so a Float32 accumulator reads out as Float32, and the sentinel does not widen it. An element type that cannot hold NaN cannot hold the answer either. An exact accumulator, a Rational among them, raises an InexactError at the first cell that the policy refuses, and does not read out as Float64. A window with no refused cell is unaffected.
Mathematical definition
\[\begin{align} v_{k} &= \begin{cases} \dfrac{M_{k}}{\nu_{k} - \delta} & \text{if } \nu_{k} - \delta \geq 1 \text{ and cell } k \text{ is admitted}\,, \\ \mathrm{NaN} & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $v_{k}$: Entry $k$ of the answer of an available-case fit.
- $M_{k}$: Entry $k$ of the accumulator.
- $\nu_{k}$: Observation count of cell $k$, the number of observations at which every asset of the cell is finite and active.
- $\delta$: The Bessel correction, $1$ when
correctedistrueand $0$ otherwise. - $\mathcal{A}$: Admitted assets, those that
admitslets into the answer. A cell is admitted when every asset of the cell is in $\mathcal{A}$.
Algorithm
- Divide each entry of
Mby its corrected denominator, givingval, withNaNwhere the corrected denominator is below one. - Write
NaNinto every entry ofvalthat touches an asset outsidecmsk, withcoverage_refuse!.
Arguments
M: The accumulator, of the shape of the answer.nu: The per-cell denominator, of the shape ofM.corrected: The Bessel correction, subtracted from every denominator.cmsk: The admitted assets, ornothing.
Returns
val::Array{<:AbstractFloat}: The answer, of the shape ofM, carryingNaNwhere the fit has no number.
Related
PortfolioOptimisers.coverage_frame — Function
coverage_frame(
A::AbstractArray,
nu::AbstractArray,
cmsk::Union{Nothing, BitVector}
) -> Any
Frames an available-case estimate that needs no division, and frames the assets the policy refuses.
It reads out a quantity that is already a ratio, such as a running per-asset mean. It copies the value where its cell has an observation and writes NaN where the cell has none, and coverage_refuse! then writes the frame. It is separate from coverage_divide so that a Welford mean is never multiplied by its count and divided by it again, which would change the last bits of the mean that the fold computed.
The answer carries the element type of A, which the fold derived from its own division. As in coverage_divide, an exact element type cannot hold the NaN sentinel, and it raises an InexactError at the first cell that has no observation.
Mathematical definition
\[\begin{align} v_{k} &= \begin{cases} A_{k} & \text{if } \nu_{k} \geq 1 \text{ and cell } k \text{ is admitted}\,, \\ \mathrm{NaN} & \text{otherwise}\,. \end{cases} \end{align}\]
Where:
- $v_{k}$: Entry $k$ of the answer of an available-case fit.
- $A_{k}$: Entry $k$ of the estimate.
- $\nu_{k}$: Observation count of cell $k$, the number of observations at which every asset of the cell is finite and active.
- $\mathcal{A}$: Admitted assets, those that
admitslets into the answer. A cell is admitted when every asset of the cell is in $\mathcal{A}$.
Arguments
A: The estimate, of the shape of the answer.nu: The per-cell observation count, of the shape ofA.cmsk: The admitted assets, ornothing.
Returns
val::Array{<:AbstractFloat}: The answer, of the shape ofA, carryingNaNwhere the fit has no number.
Related
PortfolioOptimisers.coverage_refuse! — Function
coverage_refuse!(val::AbstractArray, cmsk::Nothing) -> Nothing
coverage_refuse!(val::AbstractArray, cmsk::BitVector) -> NothingWrites NaN into every entry of an answer that touches an asset the policy refuses.
It is the per-asset half of the rule that an available-case estimate applies, and coverage_divide and coverage_frame share it. A nothing mask is the sentinel of coverage_admission. It means that every asset is admitted, so the method writes nothing.
Arguments
val: The answer, mutated in place.cmsk: The admitted assets, ornothing.
Returns
nothing.
Related
PortfolioOptimisers.coverage_refuse_comoment! — Function
coverage_refuse_comoment!(val::AbstractMatrix, cmsk::Nothing, ::Val) -> Nothing
coverage_refuse_comoment!(val::AbstractMatrix, cmsk::BitVector, ::Val{:sk}) -> Nothing
coverage_refuse_comoment!(val::AbstractMatrix, cmsk::BitVector, ::Val{:kt}) -> NothingWrites NaN into every entry of a higher-order co-moment answer that touches an asset the policy refuses.
It is the co-moment form of coverage_refuse!, for an answer whose axes are not all the asset axis. A coskewness tensor is assets × assets², so its rows take the asset mask and its columns take the pair mask. A cokurtosis matrix is assets² × assets², so both of its axes take the pair mask. The Val marker names the shape, as it does for expand_moment. The mask stays one argument, so the arm is a plain dispatch and not a product of two Options.
A pair is admitted when both of its assets are. A co-moment tensor indexes the pair (i, j) at the column (i - 1) * N + j, which is the layout that coverage_pair_index states. kron(cmsk, cmsk) is that conjunction in that order, entry for entry.
Arguments
val: The answer, mutated in place.cmsk: The admitted assets, ornothingwhen every asset is admitted.::Val:Val(:sk)for a coskewness tensor andVal(:kt)for a cokurtosis matrix.
Returns
nothing.
Related