Coverage Policy

By default a moment estimator fits on the assets whose returns are finite and active at every observation of the window, the coverage universe. Put a CoveragePolicy in the cvg field of the estimator, and it fits each entry of its result on the observations where every asset of that entry is finite and active. An asset with a short history then still gets an estimate. An asset whose share of the observations is below min_coverage is NaN in the result.

An AbstractCoverageAlgorithm decides what happens to a delisted asset. DecayCoverage keeps the history of a delisted asset, and drops the asset from the result as soon as it becomes inactive. ResetCoverage deletes its history, so a relisted asset starts again from zero. ExpireCoverage keeps the asset in the result for after more observations. During an incremental fit, fold_inactive! applies the rule when an asset becomes inactive, and admits decides whether the asset appears in the result when you read the estimate. The observation count of each entry is stored in a PortfolioOptimisers.CoverageCounts, which the state of the incremental fit carries.

Two admitted assets can each have a variance and share too few observations for a covariance. Such a pair is undetermined. An AbstractPeel in the peel field of the policy removes a set of assets that leaves no undetermined pair, and the covariance warns and names them, or refuses under strict = true. MinimalPeel removes the fewest assets, GreedyPeel removes the asset with the most undetermined pairs first, and peel = nothing skips the peel, so the matrix repair refuses the NaN. A rule of your own implements peel_assets.

PortfolioOptimisers.CoveragePolicy — Type
struct CoveragePolicy{__T_min_coverage, __T_alg, __T_peel} <: AbstractEstimator

Fits each cell of a moment on the observations that cell has, instead of on the Coverage Universe.

A policy replaces the all-or-nothing Coverage Universe with available-case estimation. It goes in the cvg field of a moment estimator, where the default is nothing. With nothing the estimator takes the reduce-and-expand path by dispatch, and the policy costs nothing. With a policy set, the estimator fits every cell of the answer on the observations at which every asset of that cell is finite and active, and each cell carries its own denominator. An asset reaches the answer only where admits lets it in.

min_coverage is also the share that a scenario_fill passes in silence. admits reads the coverage share of an asset as its own observation count over the number of observations folded. The fill counts the non-finite entries of the column over the same denominator. So a column that DecayCoverage, ResetCoverage or ExpireCoverage admits is at most a 1 - min_coverage share non-finite. Where the fill_limit of a prior is nothing, resolve_fill_limit derives 1 - min_coverage, and no admitted column goes past it. A policy therefore turns on no warning, and an available-case walk-forward gives no warning for the gaps that it was set up to fill.

The default min_coverage = 0 admits every column, and a prior fills in silence every column that has a variance. Under the default Bessel correction that is a column with two or more observations. A column with one observation has a NaN variance, so the Investable Mask drops the asset and nothing is filled. To admit broadly and still get the warning, set the fill_limit of the prior explicitly, tighter than 1 - min_coverage.

A covariance peels the assets of an undetermined pair at admission. Two admitted assets can each have a variance and share too few observations for a covariance. The peel rule then removes a set of assets that leaves no such pair, and coverage_peel reports it through strict_diagnostic: a warning names the peeled assets, and strict = true refuses. A peeled asset is NaN across its row and column, so it leaves the Investable Mask, and the matrix repair meets no NaN inside the finite block. The default MinimalPeel keeps the most assets. GreedyPeel is the oracle's rule, and peel = nothing skips the peel and keeps the NaN for the matrix repair to refuse. A mean and a variance have no pairs, so the rule binds on a covariance and a correlation alone. On a fold, the peel runs each time the folded estimator answers a cov or a cor call, on the counts folded so far, and not in partial_fit!. So a walk-forward warns at each step that holds an undetermined pair, and the peeled assets can change from one step to the next.

Fields

  • min_coverage: min_coverage: Coverage floor, a share in [0, 1]. An asset whose own observation count over the number of observations folded falls below it is NaN in the answer. It is also the share that a scenario_fill fills in silence, so under the default of 0 a prior fills every column that has a variance and says nothing.
  • alg: alg: Coverage algorithm, the rule that decides what happens to a delisted asset.
  • peel: peel: Peel rule, which chooses the assets an available-case covariance removes so that no pair of admitted assets with a variance is left without a covariance, or nothing to skip the peel and leave the NaN of each undetermined pair for the matrix repair to refuse.

Constructors

CoveragePolicy(;    min_coverage::Real = 0.0,    alg::AbstractCoverageAlgorithm = DecayCoverage(),    peel::Option{<:AbstractPeel} = MinimalPeel()) -> CoveragePolicy

Keywords correspond to the struct's fields.

Validation

  • 0 <= min_coverage <= 1.

Examples

julia> CoveragePolicy(; min_coverage = 0.25)CoveragePolicy  min_coverage ┼ Float64: 0.25           alg ┼ DecayCoverage()          peel ┴ MinimalPeel()

Related

source
PortfolioOptimisers.AbstractCoverageAlgorithm — Type
abstract type AbstractCoverageAlgorithm <: AbstractAlgorithm

Abstract supertype of the rules that decide what an available-case fit does with a delisted asset.

All concrete subtypes should subtype AbstractCoverageAlgorithm. A member of the family goes in the alg field of a CoveragePolicy. The family exists because a delisting has more than one correct treatment. One caller keeps the history of the asset and drops the asset at once. Another caller throws the history away, so that a relisting starts cold. A third holds the asset in the frame for a stated number of observations after it goes.

Interfaces

In order to implement a new concrete type that works seamlessly with the library, subtype AbstractCoverageAlgorithm and implement the following methods:

fold_inactive!

  • fold_inactive!(alg::AbstractCoverageAlgorithm, state::AbstractPartialFitState, ni::AbstractVector{<:Bool}) -> AbstractPartialFitState: The fold-time hook, called once per observation with the assets that have just become inactive. Only the arms that fold call it. The two-pass arms, which are the semi-covariance, the coskewness and the cokurtosis, read the window whole and never call it, so a new subtype keeps the history in those arms.

Arguments

  • alg: The concrete subtype instance.
  • state: The partial-fit state being folded, mutated in place.
  • ni: One entry per asset, true where the asset was active at the previous observation and is inactive at this one.

Returns

  • state::AbstractPartialFitState: The state to fold the observation into.

admits

  • admits(alg::AbstractCoverageAlgorithm, share::Real, active::Bool, stale::Integer, min_coverage::Real) -> Bool: The predicate that the estimate from the state calls once per asset.

Arguments

  • alg: The concrete subtype instance.
  • share: The asset's coverage share, its own observation count over the number of observations folded.
  • active: Whether the asset was active at the last observation.
  • stale: The number of observations folded since the asset was last finite and active.
  • min_coverage: The coverage floor of the policy.

Returns

  • admitted::Bool: Whether the asset appears in the answer, rather than as NaN.

Examples

julia> struct KeepEverything <: PortfolioOptimisers.AbstractCoverageAlgorithm endjulia> PortfolioOptimisers.admits(::KeepEverything, args...) = true;julia> PortfolioOptimisers.admits(KeepEverything(), 0.0, false, 99, 1.0)true

Related

source
PortfolioOptimisers.DecayCoverage — Type
struct DecayCoverage <: AbstractCoverageAlgorithm

Keeps a delisted asset's history, and drops the asset from the frame the moment it goes inactive.

This is the default rule. It does nothing at fold time, and it does not read the staleness when the estimate is made from the state. A relisting resumes the history. An asset that lists, delists and lists again folds into the counts and accumulators that it left behind, and does not start from zero. ExpireCoverage keeps the history in the same way, and it differs only in how long a delisted asset stays in the frame.

Related

source
PortfolioOptimisers.ResetCoverage — Type
struct ResetCoverage <: AbstractCoverageAlgorithm

Throws a delisted asset's history away, so that a relisting starts the asset cold.

The reset zeroes the counts of the asset, and the row and the column of every accumulator that touches it. ExpWeightedCovariance resets an inactive asset in the same way. Use it when a relisted ticker is a different company, or when a corporate action makes the old series incomparable with the new one.

Every arm applies the reset. A folding arm applies it through fold_inactive! at the observation where the asset goes inactive, and a two-pass arm applies it to the whole window through coverage_inactive_block!. The reset needs an active mask. With no mask no asset goes inactive, a gap reads as a holiday, and the answer is the answer of DecayCoverage. The reset does not zero the number of observations folded, so the coverage share of a relisted asset is its count since the relisting over every observation folded.

Related

source
PortfolioOptimisers.ExpireCoverage — Type
struct ExpireCoverage{__T_after} <: AbstractCoverageAlgorithm

Holds a delisted asset in the frame for a stated number of observations, then drops it.

The count is the staleness of the asset, the number of observations folded since the asset was last finite and active. A holiday raises the staleness as a delisting does, so a holiday just before a delisting shortens the hold. A finite return at an active observation sets the staleness back to zero. after = 0 admits exactly what DecayCoverage admits. An observation at which an asset is inactive raises its staleness, so no fold leaves an inactive asset at a staleness of zero.

Fields

  • after: after: Number of observations an inactive asset stays in the frame for, counted from the last observation at which it was finite and active.

Constructors

ExpireCoverage(; after::Integer = 0) -> ExpireCoverage

Keywords correspond to the struct's fields.

Validation

  • after >= 0.

Examples

julia> ExpireCoverage(; after = 21)ExpireCoverage  after ┴ Int64: 21

Related

source
PortfolioOptimisers.AbstractPeel — Type
abstract type AbstractPeel <: AbstractAlgorithm

Abstract supertype of the rules that choose which assets an available-case covariance peels away.

An available-case covariance divides each pair by the observations that the pair shares. Two admitted assets can each have a variance and share too few observations for a covariance, so their cell is NaN inside the finite block. Such a pair is undetermined. The undetermined pairs make a graph, with one vertex per admitted asset that has a variance and one edge per undetermined pair. A set of assets removes every undetermined pair exactly when it is a vertex cover of this graph. A member of the family goes in the peel field of a CoveragePolicy and chooses the cover, and nothing skips the peel. The peel runs at admission, so a peeled asset is NaN across its row and column and leaves the Investable Mask.

Interfaces

In order to implement a new concrete type that works seamlessly with the library, subtype AbstractPeel and implement the following method:

peel_assets

  • peel_assets(peel::AbstractPeel, U::AbstractMatrix{Bool}) -> Vector{Int}: The vertices to remove from the graph of undetermined pairs.

Arguments

  • peel: The concrete subtype instance.
  • U: The adjacency matrix of the graph. It is symmetric, true at each undetermined pair, and false on the diagonal.

Returns

  • idx::Vector{Int}: The vertices to remove, sorted. A set that does not cover every edge leaves a NaN inside the block, and the matrix repair refuses it with an IsNonFiniteError.

Examples

julia> struct PeelEveryEnd <: PortfolioOptimisers.AbstractPeel endjulia> PortfolioOptimisers.peel_assets(::PeelEveryEnd, U) = findall(vec(any(U; dims = 2)));julia> PortfolioOptimisers.peel_assets(PeelEveryEnd(), Bool[0 1 0; 1 0 0; 0 0 0])2-element Vector{Int64}: 1 2

Related

source
PortfolioOptimisers.MinimalPeel — Type
struct MinimalPeel <: AbstractPeel

Peels the fewest assets that remove every undetermined pair, and keeps the greedy peel whenever it is one of them.

This is the default rule. It solves the minimum vertex cover of the graph of undetermined pairs exactly, so it keeps the most assets that a peel can keep. When the cover of GreedyPeel has the minimum size, the rule returns it, so it gives the greedy answer on every graph where the greedy answer is optimal. Otherwise it returns the minimum cover whose sorted index vector is lexicographically smallest.

Mathematical definition

\[\begin{align} \mathcal{P} &= \underset{\substack{\mathcal{C} \subseteq \mathcal{V} \\ \forall (i, j) \in \mathcal{E},\ i \in \mathcal{C} \lor j \in \mathcal{C}}}{\operatorname{arg\,min}} \left\lvert \mathcal{C} \right\rvert\,. \end{align}\]

Where:

  • $\mathcal{P}$: The assets the rule peels.
  • $\mathcal{V}$: Vertices of the graph of undetermined pairs, the admitted assets that have a variance.
  • $\mathcal{E}$: Edges of the graph of undetermined pairs, the pairs of vertices that share too few observations for a covariance.
  • $\mathcal{C}$: A candidate set of assets to remove.

Algorithm

  1. Take the greedy cover g with GreedyPeel.
  2. Find the minimum cover size k with peel_cover_within. Start at length(g) and lower the size while a smaller cover exists, so that only the size below the minimum needs a full search. Return g when no smaller cover exists.
  3. Otherwise build the lexicographically smallest cover of size k with peel_lexicographic.

The search is exact. Minimum vertex cover is NP-hard in general, but the search is exponential in the cover size alone, of order $1.47^{k}$ with $O(N^{2})$ work at each step. A tree, a path and a cycle need no branch at all. The graph holds only the pairs of admitted assets that share too few observations, so k is small in practice.

Related

source
PortfolioOptimisers.GreedyPeel — Type
struct GreedyPeel <: AbstractPeel

Peels assets greedily, the asset with the most undetermined pairs first.

It is the oracle's rule. It removes the asset with the most undetermined pairs to the assets that are still in the graph, the smallest index on a tie, and repeats until no undetermined pair is left. The greedy cover can be larger than the minimum. On a star whose three leaves each carry one more pendant edge it removes four assets, where three suffice. MinimalPeel returns the minimum.

Algorithm

  1. Find the asset of the largest degree among the assets still in the graph, the smallest index on a tie, with peel_max_degree.
  2. Stop when that degree is zero. Otherwise remove the asset and go to step 1.

Related

source
PortfolioOptimisers.peel_assets — Function
peel_assets(peel, U)

Returns the assets to remove from a graph of undetermined pairs.

It is the verb of the AbstractPeel interface. coverage_peel calls it once per available-case covariance, on the graph of the admitted assets that have a variance.

Arguments

  • peel: The peel rule, or nothing to skip the peel.
  • U: The adjacency matrix of the graph. It is symmetric, true at each undetermined pair, and false on the diagonal.

Returns

  • idx::Vector{Int}: The vertices to remove, sorted.

Examples

julia> U = falses(7, 7);julia> for (i, j) in ((1, 2), (1, 3), (1, 4), (2, 5), (3, 6), (4, 7))           U[i, j] = U[j, i] = true       endjulia> PortfolioOptimisers.peel_assets(GreedyPeel(), U)4-element Vector{Int64}: 1 2 3 4julia> PortfolioOptimisers.peel_assets(MinimalPeel(), U)3-element Vector{Int64}: 2 3 4julia> PortfolioOptimisers.peel_assets(nothing, U)Int64[]

Related

source
PortfolioOptimisers.peel_assets — Method
peel_assets(
    _::Nothing,
    _::AbstractMatrix{Bool}
) -> Vector{Int64}

nothing method of peel_assets. It skips the peel and removes no vertex.

An undetermined pair then stays NaN in the covariance, and the matrix repair refuses the NaN inside the finite block with an IsNonFiniteError that names the cell. Skip the peel to see the undetermined pairs, or to keep a covariance whose gaps a later step fills.

Related

source
PortfolioOptimisers.fold_inactive! — Function
fold_inactive!(alg, state, ni)

Applies a coverage algorithm's fold-time rule to the assets that have just become inactive.

It is the first of the two verbs of the AbstractCoverageAlgorithm interface. It runs once per observation, before the fold of that observation, so a rule that throws the history away runs while the state still holds that history. Only the folding arms call it. A two-pass arm applies the rule of a built-in algorithm to the whole window through coverage_inactive_block!.

Arguments

  • alg: The coverage algorithm.
  • state: The partial-fit state, mutated in place.
  • ni: One entry per asset, true where the asset was active at the previous observation and is inactive at this one.

Returns

  • state: The state to fold the observation into.

Related

source
PortfolioOptimisers.admits — Function
admits(alg, share, active, stale, min_coverage) -> Bool

Decides whether an asset reaches the answer of an available-case fit.

It is the second of the two verbs of the AbstractCoverageAlgorithm interface. It runs once per asset when the estimate is made from the state, and the estimate holds NaN across every asset that admits refuses.

Arguments

  • alg: The coverage algorithm.
  • share: The asset's coverage share, its own observation count over the number of observations folded.
  • active: Whether the asset was active at the last observation.
  • stale: The number of observations folded since the asset was last finite and active.
  • min_coverage: The coverage floor of the CoveragePolicy.

Returns

  • admitted::Bool: Whether the asset reaches the answer.

Related

source
PortfolioOptimisers.admits — Method
admits(
    _::Union{DecayCoverage, ResetCoverage},
    share::Real,
    active::Bool,
    _::Integer,
    min_coverage::Real
) -> Any

DecayCoverage and ResetCoverage method of admits. An asset is admitted while it is active and its coverage share reaches the floor. So both algorithms drop a delisted asset at once, and they differ only in the history they keep.

Mathematical definition

\[\begin{align} i \in \mathcal{A} &\iff a_{i} = 1 \land s_{i} \geq c\,. \end{align}\]

Where:

  • $\mathcal{A}$: Admitted assets, those that admits lets into the answer. A cell is admitted when every asset of the cell is in $\mathcal{A}$.
  • $a_{i}$: Active flag of asset $i$ at the last observation folded, $1$ when the asset is active and $0$ when it is not.
  • $s_{i}$: Coverage share of asset $i$, its own observation count over the number of observations folded.
  • $c$: Coverage floor of the policy, its min_coverage.

Related

source
PortfolioOptimisers.admits — Method
admits(
    alg::ExpireCoverage,
    share::Real,
    active::Bool,
    stale::Integer,
    min_coverage::Real
) -> Any

ExpireCoverage method of admits. An inactive asset stays in the frame while its staleness is at most after, and its coverage share must reach the floor in both cases.

Mathematical definition

\[\begin{align} i \in \mathcal{A} &\iff s_{i} \geq c \land \left(a_{i} = 1 \lor \tau_{i} \leq h\right)\,. \end{align}\]

Where:

  • $\mathcal{A}$: Admitted assets, those that admits lets into the answer. A cell is admitted when every asset of the cell is in $\mathcal{A}$.
  • $s_{i}$: Coverage share of asset $i$, its own observation count over the number of observations folded.
  • $c$: Coverage floor of the policy, its min_coverage.
  • $a_{i}$: Active flag of asset $i$ at the last observation folded, $1$ when the asset is active and $0$ when it is not.
  • $\tau_{i}$: Staleness of asset $i$, the number of observations folded since the asset was last finite and active.
  • $h$: The after field of the algorithm.

Related

source