Base moments: private API

Abstract moment types and fallbacks

The abstract types below are the roots of the moment estimators. The functions are the steps that many moment estimators share, such as the refusal of a variance estimator in a covariance slot, the choice of the observation window and its weights, and the centring of the returns.

PortfolioOptimisers.AbstractExpectedReturnsAlgorithm — Type
abstract type AbstractExpectedReturnsAlgorithm <: AbstractAlgorithm

Abstract supertype for all expected returns algorithm types.

Every concrete or abstract type that is an algorithm of an expected returns estimator must subtype AbstractExpectedReturnsAlgorithm.

Interfaces

A subtype needs no method of its own. An expected returns estimator holds the algorithm in a field and dispatches on its type. Define the methods on the expected returns estimator that uses the algorithm.

Related

source
PortfolioOptimisers.AbstractMomentAlgorithm — Type
abstract type AbstractMomentAlgorithm <: AbstractAlgorithm

Abstract supertype for the forms of the deviations of a moment from its target.

The family holds two forms. FullMoment keeps every deviation, and SemiMoment keeps the deviations below the target. A moment estimator or a moment risk measure bounds its alg field to this type, and it implements each form. So the bound refuses in the constructor an algorithm that the verbs do not implement.

An algorithm that selects something other than a form of the deviations is not a member. The Gerber, Smyth-Broby and Gerber IQ algorithms and the shrinkage targets are orthogonal to the two forms, so they subtype AbstractAlgorithm directly.

Interfaces

A form holds no data, and no verb takes it alone. A moment estimator holds the form in its alg field, and each verb of the estimator dispatches on the type of that field. So a new form needs a method of each verb of every type that bounds alg to AbstractMomentAlgorithm, and a method of coverage_comoment_deviations.

Related

source
PortfolioOptimisers.assert_covariance_estimator — Function
assert_covariance_estimator(ce, name::Symbol) -> nothing
assert_covariance_estimator(ce::AbstractVarianceEstimator, name::Symbol)

Refuses a variance estimator in a slot that calls Statistics.cov or Statistics.cor.

AbstractVarianceEstimator subtypes AbstractCovarianceEstimator, so a field bound to StatsBase.CovarianceEstimator accepts it. But a variance estimator computes one marginal variance for each asset and no relation between two assets, and its cov and cor throw a MethodError. The configuration alone decides that every call of the slot fails. So the inner constructor of each such slot calls this function, and the refusal comes at the construction.

A slot that calls only Statistics.var or Statistics.std does not call this function, because a variance estimator is the estimator that it reads.

Algorithm

The method that Julia selects is the algorithm.

  1. Every value that is not an AbstractVarianceEstimator passes. A covariance estimator, a matrix and nothing are examples.
  2. An AbstractVarianceEstimator raises an ArgumentError.

Arguments

  • ce: The value of the slot.
  • name: The name of the slot, for the error message.

Validation

  • ce is not an AbstractVarianceEstimator. Otherwise, the function raises an ArgumentError that names the slot and the type of ce.

Returns

  • nothing.

Related

source
PortfolioOptimisers.gap_fill_value — Method
gap_fill_value(ce::StatsBase.CovarianceEstimator) -> Number

Return the value that replaces a gapped cell of a sample before the sample goes to ce.

A gapped sample has a non-finite cell because the asset was absent at that observation, and not because a computation failed. A consumer that holds such a sample and a covariance estimator cannot know which value the estimator needs. The consumer calls this function, and the value depends on the estimator alone.

When the value is finite, the consumer writes it over every gapped cell before it gives the sample to the estimator. The estimator then gets a complete sample and no mask. When the value is not finite, the consumer keeps the gaps. It gives the active mask that explains them to Statistics.cov(ce, X; dims, active_mask), and an estimator that returns a non-finite value must implement that method.

The fallback value is zero. A plain moment estimator refuses a gapped sample, and zero is the neutral value of a standardised series. An estimator that accepts gaps returns NaN. That one method is all that such an estimator adds to this function.

A covariance estimator can hold another covariance estimator, so the value is recursive:

  • An estimator that gives the sample and its keywords to its inner estimator with no change returns the value of its inner estimator. PortfolioOptimisersCovariance and CorrelationCovariance are two of them. So a wrapper around an estimator that accepts gaps keeps the gaps.
  • An estimator that reads the sample before it calls its inner estimator keeps the fallback. Covariance is one of them, because its expected returns estimator reads the sample. It refuses the gap itself, whatever its inner estimator returns.
  • A Covariance with a CoveragePolicy reads the gaps itself, over the observations of each pair, so it returns NaN. A wrapper around it keeps the gaps.

Arguments

  • ce: Covariance estimator.

Returns

  • fv::Number: The value of a gapped cell. It is zero for a plain estimator and NaN for an estimator that accepts gaps.

Related

source
PortfolioOptimisers.moment_window_and_weights — Function
moment_window_and_weights(
    X::VecNum_MatNum,
    w::Option{<:ObsWeights},
    args...;
    dims::Int = 1,
    kwargs...
) -> (VecNum_MatNum, Option{<:StatsBase.AbstractWeights})
moment_window_and_weights(
    X::VecNum_MatNum,
    w::Option{<:ObsWeights},
    window::VecInt;
    dims::Int = 1,
    kwargs...
) -> (VecNum_MatNum, Option{<:StatsBase.AbstractWeights})

Apply the observation window to the data, and resolve the observation weights over the windowed data.

The function takes the view of X over the observations that window indexes, and resolves the observation weights over that view. The caller resolves window with get_window before the call, so an Int window is already a range here.

Algorithm

  1. Without a window, X passes through unchanged, and step 3 resolves the weights over the whole of it.
  2. With a window, take the view of X over those observations. For a matrix that is view(X, window, :) when dims == 1, and view(X, :, window) when dims == 2. For a vector it is view(X, window). Index w to the same observations with nothing_scalar_array_getindex.
  3. Resolve the observation weights with get_observation_weights, over w and the X of the step above.
  4. Return that X and the resolved w.

Arguments

  • X: Data matrix or vector.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • Either:
    • args: More positional arguments, which the method ignores.
    • window: Observation window. An integer selects the last window observations, and a vector of indices selects those observations.
  • dims: Dimension along which the computation runs. The function ignores it when X is a vector.
  • kwargs: More keyword arguments, which the method ignores.

Returns

  • X::VecNum_MatNum: The data over the window, or all of the data when there is no window.
  • w::Option{<:StatsBase.AbstractWeights}: The resolved observation weights over the same observations.

Related

source
PortfolioOptimisers.weighted_centre — Function
weighted_centre(X::MatNum, me::AbstractExpectedReturnsEstimator,
                w::Option{<:ObsWeights}; dims::Int = 1, mean = nothing,
                kwargs...) -> Union{Number, ArrNum}

Resolve the centre of a moment estimator with the same observation weights as its deviations.

The observation weights of a moment estimator apply to its centre, and not only to its deviations. This function holds that rule, so SimpleVariance, Covariance, Coskewness and Cokurtosis compute their centre in the same way.

Algorithm

  1. When mean is not nothing, return it unchanged. Use the keyword for a centre that w does not describe.
  2. When w is nothing, return Statistics.mean(me, X; dims = dims, kwargs...).
  3. Otherwise, rebuild me with w through factory, so that the centre has the weights of the deviations. Return the mean of the rebuilt estimator.

Step 2 makes the function faster and does not change the value. w comes from a field, so its type selects the branch. Without step 2, a windowed loop rebuilds the estimator tree of me once for each window.

Arguments

  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • me: Expected returns estimator.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for the expected returns estimator.

Returns

  • mu::Union{Number, ArrNum}: The centre, weighted by w when w is not nothing.

Related

source
PortfolioOptimisers.demean_returns — Function
demean_returns(X::MatNum, me::AbstractExpectedReturnsEstimator; dims::Int = 1, mean = nothing,
               kwargs...) -> MatNum

Subtracts the mean of each asset from the returns in X.

The expected returns estimator me computes the mean, unless the caller gives mean.

Algorithm

  1. Resolve the centre mu with weighted_centre. When mean is nothing, the estimator computes mu. Otherwise mu is mean. The estimator uses its own weights, and this function adds no weights.
  2. Subtract mu from X by broadcast, and return the result. dims is the axis of the observations, and the estimator shapes mu along the other axis. The broadcast then subtracts one value for each asset from every observation of that asset.

Arguments

  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • me: Expected returns estimator.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for the expected returns estimator.

Returns

  • X::MatNum: The demeaned returns matrix.

Related

source
PortfolioOptimisers.cov2cor_nan! — Function
cov2cor_nan!(sigma::MatNum, s::VecNum = sqrt.(LinearAlgebra.diag(sigma))) -> MatNum

Rescale a covariance matrix to its correlation matrix in place, with a NaN diagonal entry for each asset whose correlation is not defined.

The correlation of asset $i$ with itself is $\sigma_{ii} / (s_i s_i)$. A zero standard deviation, the one of a constant return, makes it $0 / 0$. A standard deviation that is not finite, the one of an asset outside the Coverage Universe, makes it NaN too. StatsBase.cov2cor! writes one on the whole diagonal, so it gives such an asset a correlation of one with itself and NaN with every other asset. This function writes the NaN back on the diagonal. The row of the asset is then NaN at every entry, as the row of an asset that a distance reader refuses is, see DistanceReaderRefusalError. A matrix holds no data, so the function cannot find a constant whose variance rounding left above zero. The cor of an estimator that reads the data marks such an asset, see constant_column_correlation!.

Algorithm

  1. Rescale sigma with StatsBase.cov2cor! and s.
  2. Write NaN on the diagonal wherever s[i] is zero or not finite, with undefined_correlation_diagonal!.

Arguments

  • sigma: Covariance-like or correlation-like matrix assets × assets.
  • s: The standard deviations, the square roots of the diagonal of sigma by default.

Returns

  • rho::MatNum: The input matrix, rescaled in place to a correlation matrix.

Related

source
PortfolioOptimisers.undefined_correlation_diagonal! — Function
undefined_correlation_diagonal!(rho::AbstractMatrix, s::VecNum) -> AbstractMatrix

Write NaN on the diagonal entry of each asset whose correlation with itself is not defined, in a correlation matrix whose diagonal a rescale set to one.

The correlation of asset $i$ with itself is $\sigma_{ii} / (s_i s_i)$. It is $0 / 0$ when the scale s[i] is zero, and NaN when s[i] is not finite. A standard deviation and a variance are zero together and finite together, so s can hold either of them. The rescale of StatsBase writes one on such an entry, and the entries of the row off the diagonal are already NaN.

Algorithm

  1. For each i whose s[i] is zero or not finite, write s[i] / s[i] on rho[i, i]. The quotient is NaN in the number type of s.
  2. Return rho.

Arguments

  • rho: The correlation matrix, assets × assets. It must be mutable.
  • s: The standard deviations or the variances of the assets.

Returns

  • rho::AbstractMatrix: The input matrix, with a NaN diagonal entry for each undefined correlation.

Related

source
PortfolioOptimisers.constant_column_correlation! — Function
constant_column_correlation!(rho::MatNum, X::MatNum, w::Option{<:VecNum} = nothing) -> MatNum

Write NaN on the row and the column of each constant column of X, after an estimator computed the correlation matrix rho of X.

The correlation of a constant return is 0 / 0 with every asset, itself included. The library gives such an asset a NaN row and column, its diagonal entry included, so a distance reader refuses it, see DistanceReaderRefusalError. An estimator does not always give that row:

  • StatsBase.corkendall and StatsBase.corspearman give NaN at every other column and one on the diagonal.
  • The Pearson correlation of StatsBase subtracts a mean that can differ from the constant by one unit in the last place. The variance of the column is then about 1e-38, not zero, and its correlations are rounding noise, not NaN. Cash at a constant return of 0.0005 gives such a column.

So the test reads the data, not rho or a variance. With observation weights, it reads the observations of positive weight alone, because the weighted moments do not read the others.

Algorithm

  1. Take the rows of X whose weight in w is positive, or every row when w is nothing.
  2. For each column i of X whose entries on those rows are all equal, write zero(T) / zero(T) on row i and column i of rho, where T is the element type of rho.
  3. Return rho.

Arguments

  • rho: The correlation matrix, assets × assets. It must be mutable.
  • X: The data matrix, observations × assets.
  • w: The observation weights, or nothing.

Returns

  • rho::MatNum: The input matrix, with a NaN row and column for each constant column.

Related

source
PortfolioOptimisers.mark_constant_correlation! — Function
mark_constant_correlation!(est::StatsBase.SimpleCovariance, ce::AbstractCovarianceEstimator,
                           rho::AbstractMatrix, X::MatNum,
                           w::Option{<:StatsBase.AbstractWeights}; dims::Int = 1,
                           kwargs...) -> AbstractMatrix
mark_constant_correlation!(est::StatsBase.CovarianceEstimator, ce::AbstractCovarianceEstimator,
                           rho::AbstractMatrix, X::MatNum,
                           w::Option{<:StatsBase.AbstractWeights}; dims::Int = 1,
                           mean = nothing, kwargs...) -> AbstractMatrix

Give each constant return the NaN row and column of an undefined correlation, in the correlation matrix rho that the inner estimator est of ce computed from X.

The Pearson correlation of a constant return is 0 / 0 with every asset, itself included, so the library gives the asset a NaN row and column, see cov2cor_nan!. The method depends on est:

  • StatsBase.SimpleCovariance, the Pearson correlation. The mean of a constant can differ from the constant by one unit in the last place. The variance is then about 1e-38, not zero, and the row holds rounding noise, not NaN. So the method tests the data with constant_column_correlation!, on the observations of positive weight.
  • Any other estimator. An estimator such as a shrinkage defines the dependence of a constant, and keeps its values. Where its matrix holds a NaN, the method reads the variances v from the diagonal of the covariance of the same fit. It writes NaN on the diagonal entry of each zero variance with undefined_correlation_diagonal!, because StatsBase computes a correlation as cov2cor!(cov(...)), which writes one on its diagonal. A matrix with no NaN needs no second fit.

Arguments

  • est: The inner estimator ce.ce. The method dispatches on it.
  • ce: Covariance estimator.
  • rho: The correlation matrix, assets × assets. It must be mutable where the method writes.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • w: The observation weights, or nothing.
  • dims: Dimension along which the computation runs.
  • mean: Optional precomputed centre of the data.
  • kwargs...: More keyword arguments for the covariance of the second method.

Returns

  • rho::AbstractMatrix: The input matrix, with the NaN entries of each constant return.

Related

source

Windowed estimator generation

The five windowed estimators have one shape: WindowedExpectedReturns, WindowedVariance, WindowedCovariance, WindowedCoskewness and WindowedCokurtosis. Each wraps an inner estimator, keeps only the last observations of the sample, and passes every moment call to the inner estimator. The macro @windowed_estimator writes each of them from one declaration. It writes the struct, its constructor, its factory and port_opt_view methods, the methods that pass each moment call on, and their docstrings.

The entries below are the macro and the functions it calls when it expands. You use the five estimators, and never call these directly.

PortfolioOptimisers.@windowed_estimator — Macro
@windowed_estimator Name <: Super begin
    field::FieldType = Default()
    noun    = "Noun"
    forward = [generic(::MatNum; mean) => :ret_key, ...]
    doctest = """..."""
end

Declare a windowed moment estimator, which applies an inner moment estimator to a window of observations.

The windowed estimator binds the observation weights to the window. The inner estimator computes the moment in the same way as without a window.

One call of the macro declares all the parts of the windowed estimator:

  • The @propagatable @concrete struct. Its fields are the inner estimator tagged @fprop @vprop, w tagged @wprop, window, rule, and a cache tagged @fprop @vprop that holds a partial-fit state.
  • The two constructors with their validation.
  • One forwarding method for each forward entry, and one Asset Panel method for each matrix entry.
  • The export of the type, and every docstring.

The library has five nominal types and not one parametric Windowed{E}. Each type implements a different generic, and must subtype a different abstract estimator, such as AbstractCovarianceEstimator or CoskewnessEstimator. Dispatch across the library reads these supertypes, and a Julia struct cannot take its supertype from a type parameter. This macro keeps the five types the same in every other part.

Body

  • field::FieldType = Default(): The inner estimator. The field name is also its field_dict key and the argument name of every generated method. So it must follow the naming convention of the library, which uses me, ce, ve, ske and kte.
  • noun: The capitalised noun phrase of the moment, for example "Expected returns". All the generated prose uses it.
  • forward: One mini-signature for each generic to forward, with the ret_dict keys of its return values. A mean in the mini-signature makes mean a named keyword of the generated method. Without it, mean goes into kwargs... and from there to windowed_preamble.
  • doctest: The body of the jldoctest block of the # Examples section, without its fences.

At macro-expansion time, the macro refuses an unknown key, a wrong forward entry, and an unknown key of field_dict or ret_dict. The error for an unknown key ends with a did_you_mean suggestion.

Algorithm

  1. Read name and super from the header.
  2. Read the body once. windowed_parse_field parses the one field::Type = default line, and returns field, ftype and default. The noun, forward and doctest lines bind their values. Any other key raises an error.
  3. Parse every entry of forward with windowed_parse_forward, giving specs. Append one :AssetPanel copy of each MatNum entry, so every matrix generic also gets its Asset Panel method.
  4. Render one cross-reference per entry of specs with windowed_method_ref, giving refs.
  5. For each entry of specs, build one documented forwarding method: windowed_method_doc writes its docstring, and windowed_method_def writes its body. Each method's # Related section lists the refs of its siblings and not its own.
  6. Build structexpr, the @concrete struct. It declares the inner estimator tagged @fprop @vprop, w tagged @wprop, window and rule with no tag, and cache tagged @fprop @vprop, each with its live field_dict lookup, and the inner constructor that validates w and window. When an entry of specs forwards cov or cor, the inner constructor also refuses a variance estimator as the inner estimator, with the check that windowed_relation_check builds.
  7. Build kwctor, the keyword constructor, whose defaults are default, nothing, nothing, RollingWindow() and nothing.
  8. Write the type's docstring with windowed_type_doc, and attach it to structexpr wrapped in @propagatable @concrete.
  9. Return the escaped block: the documented struct, kwctor, the forwarding methods, and the export of name.

Validation

  • The header reads Name <: Super, and Name is a Symbol.
  • The declaration body is a begin ... end block, and every line of it is an assignment.
  • At most one field::Type = default line appears.
  • Every other key names an entry of WINDOWED_ESTIMATOR_KEYS. An unknown key raises with a suggest_declared_key suffix.
  • All four of field::Type = default, noun, forward and doctest are present.
  • noun and doctest are string literals.
  • forward is a vector, and it declares at least one generic.
  • Every failure above raises an ArgumentError through windowed_estimator_error, at macro-expansion time.

Returns

  • ex::Expr: The escaped block that declares the whole family member.

Examples

@windowed_estimator WindowedVariance <: AbstractVarianceEstimator begin    ve::AbstractVarianceEstimator = SimpleVariance()    noun = "Variance"    forward = [Statistics.var(::MatNum; mean) => :vararr,               Statistics.var(::VecNum; mean) => :varnum]    doctest = """    julia> WindowedVariance()    ...    """end

Related

source
PortfolioOptimisers.windowed_parse_field — Function
windowed_parse_field(ex) -> Tuple{Symbol, Any, Any}

Parse the field::Type = default line of a @windowed_estimator body into the field name of the inner estimator, its declared type, and its default in the keyword constructor.

The field name is also the field_dict key of the generated field docstring, and the argument name of every generated forwarding method. So it must follow the naming convention of the library, which uses me, ce, ve, ske and kte.

Algorithm

  1. Read the field name and its declared type from the left of the =, giving name and type.
  2. Check name against field_dict with windowed_estimator_check_key.
  3. Return name, type, and the right of the =, which is the keyword-constructor default.

Arguments

  • ex: The one field::Type = default line of the declaration body.

Validation

  • ex reads field::Type = default. Any other shape raises.
  • The field name is a Symbol. Any other name raises.
  • The field name names an entry of field_dict.

Returns

  • (name, type, default): The field name, its declared type, and its keyword-constructor default.

Related

source
PortfolioOptimisers.windowed_parse_forward — Function
windowed_parse_forward(
    ex
) -> Tuple{Any, Any, Bool, Vector{Symbol}}

Parse one forward entry of a @windowed_estimator body, such as generic(::MatNum; mean) => :ret_key. The parts are the forwarded generic, its input type, a flag for a mean keyword, and the ret_dict keys of its return values.

A mean in the mini-signature keeps mean out of the forwarded kwargs.... Without it, mean goes into kwargs... and from there to windowed_preamble.

Algorithm

  1. Split ex at the =>, giving the mini-signature sig and the return keys rets.
  2. Read the forwarded generic from the head of sig, giving gen.
  3. Read each argument of sig. A mean keyword sets has_mean. The one positional argument type sets input.
  4. Check input against WINDOWED_ESTIMATOR_INPUTS.
  5. Check every entry of rets against ret_dict with windowed_estimator_check_key, and collect the entries into keys_.
  6. Return gen, input, has_mean and keys_.

Arguments

  • ex: One entry of the forward vector of the declaration body.

Validation

  • ex reads generic(::Input[; mean]) => :ret_key, or => (:k1, :k2) for a tuple return. Any other shape raises.
  • The left of the => is a call.
  • mean is the only keyword that the mini-signature can name.
  • The mini-signature declares exactly one positional argument type.
  • That type names an entry of WINDOWED_ESTIMATOR_INPUTS.
  • Every return key is a quoted symbol, and names an entry of ret_dict.

Returns

  • (gen, input, has_mean, keys_): The forwarded generic, its input type, whether the mini-signature names mean, and the ret_dict keys of its return values.

Related

source
PortfolioOptimisers.windowed_relation_check — Function
windowed_relation_check(
    field::Symbol,
    specs
) -> Union{Nothing, Expr}

Build the check that the inner constructor of a generated windowed estimator runs on its inner estimator.

A windowed estimator that forwards cov or cor calls them on its inner estimator, and a variance estimator has neither. So its inner constructor refuses a variance estimator with assert_covariance_estimator. A windowed estimator that forwards only mean, var, std or a higher moment needs no such check. WindowedVariance holds a variance estimator on purpose.

Algorithm

  1. Read the last name of the generic of each entry of specs. For Statistics.cov, it is cov. For a bare name, it is the name itself.
  2. When one of these names is cov or cor, return the call assert_covariance_estimator(field, :field).
  3. Otherwise, return nothing, which adds no statement to the constructor.

Arguments

  • field::Symbol: Name of the inner estimator's field.
  • specs: The parsed forward entries, as windowed_parse_forward returns them.

Returns

  • check::Option{Expr}: The call to put in the inner constructor, or nothing.

Related

source
PortfolioOptimisers.windowed_estimator_check_key — Function
windowed_estimator_check_key(
    key::Symbol,
    dict::AbstractDict,
    what::AbstractString
) -> Symbol

Check that key names an entry of dict. When it does not, raise an error that ends with a did_you_mean suggestion. The message names the table what.

Arguments

  • key::Symbol: The key the declaration wrote.
  • dict::AbstractDict: The table the key must name an entry of.
  • what::AbstractString: The name of that table. The message contains it.

Validation

Returns

  • key::Symbol: The key, unchanged, so a caller can check the key and bind it in one expression.

Related

source
PortfolioOptimisers.windowed_type_doc — Function
windowed_type_doc(
    name::Symbol,
    super,
    field::Symbol,
    ftype,
    default,
    noun::AbstractString,
    doctest::AbstractString,
    methods::Vector{String},
    refuses_variance::Bool
) -> Expr

Build the type docstring of a generated windowed estimator as an interpolation AST.

The DocStringExtensions abbreviations and the dictionary lookups stay live, as in windowed_method_doc.

Algorithm

  1. Read the name of the inner estimator from default into inner_ref. For a call such as SimpleVariance(), the name is the head of the call. For a bare name, it is the name itself.
  2. Start parts with the live DocStringExtensions.TYPEDEF abbreviation, the two summary sentences, the # Fields heading, and the live DocStringExtensions.FIELDS abbreviation.
  3. Push the # Constructors section and the keyword signature built from name, field, ftype and default. Then push the ## Validation subsection, with the live val_dict[:oow] lookup and the window rule. When refuses_variance is true, the live val_dict[:ce_cov] lookup comes first.
  4. Push the three propagation subsections, ## Propagated parameters, ## View parameters and ## Observation weight parameters. Each one names field, w and cache as the tags of the generated struct declare them.
  5. Push the # Examples section, fencing doctest as a jldoctest block.
  6. Push the # Related heading, the supertype, inner_ref, every entry of methods, the two window rules, and the four shared functions that the type implements: factory, port_opt_view, obs_weights_view and windowed_preamble.
  7. Return parts wrapped in Expr(:string, ...), so every lookup stays live.

Arguments

  • name::Symbol: Name of the windowed estimator type.
  • super: Its supertype.
  • field::Symbol: Name of the inner estimator's field.
  • ftype: Declared type of that field.
  • default: Its keyword-constructor default.
  • noun::AbstractString: The capitalised noun phrase of the moment. The generated prose uses it.
  • doctest::AbstractString: The body of the jldoctest block of the # Examples section, without its fences.
  • methods::Vector{String}: Cross-references to the generated methods of the type, from windowed_method_ref.
  • refuses_variance::Bool: true when the type forwards cov or cor, so its inner constructor refuses a variance estimator through assert_covariance_estimator.

Returns

  • doc::Expr: An Expr(:string, ...) holding the docstring, with every abbreviation and lookup left unevaluated.

Related

source
PortfolioOptimisers.windowed_method_ref — Function
windowed_method_ref(
    gen,
    field::Symbol,
    name::Symbol,
    input::Symbol
) -> String

Render the generic(field::Name, X::Input) reference to one generated forwarding method. The # Related sections of the type and of the other generated methods use it.

The reference leaves out the keyword arguments. Documenter resolves an @ref by the positional signature of the method, and the positional types are enough to find one method. The panel method has a third positional argument, pnl::Option{<:AssetPanel}, after X::MatNum.

Arguments

  • gen: The forwarded generic.
  • field::Symbol: Name of the inner estimator's field, which is also the argument name of the generated method.
  • name::Symbol: Name of the windowed estimator type.
  • input::Symbol: Input kind of the generated method, :MatNum, :VecNum or :AssetPanel.

Returns

  • ref::String: A Documenter cross-reference to the generated method. It links the code span gen(field::Name, X::Input) to that method.

Related

source
PortfolioOptimisers.windowed_method_sig — Function
windowed_method_sig(
    gen,
    field::Symbol,
    name::Symbol,
    input::Symbol,
    has_mean::Bool
) -> String

Render the signature line of the docstring of one generated forwarding method.

Algorithm

  1. For the vector input, return gen(field::Name, X::VecNum), with ; mean = nothing when has_mean is set.
  2. Otherwise open gen(field::Name, X::MatNum, add pnl::Option{<:AssetPanel} for the panel input, and add the keywords: dims, then mean when has_mean is set, then iv for the matrix input alone, then kwargs.... The panel method takes iv in kwargs....

Arguments

  • gen: The forwarded generic.
  • field::Symbol: Name of the inner estimator's field, which is also the argument name of the generated method.
  • name::Symbol: Name of the windowed estimator type.
  • input::Symbol: Input kind of the generated method, :MatNum, :VecNum or :AssetPanel.
  • has_mean::Bool: true when the method declares a mean keyword.

Returns

  • sig::String: The indented signature line.

Related

source
PortfolioOptimisers.windowed_method_doc — Function
windowed_method_doc(
    gen,
    field::Symbol,
    name::Symbol,
    input::Symbol,
    has_mean::Bool,
    ret_keys::Vector{Symbol},
    noun::AbstractString,
    siblings::Vector{String}
) -> Expr

Build the docstring for one generated forwarding method as an interpolation AST.

The function returns an Expr(:string, ...) and not a String, so each lookup of arg_dict and ret_dict stays a live part of the DocStr, as a hand-written $(arg_dict[:dims]) does.

Algorithm

  1. Build the signature line sig with windowed_method_sig. The matrix signature has dims, iv and kwargs..., and the vector signature has none of them. The panel signature has pnl after X, and takes iv in kwargs.... Each signature also has mean when has_mean is set.
  2. Start parts with sig, the summary sentences, the steps of the algorithm of the generated method, and the arguments heading with its two prose bullets. The panel method has one more step, the cut of the panel. The summary names the generic and not the noun of the type, because std on a WindowedVariance computes a standard deviation and not a variance.
  3. For the panel input, push the arg_dict[:pnl_moment] lookup as a live expression.
  4. For the matrix and panel inputs, push the arg_dict[:dims] lookup.
  5. When has_mean is set, push the mean bullet.
  6. For the matrix and panel inputs, push the arg_dict[:oiv] lookup and the kwargs... bullet.
  7. Push the returns heading, then one live ret_dict lookup per entry of ret_keys.
  8. Push the related heading, the type, every entry of siblings, and windowed_preamble.
  9. Return parts wrapped in Expr(:string, ...), so every lookup stays live.

Arguments

  • gen: The forwarded generic.
  • field::Symbol: Name of the inner estimator's field, which is also the argument name of the generated method.
  • name::Symbol: Name of the windowed estimator type.
  • input::Symbol: Input kind of the generated method, :MatNum, :VecNum or :AssetPanel.
  • has_mean::Bool: true when the method declares a mean keyword.
  • ret_keys::Vector{Symbol}: The ret_dict keys documenting the return values.
  • noun::AbstractString: The capitalised noun phrase of the moment. The generated prose uses it.
  • siblings::Vector{String}: Cross-references to the other generated methods of the type, from windowed_method_ref.

Returns

  • doc::Expr: An Expr(:string, ...) holding the docstring, with every dictionary lookup left unevaluated.

Related

source
PortfolioOptimisers.windowed_method_def — Function
windowed_method_def(
    gen,
    field::Symbol,
    name::Symbol,
    input::Symbol,
    has_mean::Bool
) -> Expr

Build one generated forwarding method. The method resolves the window with windowed_preamble, and then calls the method of the inner estimator.

Algorithm

  1. Build the three field accesses that the body reads. field.field is the inner estimator, field.w holds the observation weights, and field.window holds the window.
  2. For the vector input, build a method whose only keyword is mean, and only when has_mean is set. Its body binds inner and the windowed X with windowed_preamble, and returns gen of the two. Return this method.
  3. Build the keyword list of the signature. It holds dims, then mean when has_mean is set, then iv and kwargs.... The panel method names no iv. As in the panel methods of the moment verbs, iv comes in kwargs..., and the preamble binds it from there.
  4. Build the matching keyword list of the inner call. It has the same names, and passes each value on. It passes the windowed iv, and the keywords kw, in which the two universe masks are cut to the window.
  5. For the panel input, the positional arguments are field, X and pnl, and the body opens with the cut of pnl to the window by windowed_panel. For the matrix input, they are field and X.
  6. Build the body: one call to windowed_preamble that binds inner, the windowed X, iv and kw, then a return of gen applied to inner, that X, and the cut pnl for the panel input.
  7. Return the whole Expr(:function, ...).

Arguments

  • gen: The forwarded generic.
  • field::Symbol: Name of the inner estimator's field, which is also the argument name of the generated method.
  • name::Symbol: Name of the windowed estimator type.
  • input::Symbol: Input kind of the generated method, :MatNum, :VecNum or :AssetPanel.
  • has_mean::Bool: true when the method declares a mean keyword.

Returns

  • def::Expr: The Expr(:function, ...) of the forwarding method.

Related

source

Available-case co-moments

An available-case fit estimates each asset from the rows where its return is finite and the asset is active. The two functions below build the block that the available-case coskewness and cokurtosis read. A co-moment reads every deviation from the mean with FullMoment, or only the negative deviations with SemiMoment, and coverage_comoment_deviations is the one step where the two differ.

PortfolioOptimisers.coverage_comoment_deviations — Function
coverage_comoment_deviations(alg::FullMoment, Xo::MatNum, mu::VecNum) -> MatNum
coverage_comoment_deviations(alg::SemiMoment, Xo::MatNum, mu::VecNum) -> MatNum

Centres an available-case block on the mean of each asset, and clips the deviations when the moment algorithm is a semi-moment.

The two moment algorithms change only this step of a higher-order available-case fit, so coverage_comoment_block has one body for both. FullMoment keeps every deviation, and SemiMoment clips every positive deviation to zero. The plain path without a Coverage Policy does the same.

Arguments

  • alg: Moment algorithm of the estimator.
  • Xo: The oriented block, observations × assets, whose invalid entries are still non-finite.
  • mu: Each asset's available-case mean, over its own finite and active observations.

Returns

  • Y::MatNum: The deviations, non-finite wherever Xo is.

Related

source
PortfolioOptimisers.coverage_comoment_block — Function
coverage_comoment_block(
    alg::AbstractMomentAlgorithm,
    cvg::CoveragePolicy,
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    active_mask::Union{Nothing, AbstractMatrix{<:Bool}},
    dims::Int64
) -> Tuple{Any, Any, Any, Any, Any, Union{Nothing, BitVector}}

Builds the pairwise expansions and the admission mask a higher-order available-case fit reads.

The available-case arms of coskewness and of cokurtosis both call this function. The third and the fourth co-moment differ only in the pair of expansions that they multiply. This function returns both expansions, and each order takes its own product. Neither order folds, so the function reads the whole block and keeps no state.

The function centres the deviations on the available-case mean of each asset, and then sets the invalid entries to zero. After this step, an observation at which an asset has no quote adds nothing to the numerator or to the denominator of a cell of that asset. The mask expansion zc is the same product over the mask, so the denominator of a cell is the number of observations at which every asset of that cell is valid.

Algorithm

  1. Read the valid entries, the per-asset available-case mean and the per-asset bookkeeping of the block with coverage_valid_block.
  2. Centre and clip the block with coverage_comoment_deviations, and zero the invalid entries.
  3. Take the valid mask as integers, Mi.
  4. Build the pairwise expansions z of the deviations and zc of the mask, both observations × assets².
  5. Admit the assets with coverage_admission, against each asset's own observation count.

Arguments

  • alg: Moment algorithm of the estimator.
  • cvg: The policy the estimator carries.
  • X: Data matrix.
  • active_mask: The active mask of the Asset Panel over the window, or nothing.
  • dims: Dimension along which the computation runs.

Returns

  • (Xo, Y, Mi, z, zc, cmsk)::Tuple: The oriented block, the zeroed deviations, the valid mask as integers, the pairwise expansion of each, and the admitted assets or nothing.

Related

source