Array views: private API

Utility functions

Most functions below serve the macros that declare the types of the library. A field tagged @pprop or @cprop takes its value from the prior or from the optimiser when the caller leaves it empty, and sel makes that choice. @propagatable records the tagged fields of each type it declares, and @forward_properties gives a type properties that read through to the fields it holds. The rest are helpers that many files share, such as get_window, which gives the index range of an observation window.

A history of rank three holds its observations on its last axis, and a matrix holds them on its first. observation_axis names the axis of each rank, so a helper that cuts or appends the rows of a history of any rank reads it there.

PortfolioOptimisers.get_window — Function
get_window(window::Option{<:Colon}, args...) -> Colon
get_window(window::Integer, X::MatNum, dims::Int = 1) -> VecInt
get_window(window::Integer, X::VecNum, args...) -> VecInt
get_window(window::VecInt, args...) -> VecInt

Indices of the observation window of a data array.

Algorithm

The type of window names the rule, and each rule is one step.

  1. window is nothing or a Colon: return Colon(), which selects every observation.
  2. window is an integer: read start, the first index of X along the observation axis, and stop, its last index. Return the range max(start, stop - window + 1):stop, which is the last window observations.
  3. window is a vector of integers: return window itself, so the caller states the observations directly.

Step 2 clamps the lower end at start, so a window larger than the number of observations gives every observation and not an error. The observation axis of a matrix is dims. A vector has one axis, so its method ignores dims.

Arguments

  • window: Observation window. An integer selects the last window observations, and a vector of indices selects those observations.
    • ::Option{<:Colon}: Returns Colon().
    • ::Integer: Returns the last window observations. If window is larger than the number of observations, the result is every observation and not an error.
    • ::VecInt: Returns the window argument.
  • X: Data matrix or vector.
  • dims: Dimension along which the computation runs.

Returns

  • window::Union{Colon, VecInt}: The indices of the window.

Related

source
PortfolioOptimisers.observation_axis — Function
observation_axis(A::AbstractVector) -> Int
observation_axis(A::AbstractMatrix) -> Int
observation_axis(A::AbstractArray{<:Any, 3}) -> Int

Return the axis of a history that holds its observations.

A history of rank one or two holds one observation per row, as the returns observations × assets do. A history of rank three, such as an exposure history assets × factors × observations, holds them on its last axis, so the slice of one observation is one contiguous block. A helper that cuts or appends the rows of a history of any rank reads the axis here, so the library states the rule of the layout once.

Algorithm

The method that Julia selects is the algorithm.

  1. A vector or a matrix: return 1.
  2. An array of rank three: return 3.

Arguments

  • A: A history.

Returns

  • d::Int: The observation axis of A.

Examples

julia> PortfolioOptimisers.observation_axis(zeros(4, 2))1julia> PortfolioOptimisers.observation_axis(zeros(2, 3, 4))3

Related

source

View functions

An optimisation over a subset of the assets, such as each inner optimisation of NestedClustered, needs the part of every input that belongs to those assets. The functions below take that part of a vector, a matrix or a higher moment, and return nothing or a scalar unchanged. fourth_moment_index_generator gives the indices that select the entries of the subset from a fourth-moment matrix. view_child views one field of an estimator for port_opt_view, and keeps a precomputed optimisation result, such as a fallback, as it is.

PortfolioOptimisers.nothing_scalar_array_view — Function
nothing_scalar_array_view(
    x::Union{Nothing, Number, Pair, VecPair, Dict,
             AbstractEstimatorValueAlgorithm,
             DynamicAbstractWeights, AbstractEstimator, AbstractAlgorithm,
             StatsBase.CovarianceEstimator},
    ::Any
) -> x
nothing_scalar_array_view(x::AbstractVector, i) -> view(x, i)
nothing_scalar_array_view(x::VecScalar, i) -> VecScalar(; v = view(x.v, i), s = x.s)
nothing_scalar_array_view(x::AbstractMatrix, i) -> view(x, i, i)
nothing_scalar_array_view(
    x::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}},
    i
) -> [nothing_scalar_array_view(xi, i) for xi in x]

View of a value at the asset index i, where the value can be nothing, a scalar or an array.

Algorithm

The method that Julia selects is the algorithm. Each step is one method, and no method allocates a copy of the data.

  1. x carries no asset axis, because it is nothing, a scalar, a pair, a dictionary, a value algorithm, a set of dynamic weights, an estimator, an algorithm or a StatsBase.CovarianceEstimator: return x itself.
  2. x is a vector: return view(x, i), one entry per selected asset.
  3. x is a VecScalar: return a new VecScalar whose vector part is view(x.v, i) and whose scalar part is x.s. The scalar part has no asset axis.
  4. x is a matrix: return view(x, i, i), which selects the same index on both axes. This is the rule for a square matrix over the assets, such as a covariance matrix or a similarity matrix. A matrix whose two axes are different needs nothing_scalar_array_view_odd_order.
  5. x is a vector of vectors, matrices or VecScalars: apply step 2, 3 or 4 to each element, and collect the views into a new outer vector of the same length. The element type of the vector selects this step, and it must be a subtype of the Union that the signature names. A vector that holds both a vector and a matrix has the element type Array{T}. That type is a subtype of neither AbstractVector nor AbstractMatrix, so the vector resolves on step 2, and the index selects elements of the outer vector.

Arguments

  • x: Input value.
  • i: Index or indices to view.

Returns

  • x: Input value.

    • ::Union{Nothing, Number, Pair, VecPair, Dict, AbstractEstimatorValueAlgorithm, DynamicAbstractWeights, AbstractEstimator, AbstractAlgorithm, StatsBase.CovarianceEstimator}: Returns x unchanged.
    • ::AbstractVector: Returns view(x, i).
    • ::VecScalar: Returns VecScalar(; v = view(x.v, i), s = x.s).
    • ::AbstractMatrix: Returns view(x, i, i).
    • ::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}}: Returns a vector of views for each element in x.

Examples

julia> PortfolioOptimisers.nothing_scalar_array_view(nothing, 1:2)julia> PortfolioOptimisers.nothing_scalar_array_view(3.0, 1:2)3.0julia> PortfolioOptimisers.nothing_scalar_array_view([1.0, 2.0, 3.0], 2:3)2-element view(::Vector{Float64}, 2:3) with eltype Float64: 2.0 3.0julia> PortfolioOptimisers.nothing_scalar_array_view([[1, 2], [3, 4]], 1)2-element Vector{SubArray{Int64, 0, Vector{Int64}, Tuple{Int64}, true}}: fill(1) fill(3)

Related

source
nothing_scalar_array_view(
    x::MedianCenteringFunction,
    _
) -> MedianCenteringFunction

Return the MedianCenteringFunction x unchanged.

A centring function holds no value for each asset, so a view of a subset of the assets has nothing to slice.

Related

source
nothing_scalar_array_view(
    td::TimeDependent,
    i
) -> TimeDependent

Slices a TimeDependent schedule of scalar or array values, such as warm starts or initial weights, to the asset indices i.

A function schedule is not sliced, because it reads the sliced universe from the field rd of its context.

Algorithm

  1. When td.val is a vector, slice each entry with nothing_scalar_array_view.
  2. When td.default is not NoDefault, slice it too.
  3. Return a new schedule with the sliced values and the same bind.

Related

source
PortfolioOptimisers.nothing_scalar_array_view_odd_order — Function
nothing_scalar_array_view_odd_order(::Nothing, i, j)
nothing_scalar_array_view_odd_order(x::AbstractMatrix, i, j) -> view(x, i, j)

View of a value that can be nothing or a matrix, with a different index on each axis.

Algorithm

  1. x is nothing: return nothing.
  2. x is a matrix: return view(x, i, j), which selects i on the row axis and j on the column axis.

The two axes take different indices. This is the difference from nothing_scalar_array_view. The main case is the coskewness matrix, of size $N \times N^{2}$. Its row index selects assets, and its column index selects the pairs of assets that the columns hold. The caller gives j, and this verb does not make it. fourth_moment_index_generator makes that column index. Other callers pass : as one of the two indices.

Arguments

  • x: Input value.
  • i, j: Indices to view.

Returns

  • The corresponding view or nothing.

Examples

julia> PortfolioOptimisers.nothing_scalar_array_view_odd_order(nothing, 1, 2)julia> PortfolioOptimisers.nothing_scalar_array_view_odd_order([1 2; 3 4], 1, 2)0-dimensional view(::Matrix{Int64}, 1, 2) with eltype Int64:2

Related

source
PortfolioOptimisers.nothing_scalar_array_getindex — Function
nothing_scalar_array_getindex(
    x::Union{Nothing, Number, Pair, VecPair, Dict,
             AbstractEstimatorValueAlgorithm,
             DynamicAbstractWeights},
    ::Any
) -> x
nothing_scalar_array_getindex(x::AbstractVector, i) -> x[i]
nothing_scalar_array_getindex(x::StatsBase.AbstractWeights, i) -> weights of the same kind
nothing_scalar_array_getindex(x::VecScalar, i) -> VecScalar(; v = x.v[i], s = x.s)
nothing_scalar_array_getindex(x::AbstractMatrix, i) -> x[i, i]
nothing_scalar_array_getindex(
    x::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}},
    i
) -> [nothing_scalar_array_getindex(xi, i) for xi in x]

Copy of a value at the asset index i, where the value can be nothing, a scalar or an array.

Algorithm

The method that Julia selects is the algorithm. It is the copying twin of nothing_scalar_array_view. Every step returns a new array and not a view.

  1. x carries no asset axis, because it is nothing, a scalar, a pair, a dictionary, a value algorithm or a set of dynamic weights: return x itself.
  2. x is a vector: return x[i], a new vector with one entry per selected asset.
    • x is a set of StatsBase.Weights, AnalyticWeights, FrequencyWeights or ProbabilityWeights: build weights of the same kind from x.values[i]. The getindex of StatsBase builds the result with the type of the stored values, so it fails on weights that store a range.
  3. x is a VecScalar: return a new VecScalar whose vector part is x.v[i] and whose scalar part is x.s.
  4. x is a matrix: return x[i, i], which selects the same index on both axes. This is the rule for a square matrix over the assets. A matrix whose two axes are different needs nothing_scalar_array_getindex_odd_order.
  5. x is a vector of vectors, matrices or VecScalars: apply step 2, 3 or 4 to each element, and collect the results into a new vector. The element type of the vector selects this step, and it must be a subtype of the Union that the signature names. A vector that holds both a vector and a matrix has the element type Array{T}. That type is a subtype of neither AbstractVector nor AbstractMatrix, so the vector resolves on step 2, and the index selects elements of the outer vector.

The type list of step 1 is shorter than the list of nothing_scalar_array_view. An estimator, an algorithm and a StatsBase.CovarianceEstimator reach the view verb and not this one, because only the view verb is the leaf of port_opt_view.

Arguments

  • x: Input value.
  • i: Index or indices to copy.

Returns

  • x: Input value.

    • ::Union{Nothing, Number, Pair, VecPair, Dict, AbstractEstimatorValueAlgorithm, DynamicAbstractWeights}: Returns x unchanged.
    • ::AbstractVector: Returns x[i].
    • ::StatsBase.AbstractWeights, except StatsBase.UnitWeights: Returns weights of the same kind that hold x.values[i].
    • ::VecScalar: Returns VecScalar(; v = x.v[i], s = x.s).
    • ::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}}: Returns a vector of elements indexed by i.
    • ::AbstractMatrix: Returns x[i, i].

Examples

julia> PortfolioOptimisers.nothing_scalar_array_getindex(nothing, 1:2)julia> PortfolioOptimisers.nothing_scalar_array_getindex(3.0, 1:2)3.0julia> PortfolioOptimisers.nothing_scalar_array_getindex([1.0, 2.0, 3.0], 2:3)2-element Vector{Float64}: 2.0 3.0julia> PortfolioOptimisers.nothing_scalar_array_getindex([[1, 2], [3, 4]], 1)2-element Vector{Int64}: 1 3

Related

source
PortfolioOptimisers.nothing_scalar_array_getindex_odd_order — Function
nothing_scalar_array_getindex_odd_order(::Nothing, i, j)
nothing_scalar_array_getindex_odd_order(x::AbstractMatrix, i, j) -> x[i, j]

Copy of a value that can be nothing or a matrix, with a different index on each axis.

Algorithm

  1. x is nothing: return nothing.
  2. x is a matrix: return the copy x[i, j], which selects i on the row axis and j on the column axis.

This is the copying twin of nothing_scalar_array_view_odd_order. It takes different indices on the two axes, because the two axes of the matrix are different.

Arguments

  • x: Input value.
  • i, j: Indices to access.

Returns

  • The corresponding matrix element or nothing.

Examples

julia> PortfolioOptimisers.nothing_scalar_array_getindex_odd_order(nothing, 1, 2)julia> PortfolioOptimisers.nothing_scalar_array_getindex_odd_order([1 2; 3 4], 1, 2)2

Related

source
PortfolioOptimisers.fourth_moment_index_generator — Function
fourth_moment_index_generator(
    N::Integer,
    i
) -> Vector{Int64}

Index vector of the asset pairs that the assets i make, among the $N^{2}$ pairs of $N$ assets. It selects the sub-matrix of the assets i from a cokurtosis matrix, and the columns of the assets i from a coskewness matrix.

Mathematical definition

\[\begin{align} \mathrm{idx} &= \left( (c - 1) N + r \right)_{c \in \boldsymbol{i},\ r \in \boldsymbol{i}}\,. \end{align}\]

Where:

  • $N$: Number of assets.
  • $\boldsymbol{i}$: The selected asset indices, of length $n$.
  • $r$, $c$: The row and the column of an asset pair in the $N \times N$ grid of pairs.

$(c - 1) N + r$ is the column-major linear index of the pair $(r, c)$ in that grid. The grid is the axis of the square cokurtosis matrix $\mathbf{K}$, of size $N^{2} \times N^{2}$. idx selects the $n^{2}$ pairs that the $n$ selected assets make, on either axis of $\mathbf{K}$. c is the outer loop, so idx is also in the column-major order of the sub-grid.

Algorithm

  1. Make idx, an empty vector of integers, with room for length(i)^2 entries.
  2. For each c in i, take the linear index range of column c, which is ((c - 1) * N + 1):(c * N), and select the entries i of that range.
  3. Append the selected entries to idx.
  4. Return idx.

Arguments

  • N: Number of assets, the size of the full covariance matrix.
  • i: Indices of the selected assets.

Returns

  • idx::VecInt: Linear indices of the selected asset pairs.

Examples

julia> PortfolioOptimisers.fourth_moment_index_generator(3, [1, 2])4-element Vector{Int64}: 1 2 4 5
source
PortfolioOptimisers.view_child — Function
view_child(v, i, args...)

View of one field. The port_opt_view methods that @propagatable makes call it, and so does the hand-written view of every optimiser that holds a fallback fb.

It is the view twin of the leaf of factory. It views a child at the asset index, but it keeps a precomputed optimisation result with no change. A result is in a field as a fallback fb. The fallback loop of optimise returns that result without a solve, on the universe of its own solve. A function such as investable_reduction views an optimiser and never reads its fallback. A refusal of the result there refuses a fallback that nothing reads.

A TimeDependent schedule is not a result, so view_child sends it to port_opt_view. A schedule that holds a result still refuses a view of an asset subset.

Algorithm

The method that Julia selects is the algorithm.

  1. v is a precomputed optimisation result: return v unchanged.
  2. v is anything else: return port_opt_view of v at i, and forward args....

Arguments

  • v: The field value.
  • i: Index selection.
  • args...: Threaded tail, forwarded to port_opt_view.

Returns

  • The view of the field value, or v with no change when it is a precomputed optimisation result.

Related

source
PortfolioOptimisers.obs_weights_index — Function
obs_weights_index(w, i) -> typeof(w)
obs_weights_index(w, obs::AbstractVector{Bool}) -> typeof(w)

Index one observation-weights field at the observations i, for obs_weights_view.

@propagatable calls this function on each @wprop field of the method that it makes for obs_weights_view. A hand-written method of that verb calls it on each weights field of its own type.

Algorithm

The method that Julia selects on the type of i is the algorithm.

  1. i is a mask of the full sample, a vector of Bool: return observed_weights(w, obs). That method checks that a weight vector has one entry for each observation, so a weight vector of the wrong length raises an error and is not cut to a wrong vector with no message.
  2. i is any other index, such as the positions of a window: return nothing_scalar_array_getindex(w, i). A window is a part of the sample that the caller chose, so its positions are inside the weights.

Both methods keep the type of a StatsBase.AbstractWeights, keep a DynamicAbstractWeights, and keep nothing.

Arguments

  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • i: The observations to keep: a mask of the full sample, or their positions.

Validation

Returns

  • w: The weights at the kept observations.

Related

source
PortfolioOptimisers.obs_weights_fields — Function
obs_weights_fields(x, i) -> typeof(x)

Rebuild x with obs_weights_view applied to every field, through the keyword constructor of its type.

A hand-written method of obs_weights_view calls this function for a type that is not @propagatable and whose fields hold estimators that read the observations, such as JuMPOptimiser and the uncertainty set estimators. A field that holds no estimator comes back with no change from the fallback of the verb. The function derives the field list from the type, so a new field joins the cut with no edit here.

Algorithm

  1. Apply obs_weights_view at i to each field of x, in the order of the struct.
  2. Call the keyword constructor of the type of x with every field name and its cut value.

Arguments

  • x: A struct whose keyword constructor takes every field name as a keyword.
  • i: The observations to keep: a mask of the full sample, or their positions.

Returns

  • x: The struct at the kept observations.

Related

source