Array views: private API
Utility functions
Most functions below serve the macros that declare the types of the library. A field tagged @pprop or @cprop takes its value from the prior or from the optimiser when the caller leaves it empty, and sel makes that choice. @propagatable records the tagged fields of each type it declares, and @forward_properties gives a type properties that read through to the fields it holds. The rest are helpers that many files share, such as get_window, which gives the index range of an observation window.
A history of rank three holds its observations on its last axis, and a matrix holds them on its first. observation_axis names the axis of each rank, so a helper that cuts or appends the rows of a history of any rank reads it there.
PortfolioOptimisers.get_window — Function
get_window(window::Option{<:Colon}, args...) -> Colon
get_window(window::Integer, X::MatNum, dims::Int = 1) -> VecInt
get_window(window::Integer, X::VecNum, args...) -> VecInt
get_window(window::VecInt, args...) -> VecIntIndices of the observation window of a data array.
Algorithm
The type of window names the rule, and each rule is one step.
windowisnothingor aColon: returnColon(), which selects every observation.windowis an integer: readstart, the first index ofXalong the observation axis, andstop, its last index. Return the rangemax(start, stop - window + 1):stop, which is the lastwindowobservations.windowis a vector of integers: returnwindowitself, so the caller states the observations directly.
Step 2 clamps the lower end at start, so a window larger than the number of observations gives every observation and not an error. The observation axis of a matrix is dims. A vector has one axis, so its method ignores dims.
Arguments
window: Observation window. An integer selects the lastwindowobservations, and a vector of indices selects those observations.::Option{<:Colon}: ReturnsColon().::Integer: Returns the lastwindowobservations. Ifwindowis larger than the number of observations, the result is every observation and not an error.::VecInt: Returns thewindowargument.
X: Data matrix or vector.dims: Dimension along which the computation runs.
Returns
window::Union{Colon, VecInt}: The indices of the window.
Related
PortfolioOptimisers.observation_axis — Function
observation_axis(A::AbstractVector) -> Int
observation_axis(A::AbstractMatrix) -> Int
observation_axis(A::AbstractArray{<:Any, 3}) -> IntReturn the axis of a history that holds its observations.
A history of rank one or two holds one observation per row, as the returns observations × assets do. A history of rank three, such as an exposure history assets × factors × observations, holds them on its last axis, so the slice of one observation is one contiguous block. A helper that cuts or appends the rows of a history of any rank reads the axis here, so the library states the rule of the layout once.
Algorithm
The method that Julia selects is the algorithm.
- A vector or a matrix: return
1. - An array of rank three: return
3.
Arguments
A: A history.
Returns
d::Int: The observation axis ofA.
Examples
julia> PortfolioOptimisers.observation_axis(zeros(4, 2))1julia> PortfolioOptimisers.observation_axis(zeros(2, 3, 4))3Related
View functions
An optimisation over a subset of the assets, such as each inner optimisation of NestedClustered, needs the part of every input that belongs to those assets. The functions below take that part of a vector, a matrix or a higher moment, and return nothing or a scalar unchanged. fourth_moment_index_generator gives the indices that select the entries of the subset from a fourth-moment matrix. view_child views one field of an estimator for port_opt_view, and keeps a precomputed optimisation result, such as a fallback, as it is.
PortfolioOptimisers.nothing_scalar_array_view — Function
nothing_scalar_array_view(
x::Union{Nothing, Number, Pair, VecPair, Dict,
AbstractEstimatorValueAlgorithm,
DynamicAbstractWeights, AbstractEstimator, AbstractAlgorithm,
StatsBase.CovarianceEstimator},
::Any
) -> x
nothing_scalar_array_view(x::AbstractVector, i) -> view(x, i)
nothing_scalar_array_view(x::VecScalar, i) -> VecScalar(; v = view(x.v, i), s = x.s)
nothing_scalar_array_view(x::AbstractMatrix, i) -> view(x, i, i)
nothing_scalar_array_view(
x::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}},
i
) -> [nothing_scalar_array_view(xi, i) for xi in x]View of a value at the asset index i, where the value can be nothing, a scalar or an array.
Algorithm
The method that Julia selects is the algorithm. Each step is one method, and no method allocates a copy of the data.
xcarries no asset axis, because it isnothing, a scalar, a pair, a dictionary, a value algorithm, a set of dynamic weights, an estimator, an algorithm or aStatsBase.CovarianceEstimator: returnxitself.xis a vector: returnview(x, i), one entry per selected asset.xis aVecScalar: return a newVecScalarwhose vector part isview(x.v, i)and whose scalar part isx.s. The scalar part has no asset axis.xis a matrix: returnview(x, i, i), which selects the same index on both axes. This is the rule for a square matrix over the assets, such as a covariance matrix or a similarity matrix. A matrix whose two axes are different needsnothing_scalar_array_view_odd_order.xis a vector of vectors, matrices orVecScalars: apply step 2, 3 or 4 to each element, and collect the views into a new outer vector of the same length. The element type of the vector selects this step, and it must be a subtype of theUnionthat the signature names. A vector that holds both a vector and a matrix has the element typeArray{T}. That type is a subtype of neitherAbstractVectornorAbstractMatrix, so the vector resolves on step 2, and the index selects elements of the outer vector.
Arguments
x: Input value.i: Index or indices to view.
Returns
x: Input value.::Union{Nothing, Number, Pair, VecPair, Dict, AbstractEstimatorValueAlgorithm, DynamicAbstractWeights, AbstractEstimator, AbstractAlgorithm, StatsBase.CovarianceEstimator}: Returnsxunchanged.::AbstractVector: Returnsview(x, i).::VecScalar: ReturnsVecScalar(; v = view(x.v, i), s = x.s).::AbstractMatrix: Returnsview(x, i, i).::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}}: Returns a vector of views for each element inx.
Examples
julia> PortfolioOptimisers.nothing_scalar_array_view(nothing, 1:2)julia> PortfolioOptimisers.nothing_scalar_array_view(3.0, 1:2)3.0julia> PortfolioOptimisers.nothing_scalar_array_view([1.0, 2.0, 3.0], 2:3)2-element view(::Vector{Float64}, 2:3) with eltype Float64: 2.0 3.0julia> PortfolioOptimisers.nothing_scalar_array_view([[1, 2], [3, 4]], 1)2-element Vector{SubArray{Int64, 0, Vector{Int64}, Tuple{Int64}, true}}: fill(1) fill(3)Related
nothing_scalar_array_view(
x::MedianCenteringFunction,
_
) -> MedianCenteringFunction
Return the MedianCenteringFunction x unchanged.
A centring function holds no value for each asset, so a view of a subset of the assets has nothing to slice.
Related
nothing_scalar_array_view(
td::TimeDependent,
i
) -> TimeDependent
Slices a TimeDependent schedule of scalar or array values, such as warm starts or initial weights, to the asset indices i.
A function schedule is not sliced, because it reads the sliced universe from the field rd of its context.
Algorithm
- When
td.valis a vector, slice each entry withnothing_scalar_array_view. - When
td.defaultis notNoDefault, slice it too. - Return a new schedule with the sliced values and the same
bind.
Related
PortfolioOptimisers.nothing_scalar_array_view_odd_order — Function
nothing_scalar_array_view_odd_order(::Nothing, i, j)
nothing_scalar_array_view_odd_order(x::AbstractMatrix, i, j) -> view(x, i, j)View of a value that can be nothing or a matrix, with a different index on each axis.
Algorithm
xisnothing: returnnothing.xis a matrix: returnview(x, i, j), which selectsion the row axis andjon the column axis.
The two axes take different indices. This is the difference from nothing_scalar_array_view. The main case is the coskewness matrix, of size $N \times N^{2}$. Its row index selects assets, and its column index selects the pairs of assets that the columns hold. The caller gives j, and this verb does not make it. fourth_moment_index_generator makes that column index. Other callers pass : as one of the two indices.
Arguments
x: Input value.i,j: Indices to view.
Returns
- The corresponding view or
nothing.
Examples
julia> PortfolioOptimisers.nothing_scalar_array_view_odd_order(nothing, 1, 2)julia> PortfolioOptimisers.nothing_scalar_array_view_odd_order([1 2; 3 4], 1, 2)0-dimensional view(::Matrix{Int64}, 1, 2) with eltype Int64:2Related
PortfolioOptimisers.nothing_scalar_array_getindex — Function
nothing_scalar_array_getindex(
x::Union{Nothing, Number, Pair, VecPair, Dict,
AbstractEstimatorValueAlgorithm,
DynamicAbstractWeights},
::Any
) -> x
nothing_scalar_array_getindex(x::AbstractVector, i) -> x[i]
nothing_scalar_array_getindex(x::StatsBase.AbstractWeights, i) -> weights of the same kind
nothing_scalar_array_getindex(x::VecScalar, i) -> VecScalar(; v = x.v[i], s = x.s)
nothing_scalar_array_getindex(x::AbstractMatrix, i) -> x[i, i]
nothing_scalar_array_getindex(
x::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}},
i
) -> [nothing_scalar_array_getindex(xi, i) for xi in x]Copy of a value at the asset index i, where the value can be nothing, a scalar or an array.
Algorithm
The method that Julia selects is the algorithm. It is the copying twin of nothing_scalar_array_view. Every step returns a new array and not a view.
xcarries no asset axis, because it isnothing, a scalar, a pair, a dictionary, a value algorithm or a set of dynamic weights: returnxitself.xis a vector: returnx[i], a new vector with one entry per selected asset.xis a set ofStatsBase.Weights,AnalyticWeights,FrequencyWeightsorProbabilityWeights: build weights of the same kind fromx.values[i]. ThegetindexofStatsBasebuilds the result with the type of the stored values, so it fails on weights that store a range.
xis aVecScalar: return a newVecScalarwhose vector part isx.v[i]and whose scalar part isx.s.xis a matrix: returnx[i, i], which selects the same index on both axes. This is the rule for a square matrix over the assets. A matrix whose two axes are different needsnothing_scalar_array_getindex_odd_order.xis a vector of vectors, matrices orVecScalars: apply step 2, 3 or 4 to each element, and collect the results into a new vector. The element type of the vector selects this step, and it must be a subtype of theUnionthat the signature names. A vector that holds both a vector and a matrix has the element typeArray{T}. That type is a subtype of neitherAbstractVectornorAbstractMatrix, so the vector resolves on step 2, and the index selects elements of the outer vector.
The type list of step 1 is shorter than the list of nothing_scalar_array_view. An estimator, an algorithm and a StatsBase.CovarianceEstimator reach the view verb and not this one, because only the view verb is the leaf of port_opt_view.
Arguments
x: Input value.i: Index or indices to copy.
Returns
x: Input value.::Union{Nothing, Number, Pair, VecPair, Dict, AbstractEstimatorValueAlgorithm, DynamicAbstractWeights}: Returnsxunchanged.::AbstractVector: Returnsx[i].::StatsBase.AbstractWeights, exceptStatsBase.UnitWeights: Returns weights of the same kind that holdx.values[i].::VecScalar: ReturnsVecScalar(; v = x.v[i], s = x.s).::AbstractVector{<:Union{AbstractVector, AbstractMatrix, VecScalar}}: Returns a vector of elements indexed byi.::AbstractMatrix: Returnsx[i, i].
Examples
julia> PortfolioOptimisers.nothing_scalar_array_getindex(nothing, 1:2)julia> PortfolioOptimisers.nothing_scalar_array_getindex(3.0, 1:2)3.0julia> PortfolioOptimisers.nothing_scalar_array_getindex([1.0, 2.0, 3.0], 2:3)2-element Vector{Float64}: 2.0 3.0julia> PortfolioOptimisers.nothing_scalar_array_getindex([[1, 2], [3, 4]], 1)2-element Vector{Int64}: 1 3Related
PortfolioOptimisers.nothing_scalar_array_getindex_odd_order — Function
nothing_scalar_array_getindex_odd_order(::Nothing, i, j)
nothing_scalar_array_getindex_odd_order(x::AbstractMatrix, i, j) -> x[i, j]Copy of a value that can be nothing or a matrix, with a different index on each axis.
Algorithm
xisnothing: returnnothing.xis a matrix: return the copyx[i, j], which selectsion the row axis andjon the column axis.
This is the copying twin of nothing_scalar_array_view_odd_order. It takes different indices on the two axes, because the two axes of the matrix are different.
Arguments
x: Input value.i,j: Indices to access.
Returns
- The corresponding matrix element or
nothing.
Examples
julia> PortfolioOptimisers.nothing_scalar_array_getindex_odd_order(nothing, 1, 2)julia> PortfolioOptimisers.nothing_scalar_array_getindex_odd_order([1 2; 3 4], 1, 2)2Related
PortfolioOptimisers.fourth_moment_index_generator — Function
fourth_moment_index_generator(
N::Integer,
i
) -> Vector{Int64}
Index vector of the asset pairs that the assets i make, among the $N^{2}$ pairs of $N$ assets. It selects the sub-matrix of the assets i from a cokurtosis matrix, and the columns of the assets i from a coskewness matrix.
Mathematical definition
\[\begin{align} \mathrm{idx} &= \left( (c - 1) N + r \right)_{c \in \boldsymbol{i},\ r \in \boldsymbol{i}}\,. \end{align}\]
Where:
- $N$: Number of assets.
- $\boldsymbol{i}$: The selected asset indices, of length $n$.
- $r$, $c$: The row and the column of an asset pair in the $N \times N$ grid of pairs.
$(c - 1) N + r$ is the column-major linear index of the pair $(r, c)$ in that grid. The grid is the axis of the square cokurtosis matrix $\mathbf{K}$, of size $N^{2} \times N^{2}$. idx selects the $n^{2}$ pairs that the $n$ selected assets make, on either axis of $\mathbf{K}$. c is the outer loop, so idx is also in the column-major order of the sub-grid.
Algorithm
- Make
idx, an empty vector of integers, with room forlength(i)^2entries. - For each
cini, take the linear index range of columnc, which is((c - 1) * N + 1):(c * N), and select the entriesiof that range. - Append the selected entries to
idx. - Return
idx.
Arguments
N: Number of assets, the size of the full covariance matrix.i: Indices of the selected assets.
Returns
idx::VecInt: Linear indices of the selected asset pairs.
Examples
julia> PortfolioOptimisers.fourth_moment_index_generator(3, [1, 2])4-element Vector{Int64}: 1 2 4 5PortfolioOptimisers.view_child — Function
view_child(v, i, args...)View of one field. The port_opt_view methods that @propagatable makes call it, and so does the hand-written view of every optimiser that holds a fallback fb.
It is the view twin of the leaf of factory. It views a child at the asset index, but it keeps a precomputed optimisation result with no change. A result is in a field as a fallback fb. The fallback loop of optimise returns that result without a solve, on the universe of its own solve. A function such as investable_reduction views an optimiser and never reads its fallback. A refusal of the result there refuses a fallback that nothing reads.
A TimeDependent schedule is not a result, so view_child sends it to port_opt_view. A schedule that holds a result still refuses a view of an asset subset.
Algorithm
The method that Julia selects is the algorithm.
vis a precomputed optimisation result: returnvunchanged.vis anything else: returnport_opt_viewofvati, and forwardargs....
Arguments
v: The field value.i: Index selection.args...: Threaded tail, forwarded toport_opt_view.
Returns
- The view of the field value, or
vwith no change when it is a precomputed optimisation result.
Related
PortfolioOptimisers.obs_weights_index — Function
obs_weights_index(w, i) -> typeof(w)
obs_weights_index(w, obs::AbstractVector{Bool}) -> typeof(w)Index one observation-weights field at the observations i, for obs_weights_view.
@propagatable calls this function on each @wprop field of the method that it makes for obs_weights_view. A hand-written method of that verb calls it on each weights field of its own type.
Algorithm
The method that Julia selects on the type of i is the algorithm.
iis a mask of the full sample, a vector ofBool: returnobserved_weights(w, obs). That method checks that a weight vector has one entry for each observation, so a weight vector of the wrong length raises an error and is not cut to a wrong vector with no message.iis any other index, such as the positions of a window: returnnothing_scalar_array_getindex(w, i). A window is a part of the sample that the caller chose, so its positions are inside the weights.
Both methods keep the type of a StatsBase.AbstractWeights, keep a DynamicAbstractWeights, and keep nothing.
Arguments
w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.i: The observations to keep: a mask of the full sample, or their positions.
Validation
- For a mask, the rules of
observed_weights.
Returns
w: The weights at the kept observations.
Related
PortfolioOptimisers.obs_weights_fields — Function
obs_weights_fields(x, i) -> typeof(x)Rebuild x with obs_weights_view applied to every field, through the keyword constructor of its type.
A hand-written method of obs_weights_view calls this function for a type that is not @propagatable and whose fields hold estimators that read the observations, such as JuMPOptimiser and the uncertainty set estimators. A field that holds no estimator comes back with no change from the fallback of the verb. The function derives the field list from the type, so a new field joins the cut with no edit here.
Algorithm
- Apply
obs_weights_viewatito each field ofx, in the order of the struct. - Call the keyword constructor of the type of
xwith every field name and its cut value.
Arguments
x: A struct whose keyword constructor takes every field name as a keyword.i: The observations to keep: a mask of the full sample, or their positions.
Returns
x: The struct at the kept observations.
Related