Windowed preamble: private API

Functions

PortfolioOptimisers.windowed_preamble — Function
windowed_preamble(
    est,
    w::Union{Nothing, DynamicAbstractWeights, AbstractWeights},
    window::Union{Nothing, Integer, AbstractVector{<:Integer}},
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    iv,
    dims,
    kwargs...
) -> Tuple{Any, Any, Nothing, @NamedTuple{}}

Cut the returns matrix, the observation weights and the per-observation arguments of a windowed moment call to its window.

A window that is an Int or an index vector cuts iv, active_mask and estimation_mask to the rows that it keeps in X, or to the columns when dims = 2. These arguments then align with the windowed returns. A window of nothing cuts nothing.

Algorithm

  1. Check dims with assert_dims.
  2. Resolve window to win with get_window. nothing resolves to a Colon, an Int to the range of the last window observations, and an index vector to itself.
  3. Cut X to win, and resolve the observation weights w_new over the cut X, with moment_window_and_weights.
  4. Make inner, the inner estimator of the window, with windowed_inner. It holds w_new when the wrapper holds weights. Otherwise it holds its own weights, cut to win.
  5. Cut iv to win with windowed_rows.
  6. Cut the two universe masks in kwargs to win with windowed_keywords. The result is kw.
  7. Return inner, the cut X, the cut iv and kw.

Arguments

  • est: Inner moment estimator. The function copies it with the windowed observation weights.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted. The function cuts it to the window before it resolves it.
  • window: Observation window. An integer selects the last window observations, and a vector of indices selects those observations. nothing selects every observation.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • iv: Optional implied volatility matrix. The estimator reads it only when one of its inner covariance estimators is an ImpliedVolatility. The function cuts it to the window.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keywords for moment_window_and_weights. The function also returns them as kw, with the two universe masks cut to the window.

Validation

  • dims in (1, 2).

Returns

  • (inner, X, iv, kw): The copy of est with the windowed weights, the windowed returns, the windowed iv or nothing, and the keywords with the two masks cut to the window.

Related

source
windowed_preamble(
    est,
    w::Union{Nothing, DynamicAbstractWeights, AbstractWeights},
    window::Union{Nothing, Integer, AbstractVector{<:Integer}},
    X::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
) -> Tuple{Any, Any}

Cut the returns vector and the observation weights of a windowed moment call to its window.

This method takes no dims, because a vector has one axis. It takes no iv, because no vector method of a windowed estimator takes one.

Algorithm

  1. Resolve window to win with get_window, as the matrix method does.
  2. Cut X to win, and resolve the observation weights w_new over the cut X, with moment_window_and_weights.
  3. Make inner, the inner estimator of the window, with windowed_inner, as the matrix method does.
  4. Return inner and the cut X.

Arguments

  • est: Inner moment estimator. The function copies it with the windowed observation weights.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted. The function cuts it to the window before it resolves it.
  • window: Observation window. An integer selects the last window observations, and a vector of indices selects those observations. nothing selects every observation.
  • X: Data vector observations × 1.

Returns

  • (inner, X): The copy of est with the windowed weights, and the windowed returns.

Related

source
PortfolioOptimisers.windowed_inner — Function
windowed_inner(est, w::ObsWeights, win) -> typeof(est)
windowed_inner(est, w::Nothing, win::Colon) -> typeof(est)
windowed_inner(est, w::Nothing, win::VecInt) -> typeof(est)

Make the inner estimator of one window of a windowed moment call, with observation weights that align with the rows of the window.

A window cuts the returns to the observations win. Each observation weight of the fit must then belong to one of those observations. The weights come from one of two places. The wrapper can hold them, and moment_window_and_weights cuts them to win as w. Otherwise the inner estimator can hold weights of its own, one for each observation of the full sample. A cut of the returns that does not cut those weights gives a fit whose weights do not align with its rows.

Algorithm

The method that Julia selects is the algorithm.

  1. w holds weights: return est with w in each weights field, through factory. The weights of the wrapper replace the weights of the inner estimator, as factory does on all other paths.
  2. w is nothing and win is a Colon: return est unchanged, because the window keeps every observation.
  3. w is nothing and win is an index vector: return est with each of its own weights fields indexed to win, through obs_weights_view.

Step 3 does not normalise the weights again. The fit on the window is the weighted fit on the observations win, with the weights that the full sample gives them. A moment estimator of the library normalises its weights over the rows that it fits, so a rescaled weight vector gives the same fit.

Arguments

  • est: Inner moment estimator.
  • w: The observation weights of the wrapper, cut to win, or nothing.
  • win: The resolved window, from get_window.

Returns

  • inner: The inner estimator of the window.

Related

source
PortfolioOptimisers.windowed_rows — Function
windowed_rows(x::Nothing, win, dims::Int) -> nothing
windowed_rows(x::AbstractMatrix, win::Colon, dims::Int) -> AbstractMatrix
windowed_rows(x::AbstractMatrix, win::VecInt, dims::Int) -> SubArray

Cut a matrix with one row for each observation, or one column when dims = 2, to the observations of a resolved window.

The implied volatility matrix iv and the two universe masks have one entry for each observation. A window that cuts the returns must cut them too. If it does not, their rows do not align with the rows of the returns.

Algorithm

The method that Julia selects is the algorithm.

  1. x is nothing: return nothing.
  2. win is a Colon: return x unchanged.
  3. win is an index vector: return a view of the rows of x in win, or of its columns when dims == 2.

Arguments

  • x: The matrix to cut, or nothing.
  • win: The resolved window, from get_window.
  • dims: Dimension along which the computation runs.

Returns

  • x′: x over the observations of the window, or nothing.

Related

source
PortfolioOptimisers.windowed_keywords — Function
windowed_keywords(
    win,
    dims::Int64;
    kwargs...
) -> @NamedTuple{}

Cut the two universe mask keywords of a windowed moment call to the observations of a resolved window.

A mask-aware estimator, such as ExpWeightedCovariance, takes an active_mask keyword. A regime-adjusted estimator also takes an estimation_mask keyword. Each mask has the size of the returns. The windowed estimator cuts the returns to the window, so the function cuts the two masks with them. It passes every other keyword unchanged, and it adds no keyword that the caller did not give.

Algorithm

  1. Read the keywords as the NamedTuple kw.
  2. Cut the entries active_mask and estimation_mask with windowed_rows. Keep every other entry.
  3. Return the entries under the names of kw.

Arguments

  • win: The resolved window, from get_window.
  • dims: Dimension along which the computation runs.
  • kwargs...: The keywords of the windowed call.

Returns

  • kw::NamedTuple: The keywords, with the two masks cut to the window.

Related

source
PortfolioOptimisers.windowed_panel — Function
windowed_panel(pnl::Option{<:AssetPanel}, win::Colon) -> Option{<:AssetPanel}
windowed_panel(pnl::Option{<:AssetPanel}, win::VecInt) -> Option{<:AssetPanel}

Cut an Asset Panel to the observations of a resolved window.

The panel method of a windowed estimator cuts the returns to the window. It gives the inner estimator the panel over the same observations. So a mask-aware inner estimator reads the active mask of the rows that it fits. A plain inner estimator reduces to the Coverage Universe of the window, not of the whole sample. The panel holds its observations on the rows for each orientation of the returns.

Algorithm

The method that Julia selects is the algorithm.

  1. win is a Colon: return pnl unchanged.
  2. win is an index vector: return the view of pnl over the rows in win and every asset, with port_opt_view. A nothing panel stays nothing.

Arguments

  • pnl: Optional AssetPanel. The Coverage Universe of the fit comes from its active mask. If it is nothing, the rule reads only the finiteness of the returns.
  • win: The resolved window, from get_window.

Returns

  • pnl′::Option{<:AssetPanel}: The Asset Panel over the observations of the window, or nothing.

Related

source
PortfolioOptimisers.windowed_mask_panel — Function
windowed_mask_panel(pnl::Union{Nothing, AssetPanel{<:Any, Nothing, Nothing}},
                    rows::VecInt) -> Nothing
windowed_mask_panel(pnl::AssetPanel, rows::VecInt) -> AssetPanel

Cut the active mask and the estimation mask of an Asset Panel to the observations rows, and keep no Panel Field.

A variance fit reads the two masks of a panel and no Panel Field. A variance series cuts the panel at each of its rows, and the view of every Panel Field costs more than the fit itself. On a panel of 21 Panel Fields over 300 observations, the view of the fields takes about 0.4 ms for each row, and the fit about 0.03 ms. So panel_series_row cuts the masks alone. A panel with no mask has no observation axis, so its cut is nothing, which every moment verb reads as a panel with no mask.

Arguments

  • pnl: Optional AssetPanel. The Coverage Universe of the fit comes from its active mask. If it is nothing, the rule reads only the finiteness of the returns.
  • rows: The observations of the fit.

Returns

  • pt::Option{<:AssetPanel}: A panel that holds the two masks over rows, or nothing.

Related

source
PortfolioOptimisers.windowed_series_rows — Function
windowed_series_rows(window::Nothing, t::Integer) -> UnitRange
windowed_series_rows(window::Integer, t::Integer) -> UnitRange
windowed_series_rows(window::VecInt, t::Integer) -> VecInt

Return the observations that a windowed estimator keeps in a fit on the observations 1 to t.

Row t of a variance series is a fit on the first t observations. A windowed estimator keeps the observations of its window among them, so row t of its series reads no observation after t.

Algorithm

The method that Julia selects is the algorithm.

  1. window is nothing: keep 1:t.
  2. window is an Int: keep the last window observations of 1:t, which are max(1, t - window + 1):t.
  3. window is an index vector: keep its entries that are at most t, in their order. The result is empty when t is less than the first entry.

Arguments

  • window: Observation window. An integer selects the last window observations, and a vector of indices selects those observations. nothing selects every observation.
  • t: The last observation of the fit.

Returns

  • rows: The observations that the fit keeps.

Related

source
PortfolioOptimisers.windowed_series_row — Function
windowed_series_row(est, w, rows, X, pnl::Nothing; dims::Int = 1, kwargs...)
windowed_series_row(est, w, rows, X, pnl::AssetPanel; dims::Int = 1, kwargs...)

Fit the variance of the inner estimator of a windowed estimator on the observations rows, as one row of its variance series.

Algorithm

  1. pnl is an Asset Panel: return the row of panel_series_row, which reads the panel over rows.
  2. pnl is nothing: cut X, the observation weights, iv and the two universe masks to rows with windowed_preamble. The results are inner, Xt, ivt and kw.
  3. Call Statistics.var(inner, Xt; dims, iv = ivt, kw...), and return the result as a vector.

Step 2 passes iv = ivt also when ivt is nothing, as the generated var method of a windowed estimator does. So every row reads the implied volatility of its own observations.

Arguments

  • est: The inner estimator of the windowed estimator.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • rows: The observations of the fit, from windowed_series_rows.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • pnl: Optional AssetPanel. The Coverage Universe of the fit comes from its active mask. If it is nothing, the rule reads only the finiteness of the returns.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keywords for the variance estimator. The function cuts iv and the two universe masks to rows.

Returns

  • v::VecNum: The variance of each asset.

Related

source
PortfolioOptimisers.panel_series_row — Function
panel_series_row(est, w::Option{<:ObsWeights}, rows::VecInt, X::MatNum,
                 pnl::Option{<:AssetPanel}; dims::Int = 1, kwargs...) -> VecNum
panel_series_row(est::Union{WindowedVariance, WindowedCovariance,
                            PortfolioOptimisersCovariance},
                 w::Option{<:ObsWeights}, rows::VecInt, X::MatNum,
                 pnl::Option{<:AssetPanel}; dims::Int = 1, kwargs...) -> VecNum

Fit the panel method of the variance of an estimator on the observations rows, as one row of a variance series over an Asset Panel.

A plain estimator reduces the window to its Coverage Universe, and the panel method of Statistics.var refuses a window that covers no asset, as coverage_mask states for one fit. A variance series fits many windows, and a window can cover no asset while a later one covers some. An example is a window that holds a warm-up row, where every asset is NaN. So the row of such a window is NaN, as the row of a window that keeps no observation is, and the series goes on. A mask-aware estimator reads the window itself and never meets the refusal.

The function reads pnl through the panel route also when pnl is nothing. That route then reduces the window to the assets whose returns are finite, as it does for a matrix call. So the expanding series of a PortfolioOptimisersCovariance and the windowed series share this row.

The first method holds a leaf estimator. A mask-aware leaf, by reads_active_mask, takes the panel method of var. A plain leaf scans the window one time with coverage_keyword_flags, and reduces it with those flags, as the reduce-and-expand root of var does. So a plain leaf pays no scan more than one fit of the root. The second method holds a wrapper, whose panel method of var cuts the window again or passes it to an inner estimator. It asks covers_an_asset, which follows the wrapper to the rows that the innermost fit reads, and then it fits the panel method of var.

Algorithm

  1. Cut X, the observation weights, iv and the two universe masks to rows with windowed_preamble. The results are inner, Xt, ivt and kw. Cut the masks of pnl to rows with windowed_mask_panel, which gives pt.
  2. A mask-aware leaf: return Statistics.var(inner, Xt, pt; dims, iv = ivt, kw...) as a vector.
  3. A plain leaf: flag the covered assets with coverage_keyword_flags. Return a NaN vector with one entry for each asset when no flag is true. Otherwise slice Xt, iv and ivpa with coverage_keyword_columns, fit the matrix method of var on the slice, and expand it with expand_moment.
  4. A wrapper: return the NaN vector when covers_an_asset answers false, and the panel method of var as a vector otherwise.

Arguments

  • est: The variance estimator, or the inner estimator of a windowed estimator.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • rows: The observations of the fit.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • pnl: Optional AssetPanel. The Coverage Universe of the fit comes from its active mask. If it is nothing, the rule reads only the finiteness of the returns.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keywords for the variance estimator. The function cuts iv and the two universe masks to rows.

Validation

  • When an asset is outside the Coverage Universe, the element type of the moment holds NaN. An Integer or a Rational element type raises an ArgumentError. The check runs only on a window that covers no asset.

Returns

  • v::VecNum: The variance of each asset, NaN outside the Coverage Universe of the window.

Related

source
PortfolioOptimisers.reduces_each_window — Function
reduces_each_window(est::AbstractCovarianceEstimator) -> Bool
reduces_each_window(est::Union{WindowedVariance, WindowedCovariance}) -> Bool
reduces_each_window(est::PortfolioOptimisersCovariance) -> Bool

Answer whether the panel method of the variance of est is the fit over the Coverage Universe of the window that it is given.

The variance series of a PortfolioOptimisersCovariance over an Asset Panel reads the answer for its inner estimator:

  • A plain leaf answers true. Its fit over the rows 1 to t is the fit over the Coverage Universe of those rows, so the composite takes the reduce-and-expand series of coverage_variance_series. That series gives the values of a refit through panel_series_row at a lower cost for each row, and it is the series of v0.33.
  • A mask-aware leaf, by reads_active_mask, answers false, because it reads the window itself.
  • A windowed estimator answers false. It fits a window inside the rows 1 to t, so an asset whose gap is before that window is in its fit. So the composite over it refits each row through panel_series_row.
  • A PortfolioOptimisersCovariance passes the window to its own inner estimator without a cut, so it answers for that estimator. A composite over a composite over a plain leaf takes the reduce-and-expand series too.

Arguments

  • est: The inner estimator of the composite.

Returns

  • flag::Bool: true when the reduce-and-expand series gives the values of the refit.

Related

source
PortfolioOptimisers.variance_series — Method
variance_series(
    est,
    w::Union{Nothing, DynamicAbstractWeights, AbstractWeights},
    window::Union{Nothing, Integer, AbstractVector{<:Integer}},
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    pnl::Union{Nothing, AssetPanel};
    dims,
    kwargs...
) -> Matrix

Compute the point-in-time variance series of a windowed estimator.

Row t of a variance series is the estimate of the estimator on the observations 1 to t. A windowed estimator keeps the last observations of its window, so row t is a fit on the window that ends at t. The window moves with t. Row t reads no observation after t, and the last row equals the batch fit of the estimator on the whole sample.

The method takes the parts of the windowed estimator as positional arguments. WindowedCovariance and WindowedVariance pass their inner estimator, their weights and their window. The expanding series of an AbstractCovarianceEstimator passes the estimator with no weights and no window, so the two series share one definition of row t.

Mathematical definition

\[\begin{align} \mathbf{V}_{tj} &= \hat{\sigma}^2_j\left(\mathbf{X}_{\mathcal{W}_{t}}\right)\,, \\ \mathcal{W}_{t} &= \begin{cases} \left\{1,\, \ldots,\, t\right\} & \text{without a window}\,, \\ \left\{\max(1,\, t - n + 1),\, \ldots,\, t\right\} & \text{for a window of } n \text{ observations}\,, \\ \left\{s \in \mathcal{S} : s \leq t\right\} & \text{for a window of the indices } \mathcal{S}\,. \end{cases} \end{align}\]

Where:

  • $\mathbf{V}_{tj}$: Point-in-time variance of asset $j$ at observation $t$. No observation after $t$ enters it.
  • $\hat{\sigma}^2_j$: Estimated variance of asset $j$. Here the inner estimator estimates it.
  • $\mathbf{X}_{\mathcal{W}_{t}}$: Rows of the returns matrix in $\mathcal{W}_{t}$, with the active mask of the same rows.
  • $\mathcal{W}_{t}$: Observations that the window keeps at $t$.
  • $n$: Number of observations in the window.
  • $\mathcal{S}$: Indices of the observations that an index window selects.

$\mathbf{V}_{tj}$ is NaN when $\mathcal{W}_{t}$ is empty. Over an Asset Panel, row $t$ is NaN also when the window of a plain inner estimator covers no asset, as panel_series_row states. Before an integer window is full, $\mathcal{W}_{t}$ holds every observation from $1$ to $t$.

Algorithm

  1. Check dims with assert_dims.
  2. Fit row T, the last observation, with windowed_series_row on the rows from windowed_series_rows. The result is vT.
  3. If row 1 keeps no observation, refuse an Integer or a Rational element type of vT. The rows that a window keeps grow with t, so a row holds NaN only in this case.
  4. Allocate the series val with the element type of vT, and write vT into row T.
  5. For each earlier observation t, fit row t the same way. A row that keeps no observation is NaN, and so is a row over an Asset Panel whose window covers no asset.
  6. Return val, transposed when dims == 2.

Arguments

  • est: The inner estimator of the windowed estimator.
  • w: Optional observation weights vector observations × 1, or a concrete subtype of DynamicAbstractWeights. If nothing, the computation is unweighted.
  • window: Observation window. An integer selects the last window observations, and a vector of indices selects those observations. nothing selects every observation.
  • X: Data matrix observations × assets if the dims keyword does not exist or dims = 1, assets × observations when dims = 2.
  • pnl: Optional AssetPanel. The Coverage Universe of the fit comes from its active mask. If it is nothing, the rule reads only the finiteness of the returns.
  • dims: Dimension along which the computation runs.
  • kwargs...: Keywords for the variance estimator. The function cuts the two universe masks to the rows of each fit.

Validation

  • dims in (1, 2).
  • When the window keeps no observation of the first row, the element type of the variance holds NaN. An Integer or a Rational element type raises an ArgumentError.

Returns

  • val::Matrix{<:Number}: Variance series, shaped as (T, N) if dims == 1 or (N, T) if dims == 2.

Related

source
PortfolioOptimisers.SeedWindowed — Type
const SeedWindowed = Union{WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow},
                           WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow},
                           WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow},
                           WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow},
                           WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}}

Groups the windowed estimators under a SeedWindow, which are the windowed estimators that fold.

The fourth type parameter of a windowed estimator is the type of its rule field, so the alias selects the rule by dispatch. A windowed estimator under a RollingWindow is not in the group, and it does not fold.

Related

source
PortfolioOptimisers.seed_window_path — Method
seed_window_path(
    _::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> String

Return "rule", the path to the seed window inside a windowed estimator under a SeedWindow.

The walk of seed_window_path over an estimator tree stops at this method. The walk puts the names of the fields above the estimator before "rule".

Arguments

  • est: Windowed estimator under a seed window.

Returns

  • path::String: "rule".

Related

source
PortfolioOptimisers.observation_count — Method
observation_count(
    est::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> Any

SeedWindowed method of observation_count.

A windowed estimator under a SeedWindow keeps no state of its own, and its inner estimator holds the state of the fold. So the method reads the count of the inner estimator. That count is the number of rows that the inner estimator folded: the rows of the first block that the window kept, and every later row.

Arguments

  • est: Windowed estimator under a seed window.

Validation

  • The inner estimator holds a partial-fit state with a count, as observation_count states. An ArgumentError is thrown otherwise.

Returns

  • n::Integer: The number of observations that the inner estimator folded.

Related

source
PortfolioOptimisers.show_fields — Method
show_fields(
    est::Union{WindowedCokurtosis, WindowedCoskewness, WindowedCovariance, WindowedExpectedReturns, WindowedVariance}
) -> Union{Tuple{Symbol, Symbol, Symbol}, NTuple{4, Symbol}}

Return the fields of a windowed estimator that the printout shows, which are every field except cache.

The cache field holds the Partial Fit State of an online fit. It is not configuration. Without this method, the state prints under the estimator each time a Prior or an optimiser that holds the estimator prints. To show the field for one type, call set_show_nothing_fields! with the name of the type and true, for example set_show_nothing_fields!(:WindowedCovariance, true).

Arguments

  • est: Windowed estimator. The function reads its type alone.

Returns

  • fields::Tuple: The field names to show, which are every field name except :cache.

Related

source