Windowed preamble: private API
Functions
PortfolioOptimisers.windowed_preamble — Function
windowed_preamble(
est,
w::Union{Nothing, DynamicAbstractWeights, AbstractWeights},
window::Union{Nothing, Integer, AbstractVector{<:Integer}},
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
iv,
dims,
kwargs...
) -> Tuple{Any, Any, Nothing, @NamedTuple{}}
Cut the returns matrix, the observation weights and the per-observation arguments of a windowed moment call to its window.
A window that is an Int or an index vector cuts iv, active_mask and estimation_mask to the rows that it keeps in X, or to the columns when dims = 2. These arguments then align with the windowed returns. A window of nothing cuts nothing.
Algorithm
- Check
dimswithassert_dims. - Resolve
windowtowinwithget_window.nothingresolves to aColon, anIntto the range of the lastwindowobservations, and an index vector to itself. - Cut
Xtowin, and resolve the observation weightsw_newover the cutX, withmoment_window_and_weights. - Make
inner, the inner estimator of the window, withwindowed_inner. It holdsw_newwhen the wrapper holds weights. Otherwise it holds its own weights, cut towin. - Cut
ivtowinwithwindowed_rows. - Cut the two universe masks in
kwargstowinwithwindowed_keywords. The result iskw. - Return
inner, the cutX, the cutivandkw.
Arguments
est: Inner moment estimator. The function copies it with the windowed observation weights.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted. The function cuts it to the window before it resolves it.window: Observation window. An integer selects the lastwindowobservations, and a vector of indices selects those observations.nothingselects every observation.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.iv: Optional implied volatility matrix. The estimator reads it only when one of its inner covariance estimators is anImpliedVolatility. The function cuts it to the window.dims: Dimension along which the computation runs.kwargs...: Keywords formoment_window_and_weights. The function also returns them askw, with the two universe masks cut to the window.
Validation
dims in (1, 2).
Returns
(inner, X, iv, kw): The copy ofestwith the windowed weights, the windowed returns, the windowedivornothing, and the keywords with the two masks cut to the window.
Related
windowed_preamble(
est,
w::Union{Nothing, DynamicAbstractWeights, AbstractWeights},
window::Union{Nothing, Integer, AbstractVector{<:Integer}},
X::AbstractVector{<:Union{Number, AbstractJuMPScalar}}
) -> Tuple{Any, Any}
Cut the returns vector and the observation weights of a windowed moment call to its window.
This method takes no dims, because a vector has one axis. It takes no iv, because no vector method of a windowed estimator takes one.
Algorithm
- Resolve
windowtowinwithget_window, as the matrix method does. - Cut
Xtowin, and resolve the observation weightsw_newover the cutX, withmoment_window_and_weights. - Make
inner, the inner estimator of the window, withwindowed_inner, as the matrix method does. - Return
innerand the cutX.
Arguments
est: Inner moment estimator. The function copies it with the windowed observation weights.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted. The function cuts it to the window before it resolves it.window: Observation window. An integer selects the lastwindowobservations, and a vector of indices selects those observations.nothingselects every observation.X: Data vectorobservations × 1.
Returns
(inner, X): The copy ofestwith the windowed weights, and the windowed returns.
Related
PortfolioOptimisers.windowed_inner — Function
windowed_inner(est, w::ObsWeights, win) -> typeof(est)
windowed_inner(est, w::Nothing, win::Colon) -> typeof(est)
windowed_inner(est, w::Nothing, win::VecInt) -> typeof(est)Make the inner estimator of one window of a windowed moment call, with observation weights that align with the rows of the window.
A window cuts the returns to the observations win. Each observation weight of the fit must then belong to one of those observations. The weights come from one of two places. The wrapper can hold them, and moment_window_and_weights cuts them to win as w. Otherwise the inner estimator can hold weights of its own, one for each observation of the full sample. A cut of the returns that does not cut those weights gives a fit whose weights do not align with its rows.
Algorithm
The method that Julia selects is the algorithm.
wholds weights: returnestwithwin each weights field, throughfactory. The weights of the wrapper replace the weights of the inner estimator, asfactorydoes on all other paths.wisnothingandwinis aColon: returnestunchanged, because the window keeps every observation.wisnothingandwinis an index vector: returnestwith each of its own weights fields indexed towin, throughobs_weights_view.
Step 3 does not normalise the weights again. The fit on the window is the weighted fit on the observations win, with the weights that the full sample gives them. A moment estimator of the library normalises its weights over the rows that it fits, so a rescaled weight vector gives the same fit.
Arguments
est: Inner moment estimator.w: The observation weights of the wrapper, cut towin, ornothing.win: The resolved window, fromget_window.
Returns
inner: The inner estimator of the window.
Related
PortfolioOptimisers.windowed_rows — Function
windowed_rows(x::Nothing, win, dims::Int) -> nothing
windowed_rows(x::AbstractMatrix, win::Colon, dims::Int) -> AbstractMatrix
windowed_rows(x::AbstractMatrix, win::VecInt, dims::Int) -> SubArrayCut a matrix with one row for each observation, or one column when dims = 2, to the observations of a resolved window.
The implied volatility matrix iv and the two universe masks have one entry for each observation. A window that cuts the returns must cut them too. If it does not, their rows do not align with the rows of the returns.
Algorithm
The method that Julia selects is the algorithm.
xisnothing: returnnothing.winis aColon: returnxunchanged.winis an index vector: return a view of the rows ofxinwin, or of its columns whendims == 2.
Arguments
x: The matrix to cut, ornothing.win: The resolved window, fromget_window.dims: Dimension along which the computation runs.
Returns
x′:xover the observations of the window, ornothing.
Related
PortfolioOptimisers.windowed_keywords — Function
windowed_keywords(
win,
dims::Int64;
kwargs...
) -> @NamedTuple{}
Cut the two universe mask keywords of a windowed moment call to the observations of a resolved window.
A mask-aware estimator, such as ExpWeightedCovariance, takes an active_mask keyword. A regime-adjusted estimator also takes an estimation_mask keyword. Each mask has the size of the returns. The windowed estimator cuts the returns to the window, so the function cuts the two masks with them. It passes every other keyword unchanged, and it adds no keyword that the caller did not give.
Algorithm
- Read the keywords as the
NamedTuplekw. - Cut the entries
active_maskandestimation_maskwithwindowed_rows. Keep every other entry. - Return the entries under the names of
kw.
Arguments
win: The resolved window, fromget_window.dims: Dimension along which the computation runs.kwargs...: The keywords of the windowed call.
Returns
kw::NamedTuple: The keywords, with the two masks cut to the window.
Related
PortfolioOptimisers.windowed_panel — Function
windowed_panel(pnl::Option{<:AssetPanel}, win::Colon) -> Option{<:AssetPanel}
windowed_panel(pnl::Option{<:AssetPanel}, win::VecInt) -> Option{<:AssetPanel}Cut an Asset Panel to the observations of a resolved window.
The panel method of a windowed estimator cuts the returns to the window. It gives the inner estimator the panel over the same observations. So a mask-aware inner estimator reads the active mask of the rows that it fits. A plain inner estimator reduces to the Coverage Universe of the window, not of the whole sample. The panel holds its observations on the rows for each orientation of the returns.
Algorithm
The method that Julia selects is the algorithm.
winis aColon: returnpnlunchanged.winis an index vector: return the view ofpnlover the rows inwinand every asset, withport_opt_view. Anothingpanel staysnothing.
Arguments
pnl: OptionalAssetPanel. The Coverage Universe of the fit comes from its active mask. If it isnothing, the rule reads only the finiteness of the returns.win: The resolved window, fromget_window.
Returns
pnl′::Option{<:AssetPanel}: The Asset Panel over the observations of the window, ornothing.
Related
PortfolioOptimisers.windowed_mask_panel — Function
windowed_mask_panel(pnl::Union{Nothing, AssetPanel{<:Any, Nothing, Nothing}},
rows::VecInt) -> Nothing
windowed_mask_panel(pnl::AssetPanel, rows::VecInt) -> AssetPanelCut the active mask and the estimation mask of an Asset Panel to the observations rows, and keep no Panel Field.
A variance fit reads the two masks of a panel and no Panel Field. A variance series cuts the panel at each of its rows, and the view of every Panel Field costs more than the fit itself. On a panel of 21 Panel Fields over 300 observations, the view of the fields takes about 0.4 ms for each row, and the fit about 0.03 ms. So panel_series_row cuts the masks alone. A panel with no mask has no observation axis, so its cut is nothing, which every moment verb reads as a panel with no mask.
Arguments
pnl: OptionalAssetPanel. The Coverage Universe of the fit comes from its active mask. If it isnothing, the rule reads only the finiteness of the returns.rows: The observations of the fit.
Returns
pt::Option{<:AssetPanel}: A panel that holds the two masks overrows, ornothing.
Related
PortfolioOptimisers.windowed_series_rows — Function
windowed_series_rows(window::Nothing, t::Integer) -> UnitRange
windowed_series_rows(window::Integer, t::Integer) -> UnitRange
windowed_series_rows(window::VecInt, t::Integer) -> VecIntReturn the observations that a windowed estimator keeps in a fit on the observations 1 to t.
Row t of a variance series is a fit on the first t observations. A windowed estimator keeps the observations of its window among them, so row t of its series reads no observation after t.
Algorithm
The method that Julia selects is the algorithm.
windowisnothing: keep1:t.windowis anInt: keep the lastwindowobservations of1:t, which aremax(1, t - window + 1):t.windowis an index vector: keep its entries that are at mostt, in their order. The result is empty whentis less than the first entry.
Arguments
window: Observation window. An integer selects the lastwindowobservations, and a vector of indices selects those observations.nothingselects every observation.t: The last observation of the fit.
Returns
rows: The observations that the fit keeps.
Related
PortfolioOptimisers.windowed_series_row — Function
windowed_series_row(est, w, rows, X, pnl::Nothing; dims::Int = 1, kwargs...)
windowed_series_row(est, w, rows, X, pnl::AssetPanel; dims::Int = 1, kwargs...)Fit the variance of the inner estimator of a windowed estimator on the observations rows, as one row of its variance series.
Algorithm
pnlis an Asset Panel: return the row ofpanel_series_row, which reads the panel overrows.pnlisnothing: cutX, the observation weights,ivand the two universe masks torowswithwindowed_preamble. The results areinner,Xt,ivtandkw.- Call
Statistics.var(inner, Xt; dims, iv = ivt, kw...), and return the result as a vector.
Step 2 passes iv = ivt also when ivt is nothing, as the generated var method of a windowed estimator does. So every row reads the implied volatility of its own observations.
Arguments
est: The inner estimator of the windowed estimator.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.rows: The observations of the fit, fromwindowed_series_rows.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.pnl: OptionalAssetPanel. The Coverage Universe of the fit comes from its active mask. If it isnothing, the rule reads only the finiteness of the returns.dims: Dimension along which the computation runs.kwargs...: Keywords for the variance estimator. The function cutsivand the two universe masks torows.
Returns
v::VecNum: The variance of each asset.
Related
PortfolioOptimisers.panel_series_row — Function
panel_series_row(est, w::Option{<:ObsWeights}, rows::VecInt, X::MatNum,
pnl::Option{<:AssetPanel}; dims::Int = 1, kwargs...) -> VecNum
panel_series_row(est::Union{WindowedVariance, WindowedCovariance,
PortfolioOptimisersCovariance},
w::Option{<:ObsWeights}, rows::VecInt, X::MatNum,
pnl::Option{<:AssetPanel}; dims::Int = 1, kwargs...) -> VecNumFit the panel method of the variance of an estimator on the observations rows, as one row of a variance series over an Asset Panel.
A plain estimator reduces the window to its Coverage Universe, and the panel method of Statistics.var refuses a window that covers no asset, as coverage_mask states for one fit. A variance series fits many windows, and a window can cover no asset while a later one covers some. An example is a window that holds a warm-up row, where every asset is NaN. So the row of such a window is NaN, as the row of a window that keeps no observation is, and the series goes on. A mask-aware estimator reads the window itself and never meets the refusal.
The function reads pnl through the panel route also when pnl is nothing. That route then reduces the window to the assets whose returns are finite, as it does for a matrix call. So the expanding series of a PortfolioOptimisersCovariance and the windowed series share this row.
The first method holds a leaf estimator. A mask-aware leaf, by reads_active_mask, takes the panel method of var. A plain leaf scans the window one time with coverage_keyword_flags, and reduces it with those flags, as the reduce-and-expand root of var does. So a plain leaf pays no scan more than one fit of the root. The second method holds a wrapper, whose panel method of var cuts the window again or passes it to an inner estimator. It asks covers_an_asset, which follows the wrapper to the rows that the innermost fit reads, and then it fits the panel method of var.
Algorithm
- Cut
X, the observation weights,ivand the two universe masks torowswithwindowed_preamble. The results areinner,Xt,ivtandkw. Cut the masks ofpnltorowswithwindowed_mask_panel, which givespt. - A mask-aware leaf: return
Statistics.var(inner, Xt, pt; dims, iv = ivt, kw...)as a vector. - A plain leaf: flag the covered assets with
coverage_keyword_flags. Return aNaNvector with one entry for each asset when no flag istrue. Otherwise sliceXt,ivandivpawithcoverage_keyword_columns, fit the matrix method ofvaron the slice, and expand it withexpand_moment. - A wrapper: return the
NaNvector whencovers_an_assetanswersfalse, and the panel method ofvaras a vector otherwise.
Arguments
est: The variance estimator, or the inner estimator of a windowed estimator.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.rows: The observations of the fit.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.pnl: OptionalAssetPanel. The Coverage Universe of the fit comes from its active mask. If it isnothing, the rule reads only the finiteness of the returns.dims: Dimension along which the computation runs.kwargs...: Keywords for the variance estimator. The function cutsivand the two universe masks torows.
Validation
- When an asset is outside the Coverage Universe, the element type of the moment holds
NaN. AnIntegeror aRationalelement type raises anArgumentError. The check runs only on a window that covers no asset.
Returns
v::VecNum: The variance of each asset,NaNoutside the Coverage Universe of the window.
Related
PortfolioOptimisers.reduces_each_window — Function
reduces_each_window(est::AbstractCovarianceEstimator) -> Bool
reduces_each_window(est::Union{WindowedVariance, WindowedCovariance}) -> Bool
reduces_each_window(est::PortfolioOptimisersCovariance) -> BoolAnswer whether the panel method of the variance of est is the fit over the Coverage Universe of the window that it is given.
The variance series of a PortfolioOptimisersCovariance over an Asset Panel reads the answer for its inner estimator:
- A plain leaf answers
true. Its fit over the rows1totis the fit over the Coverage Universe of those rows, so the composite takes the reduce-and-expand series ofcoverage_variance_series. That series gives the values of a refit throughpanel_series_rowat a lower cost for each row, and it is the series of v0.33. - A mask-aware leaf, by
reads_active_mask, answersfalse, because it reads the window itself. - A windowed estimator answers
false. It fits a window inside the rows1tot, so an asset whose gap is before that window is in its fit. So the composite over it refits each row throughpanel_series_row. - A
PortfolioOptimisersCovariancepasses the window to its own inner estimator without a cut, so it answers for that estimator. A composite over a composite over a plain leaf takes the reduce-and-expand series too.
Arguments
est: The inner estimator of the composite.
Returns
flag::Bool:truewhen the reduce-and-expand series gives the values of the refit.
Related
PortfolioOptimisers.variance_series — Method
variance_series(
est,
w::Union{Nothing, DynamicAbstractWeights, AbstractWeights},
window::Union{Nothing, Integer, AbstractVector{<:Integer}},
X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
pnl::Union{Nothing, AssetPanel};
dims,
kwargs...
) -> Matrix
Compute the point-in-time variance series of a windowed estimator.
Row t of a variance series is the estimate of the estimator on the observations 1 to t. A windowed estimator keeps the last observations of its window, so row t is a fit on the window that ends at t. The window moves with t. Row t reads no observation after t, and the last row equals the batch fit of the estimator on the whole sample.
The method takes the parts of the windowed estimator as positional arguments. WindowedCovariance and WindowedVariance pass their inner estimator, their weights and their window. The expanding series of an AbstractCovarianceEstimator passes the estimator with no weights and no window, so the two series share one definition of row t.
Mathematical definition
\[\begin{align} \mathbf{V}_{tj} &= \hat{\sigma}^2_j\left(\mathbf{X}_{\mathcal{W}_{t}}\right)\,, \\ \mathcal{W}_{t} &= \begin{cases} \left\{1,\, \ldots,\, t\right\} & \text{without a window}\,, \\ \left\{\max(1,\, t - n + 1),\, \ldots,\, t\right\} & \text{for a window of } n \text{ observations}\,, \\ \left\{s \in \mathcal{S} : s \leq t\right\} & \text{for a window of the indices } \mathcal{S}\,. \end{cases} \end{align}\]
Where:
- $\mathbf{V}_{tj}$: Point-in-time variance of asset $j$ at observation $t$. No observation after $t$ enters it.
- $\hat{\sigma}^2_j$: Estimated variance of asset $j$. Here the inner estimator estimates it.
- $\mathbf{X}_{\mathcal{W}_{t}}$: Rows of the returns matrix in $\mathcal{W}_{t}$, with the active mask of the same rows.
- $\mathcal{W}_{t}$: Observations that the window keeps at $t$.
- $n$: Number of observations in the window.
- $\mathcal{S}$: Indices of the observations that an index window selects.
$\mathbf{V}_{tj}$ is NaN when $\mathcal{W}_{t}$ is empty. Over an Asset Panel, row $t$ is NaN also when the window of a plain inner estimator covers no asset, as panel_series_row states. Before an integer window is full, $\mathcal{W}_{t}$ holds every observation from $1$ to $t$.
Algorithm
- Check
dimswithassert_dims. - Fit row
T, the last observation, withwindowed_series_rowon the rows fromwindowed_series_rows. The result isvT. - If row
1keeps no observation, refuse anIntegeror aRationalelement type ofvT. The rows that a window keeps grow witht, so a row holdsNaNonly in this case. - Allocate the series
valwith the element type ofvT, and writevTinto rowT. - For each earlier observation
t, fit rowtthe same way. A row that keeps no observation isNaN, and so is a row over an Asset Panel whose window covers no asset. - Return
val, transposed whendims == 2.
Arguments
est: The inner estimator of the windowed estimator.w: Optional observation weights vectorobservations × 1, or a concrete subtype ofDynamicAbstractWeights. Ifnothing, the computation is unweighted.window: Observation window. An integer selects the lastwindowobservations, and a vector of indices selects those observations.nothingselects every observation.X: Data matrixobservations × assetsif thedimskeyword does not exist ordims = 1,assets × observationswhendims = 2.pnl: OptionalAssetPanel. The Coverage Universe of the fit comes from its active mask. If it isnothing, the rule reads only the finiteness of the returns.dims: Dimension along which the computation runs.kwargs...: Keywords for the variance estimator. The function cuts the two universe masks to the rows of each fit.
Validation
dims in (1, 2).- When the window keeps no observation of the first row, the element type of the variance holds
NaN. AnIntegeror aRationalelement type raises anArgumentError.
Returns
val::Matrix{<:Number}: Variance series, shaped as(T, N)ifdims == 1or(N, T)ifdims == 2.
Related
PortfolioOptimisers.SeedWindowed — Type
const SeedWindowed = Union{WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow},
WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow},
WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow},
WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow},
WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}}Groups the windowed estimators under a SeedWindow, which are the windowed estimators that fold.
The fourth type parameter of a windowed estimator is the type of its rule field, so the alias selects the rule by dispatch. A windowed estimator under a RollingWindow is not in the group, and it does not fold.
Related
PortfolioOptimisers.seed_window_path — Method
seed_window_path(
_::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> String
Return "rule", the path to the seed window inside a windowed estimator under a SeedWindow.
The walk of seed_window_path over an estimator tree stops at this method. The walk puts the names of the fields above the estimator before "rule".
Arguments
est: Windowed estimator under a seed window.
Returns
path::String:"rule".
Related
PortfolioOptimisers.observation_count — Method
observation_count(
est::Union{WindowedCokurtosis{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCoskewness{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedCovariance{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedExpectedReturns{<:Any, <:Any, <:Any, <:SeedWindow}, WindowedVariance{<:Any, <:Any, <:Any, <:SeedWindow}}
) -> Any
SeedWindowed method of observation_count.
A windowed estimator under a SeedWindow keeps no state of its own, and its inner estimator holds the state of the fold. So the method reads the count of the inner estimator. That count is the number of rows that the inner estimator folded: the rows of the first block that the window kept, and every later row.
Arguments
est: Windowed estimator under a seed window.
Validation
- The inner estimator holds a partial-fit state with a count, as
observation_countstates. AnArgumentErroris thrown otherwise.
Returns
n::Integer: The number of observations that the inner estimator folded.
Related
PortfolioOptimisers.show_fields — Method
show_fields(
est::Union{WindowedCokurtosis, WindowedCoskewness, WindowedCovariance, WindowedExpectedReturns, WindowedVariance}
) -> Union{Tuple{Symbol, Symbol, Symbol}, NTuple{4, Symbol}}
Return the fields of a windowed estimator that the printout shows, which are every field except cache.
The cache field holds the Partial Fit State of an online fit. It is not configuration. Without this method, the state prints under the estimator each time a Prior or an optimiser that holds the estimator prints. To show the field for one type, call set_show_nothing_fields! with the name of the type and true, for example set_show_nothing_fields!(:WindowedCovariance, true).
Arguments
est: Windowed estimator. The function reads its type alone.
Returns
fields::Tuple: The field names to show, which are every field name except:cache.
Related