Returns result

Prices to returns

Every optimiser works on returns, except a FiniteAllocationOptimisationEstimator, which works on prices. The types and functions below hold returns and compute them from prices.

Types

A new returns result subtypes AbstractReturnsResult, carries the fields nx, X, ts and pnl, and adds a method of port_opt_view.

PortfolioOptimisers.AbstractReturnsResult — Type
abstract type AbstractReturnsResult <: AbstractResult

Abstract supertype for all returns result types.

Every concrete type that holds the result of a returns calculation subtypes AbstractReturnsResult. ReturnsResult is the member the library builds.

select_assets, fit_preprocessing and the fold generation of cross-validation dispatch on this supertype. ClusterGroups reads the Feature Matrix from the AssetPanel of the returns data, because preselection runs before any prior exists and the returns data is the only source it can read. The Pr_RR alias names the concrete ReturnsResult, and many methods dispatch on that alias, so these readers take this supertype instead. No check enforces the interface below.

PredictionReturnsResult subtypes this supertype, but its X is a vector of portfolio returns with no asset axis. It does not meet the interface, and every entry point that needs an asset axis throws an error for it.

Interfaces

To implement a new returns result that an asset selector, a cluster preselection and a cross-validation fold can read, subtype AbstractReturnsResult with these fields:

  • nx: The asset names, one per column of X.
  • X: The asset returns matrix, observations × assets. Fold generation reads its row count.
  • ts: The timestamps of the rows of X, or nothing. Fold generation reads it.
  • pnl: An AssetPanel over the assets of X, or nothing. ClusterGroups reads it.

Then implement the following method:

port_opt_view

  • port_opt_view(rd::MyReturnsResult, i) -> MyReturnsResult: Return a view of rd on the assets at i.
  • port_opt_view(rd::MyReturnsResult, i, j, k = :) -> MyReturnsResult: Return a view of rd on the observations at i, the assets at j and the factors at k.

A subtype that carries a panel subselects it together with X. A subtype that implements no method gets the fallback, which throws an ArgumentError.

Arguments

  • rd: The concrete returns result.
  • i: Indices of the assets to keep in the two-argument method, and of the observations to keep in the other.
  • j: Indices of the assets to keep.
  • k: Indices of the factors to keep.

Returns

  • rd::MyReturnsResult: A view of the same type that holds only the selected indices.

Related

source
PortfolioOptimisers.ReturnsResult — Type
struct ReturnsResult{__T_nx, __T_X, __T_nf, __T_F, __T_nb, __T_B, __T_ne, __T_E, __T_ts, __T_iv, __T_ivpa, __T_pnl} <: AbstractReturnsResult

Returns data of a universe: the asset returns, and optionally the factor returns, the benchmark returns, the Exogenous Series, the timestamps, the implied volatilities and an Asset Panel.

prices_to_returns builds it, and the priors, the optimisers and the cross-validation folds read it. All of its matrices share the observation axis, which is the rows. X, iv and a matrix B also share the asset axis, which is the columns.

The Exogenous Series E holds named series over the observation axis that belong to no asset, for example the Currency Excess Returns that the Currency Factors of a CrossSectionalFactorPrior read. A consumer reads a column of E only by its name, so no model takes the block whole as its factors, as a time-series factor model takes F. E can hold NaN: the consumer that names a series refuses a non-finite value only on the rows that it fits.

The optional AssetPanel pnl holds the two universe masks of the ingestion layer, and the Panel Fields that FeatureDistance turns into a distance. The panel is data, so this type holds it and no estimator does. The clustering code does not know which subset of assets it receives. A Feature Matrix on an estimator would reach a nested clustered subproblem or a cross-validation fold with every asset of the full universe, and its rows would no longer match the assets of the subproblem. ReturnsResult implements port_opt_view, so a view subselects the panel together with X.

Fields

  • nx: Names or identifiers of asset columns (assets × 1).
  • X: Asset returns matrix (observations × assets).
  • nf: Names or identifiers of factor columns (factors × 1).
  • F: Factor returns matrix (observations × factors).
  • nb: Names or identifiers of benchmark columns (benchmarks × 1). A vector B takes at most one name.
  • B: Benchmark returns, (observations × 1) for one benchmark that every asset shares, or (observations × assets) for one benchmark per asset.
  • ne: Names of the Exogenous Series (series × 1). A consumer selects a column of E by one of these names.
  • E: Exogenous Series (observations × series): named series over the observation axis that belong to no asset. NaN marks an absent value.
  • ts: Optional timestamps for each observation (observations × 1).
  • iv: Implied volatilities matrix (observations × assets). NaN marks an absent value.
  • ivpa: Implied volatility risk premium adjustment, a positive number that every asset shares, or a vector of positive numbers with one entry per asset.
  • pnl: Optional AssetPanel that holds the Panel Fields of the universe and its two universe masks.

Constructors

ReturnsResult(;    nx::Option{<:VecStr} = nothing,    X::Option{<:MatNum} = nothing,    nf::Option{<:VecStr} = nothing,    F::Option{<:MatNum} = nothing,    nb::Option{<:VecStr} = nothing,    B::Option{<:VecNum_MatNum} = nothing,    ne::Option{<:VecStr} = nothing,    E::Option{<:MatNum} = nothing,    ts::Option{<:VecDate} = nothing,    iv::Option{<:MatNum} = nothing,    ivpa::Option{<:Num_VecNum} = nothing,    pnl::Option{<:AssetPanel} = nothing,) -> ReturnsResult

Keywords correspond to the struct's fields.

ReturnsResult is an entry point of integer data. An integer element type of X, F, B, E or iv converts once to its floating-point type through float_if_integer, so every kernel downstream reads floating-point data. Every other element type stays, and a view stays a view.

Validation

  • If nx or X is not nothing, check_names_and_returns_matrix holds for the pair: both are given, allunique(nx), !isempty(nx), !isempty(X), and length(nx) == size(X, 2).
  • If nf or F is not nothing, the same checks hold for nf and F.
  • If X and F are given, size(X, 1) == size(F, 1).
  • If B is a matrix, the same checks hold for nb and B.
  • If B is a vector and nb is given, length(nb) == 1. A vector B needs no name.
  • If B is nothing, nb is nothing.
  • If X and a vector B are given, size(X, 1) == size(B, 1). If X and a matrix B are given, size(X) == size(B).
  • If ne or E is not nothing, check_exogenous_series holds: the checks of nx and X hold for ne and E, and E has the row count of each of X, F and ts that is given. E can hold NaN.
  • If ts is not nothing, !isempty(ts), X or F is given, and allunique(ts). length(ts) equals the row count of each of X, F and B that is given, and of E, which check_exogenous_series checks. ts must be unique because it keys the observation axis. feature_row_indices finds the rows of a subset by matching its timestamps back into this clock, and a repeated timestamp resolves to its first occurrence, which pairs an asset with the features of another period.
  • If iv is not nothing, X is given, !isempty(iv), size(iv) == size(X), and every present value is finite and non-negative. NaN marks an absent value. See assert_nonneg_where_present.
  • ivpa is checked only when iv is given, because with no implied volatility it adjusts nothing. Then all(x -> x > 0, ivpa) and all(isfinite, ivpa), and a vector ivpa has length(ivpa) == size(iv, 2). The bound is strict, so a zero adjustment is refused.
  • The asset axis of pnl is length(nx). The observation axis of a time-varying pnl is size(X, 1). See check_asset_panel.

A missing partner raises an IsNothingError, an empty input an IsEmptyError, a repeated name or timestamp an ArgumentError, a size that does not agree a DimensionMismatch, and a value outside its domain a DomainError.

Examples

julia> ReturnsResult(; nx = ["A", "B"], X = [0.1 0.2; 0.3 0.4])ReturnsResult    nx ┼ Vector{String}: ["A", "B"]     X ┼ 2×2 Matrix{Float64}    nf ┼ nothing     F ┼ nothing    nb ┼ nothing     B ┼ nothing    ne ┼ nothing     E ┼ nothing    ts ┼ nothing    iv ┼ nothing  ivpa ┼ nothing   pnl ┴ nothing

Related

source

Functions

PortfolioOptimisers.asset_panel — Method
asset_panel(ape::Nothing, pr, rd::ReturnsResult, X) -> AssetPanel
asset_panel(ape::Nothing, pr::ReturnsResult, rd::Nothing, X) -> AssetPanel
asset_panel(ape::Nothing, pr, rd::Nothing, X) -> Union{}

Resolve the AssetPanel a FeatureDistance with no producer measures.

nothing in the ape slot tells the kernel to read the panel that the ReturnsResult holds. The data reaches the kernel as the two keywords pr and rd, and this function selects the source by dispatch. A ReturnsResult in either slot gives its pnl. When both slots hold one, rd wins, because rd is the returns data. Pr_RR admits a ReturnsResult in the pr slot, and clusterise(cle, rd) and every Pipeline step pass it there. So the second method serves the shortest public call.

A prior result alone carries no panel, so the call raises an IsNothingError that names the two ways to supply one.

Algorithm

The method that Julia selects is the algorithm.

  1. rd is a ReturnsResult: answer rd.pnl.
  2. pr is a ReturnsResult and there is no rd: answer pr.pnl.
  3. Neither slot holds a ReturnsResult: raise.

Each of the first two checks that the ReturnsResult it read holds a panel, with assert_asset_panel_supplied.

Arguments

  • ape: nothing, which reads the panel of the ReturnsResult.
  • pr: Prior result or returns result. Both carry the asset returns matrix X, so either can supply it.
  • rd: The returns result to use.
  • X: Returns matrix of the subproblem. Unread here; a producer reads it.

Validation

Returns

  • pnl::AssetPanel: The Asset Panel that the ReturnsResult holds.

Related

source
PortfolioOptimisers.port_opt_view — Method
port_opt_view(rd::ReturnsResult, i) -> ReturnsResult

Return a view of the ReturnsResult object for the assets at indices i.

This is the port_opt_view method for ReturnsResult. It restricts the returns data to a subset of assets.

Warning

This two-argument method indexes assets, as every other port_opt_view method does except those of PricesResult, which index observations first. The four-argument method port_opt_view(rd, i, j, k) indexes observations first and assets second. So i means a different axis in each of the two methods. See port_opt_view(rd::ReturnsResult, i, j, k).

Algorithm

  1. Resolve the asset names of i to positions in nx with label_positions. Positions, a range and a Colon pass through.
  2. View the asset names nx at i with nothing_scalar_array_view.
  3. View the asset returns as view(rd.X, :, i). Axis 2 is the assets, and every observation is kept.
  4. When B is a matrix, it holds one column per asset, so view nb at i and view B as view(rd.B, :, i). When B is a vector or nothing, nb and B pass through unchanged.
  5. View the implied volatilities as view(rd.iv, :, i), and the adjustment ivpa at i.
  6. View the AssetPanel pnl with port_opt_view at i on the asset axis, and give it the asset names rd.nx. The observation index is a Colon, so a time-varying panel keeps every observation. The view slices the values of every Panel Field and both universe masks on the asset axis. It also slices the label axis of a tensor Panel Field whose labels are the asset names, see features_are_assets. The label axis of every other field holds features, and an asset view does not change it.
  7. Rebuild the ReturnsResult. The factor names nf, the factor returns F, the Exogenous Series ne and E, and the timestamps ts pass through unchanged, because none of them has an asset axis.

Each field that is nothing stays nothing. No step copies data.

Arguments

  • rd: A ReturnsResult object containing asset and/or factor returns.
  • i: The assets to view: one integer, integer positions, a range, a Colon, one asset name, or a vector of asset names. One integer and one name select one asset, and the view keeps its matrix shape.

Validation

  • Every asset name of i is in nx, and nx is not nothing. Raises an ArgumentError. See label_positions.
  • i is not a vector of timestamps, because this method selects assets. Raises an ArgumentError.

Returns

  • new_rr::ReturnsResult: A new ReturnsResult containing only the data for the specified index.

Examples

julia> rd = ReturnsResult(; nx = ["A", "B"], X = [0.1 0.2; 0.3 0.4])ReturnsResult    nx ┼ Vector{String}: ["A", "B"]     X ┼ 2×2 Matrix{Float64}    nf ┼ nothing     F ┼ nothing    nb ┼ nothing     B ┼ nothing    ne ┼ nothing     E ┼ nothing    ts ┼ nothing    iv ┼ nothing  ivpa ┼ nothing   pnl ┴ nothingjulia> PortfolioOptimisers.port_opt_view(rd, 2:2)ReturnsResult    nx ┼ SubArray{String, 1, Vector{String}, Tuple{UnitRange{Int64}}, true}: ["B"]     X ┼ 2×1 SubArray{Float64, 2, Matrix{Float64}, Tuple{Base.Slice{Base.OneTo{Int64}}, UnitRange{Int64}}, true}    nf ┼ nothing     F ┼ nothing    nb ┼ nothing     B ┼ nothing    ne ┼ nothing     E ┼ nothing    ts ┼ nothing    iv ┼ nothing  ivpa ┼ nothing   pnl ┴ nothingjulia> PortfolioOptimisers.port_opt_view(rd, ["B"]).X == [0.2; 0.4;;]truejulia> PortfolioOptimisers.port_opt_view(rd, ["C"])ERROR: ArgumentError: the view selects the assets by name, and 1 name(s) are not among the 2 asset names of the input data: C. A view refuses a name that it cannot find, because a dropped name gives a smaller universe with no error.[...]

Related

Return a view of the ReturnsResult object for assets at indices j, observations at indices i, and factors at indices k.

Warning

The first index of this method selects observations, not assets, and the second index selects assets. Every other port_opt_view method, port_opt_view(rd::ReturnsResult, i) too, takes the assets first. The methods of PricesResult are the exception, and they take the observations first, as this method does. Cross-validation splits observations and assets together, and this method exists for it.

Algorithm

  1. Resolve the timestamps of i to positions in ts with timestamp_positions, and the names of j and k to positions in nx and nf with label_positions. Positions, a range and a Colon pass through.
  2. View the asset names nx at j with nothing_scalar_array_view.
  3. View the asset returns as view(rd.X, i, j). Axis 1 is the observations, and axis 2 is the assets.
  4. View the factor names nf at k. When k is a Colon, nf passes through. View the factor returns as view(rd.F, i, k).
  5. When B is a matrix, it holds one column per asset, so view nb at j and view B as view(rd.B, i, j). When B is a vector, every asset shares it, so view it as view(rd.B, i) and pass nb through.
  6. View the Exogenous Series as view(rd.E, i, :), and pass its names ne through. View the timestamps ts at i, the implied volatilities as view(rd.iv, i, j), and the adjustment ivpa at j.
  7. View the AssetPanel pnl with port_opt_view at the observations i and the assets j, and give it the asset names rd.nx. The view slices both axes of every Panel Field and of both universe masks. It also slices the label axis of a tensor Panel Field whose labels are the asset names, see features_are_assets. A static panel has no observation axis and ignores i, as a scalar ivpa ignores j.
  8. Rebuild the ReturnsResult.

Each field that is nothing stays nothing. No step copies data.

Arguments

  • rd: A ReturnsResult object containing asset and/or factor returns.
  • i: The observations to view: one integer, integer positions, a range, a Colon, one timestamp, or a vector of timestamps. A timestamp of a vector that ts does not hold selects no row, and the rows keep their clock order, as in the timestamp view of a PricesResult. One integer and one timestamp select one observation, and the view keeps its matrix shape.
  • j: The assets to view: one integer, integer positions, a range, a Colon, one asset name, or a vector of asset names. One integer and one name select one asset, and the view keeps its matrix shape.
  • k: The factors to view: one integer, integer positions, a range, a Colon, one factor name, or a vector of factor names. One integer and one name select one factor, and the view keeps its matrix shape.

Validation

  • ts is not nothing when i holds timestamps, and i holds no names. Raises an ArgumentError. See timestamp_positions.
  • The timestamps of i compare with ts, and ts holds i when i is one timestamp. Raises an ArgumentError. See assert_timestamp_type.
  • Every name of j is in nx, and every name of k is in nf. Raises an ArgumentError. See label_positions.
  • A window that keeps no observation or no asset raises the IsEmptyError of the ReturnsResult constructor.

Returns

  • new_rr::ReturnsResult: A new ReturnsResult containing only the data for the specified indices.

Examples

julia> rd = ReturnsResult(; nx = ["A", "B"], X = [0.1 0.2; 0.3 0.4; 0.5 0.6], nf = ["F1"],                          F = [1.0; 2.0; 3.0;;])ReturnsResult    nx ┼ Vector{String}: ["A", "B"]     X ┼ 3×2 Matrix{Float64}    nf ┼ Vector{String}: ["F1"]     F ┼ 3×1 Matrix{Float64}    nb ┼ nothing     B ┼ nothing    ne ┼ nothing     E ┼ nothing    ts ┼ nothing    iv ┼ nothing  ivpa ┼ nothing   pnl ┴ nothingjulia> PortfolioOptimisers.port_opt_view(rd, 1:2, 2:2)ReturnsResult    nx ┼ SubArray{String, 1, Vector{String}, Tuple{UnitRange{Int64}}, true}: ["B"]     X ┼ 2×1 SubArray{Float64, 2, Matrix{Float64}, Tuple{UnitRange{Int64}, UnitRange{Int64}}, false}    nf ┼ Vector{String}: ["F1"]     F ┼ 2×1 SubArray{Float64, 2, Matrix{Float64}, Tuple{UnitRange{Int64}, Base.Slice{Base.OneTo{Int64}}}, false}    nb ┼ nothing     B ┼ nothing    ne ┼ nothing     E ┼ nothing    ts ┼ nothing    iv ┼ nothing  ivpa ┼ nothing   pnl ┴ nothing

The observations can be selected by timestamp, and the assets and the factors by name. A calendar range keeps the observations that the returns hold.

julia> using Datesjulia> rd = ReturnsResult(; nx = ["A", "B"], X = [0.1 0.2; 0.3 0.4; 0.5 0.6],                          ts = [Date(2024, 1, 2), Date(2024, 1, 3), Date(2024, 1, 5)]);julia> sub = PortfolioOptimisers.port_opt_view(rd, Date(2024, 1, 3):Day(1):Date(2024, 1, 6), ["B"]);julia> sub.ts2-element view(::Vector{Date}, [2, 3]) with eltype Date: 2024-01-03 2024-01-05julia> sub.X == [0.4; 0.6;;]true

Related

Fallback that throws for an AbstractReturnsResult subtype that does not implement port_opt_view.

Without it, the call reaches the generic fallback port_opt_view(x, i, args...), which throws a MethodError for nothing_scalar_array_view, a function that the author of the subtype never called. This method names the missing method instead. Returns data always has an asset axis, so no subtype can pass through a view unchanged.

A subtype that carries an AssetPanel subselects it as it subselects X. It subselects the asset axis in every method, and the observation axis in the methods that take one. When the labels of a tensor Panel Field are the assets, see features_are_assets, it subselects the label axis of that field too. A panel that a fold does not subselect gives a finite distance over the wrong universe, and no check catches it. The ReturnsResult methods are the model to follow.

Algorithm

  1. Throw an ArgumentError that names the concrete type and the number of index arguments of the call. The method reads neither the indices nor the fields of rd.

Related

Fallback that throws for a ReturnsResult call whose shape matches no supported method.

ReturnsResult implements port_opt_view, so the AbstractReturnsResult fallback above would report a wrong call as a missing method. This method takes the call and states its shape. The supported methods take one, two or three positional index arguments and no keyword argument. factors is the third positional index, not a keyword.

Algorithm

  1. Count the positional arguments args, and read the names of the keyword arguments kwargs.
  2. Throw an ArgumentError reporting both counts, and naming the three supported call shapes.

Related

source
PortfolioOptimisers.benchmark_relative_returns — Function
benchmark_relative_returns(rd::ReturnsResult, x_src::Option{<:DataSource}) -> Option{<:MatNum}
benchmark_relative_returns(rd::ReturnsResult, x_src::ExcessSource) -> Option{<:MatNum}
benchmark_relative_returns(rd::ReturnsResult, x_src::PriorSource)

Return the asset returns of rd that a reader with the source x_src fits on.

Every reader of the returns calls this function at the point of use with its own x_src: the prior, the factor regression of an exposure constraint or of a factor risk contribution, a clustering, phylogeny or centrality estimator that reads the data, and the step of OnlinePortfolioSelection. The function returns a matrix and never builds a ReturnsResult, so rd keeps its raw X and its B, and every other reader sees the returns data as the caller gave it.

ExcessSource on returns that carry no benchmark is a refusal and not a pass-through, because no excess return exists without a benchmark. PriorSource reads the Prior Result, which rd does not hold, so a reader at PriorSource() picks its matrix before it reaches this function.

Mathematical definition

\[\begin{align} \tilde{x}_{t,\,i} &= \begin{cases} x_{t,\,i} - b_{t} & \text{a vector benchmark}\,,\\ x_{t,\,i} - b_{t,\,i} & \text{a matrix benchmark}\,. \end{cases} \end{align}\]

Where:

  • $\tilde{x}_{t,\,i}$: Excess return of asset $i$ at observation $t$.
  • $x_{t,\,i}$: Return of asset $i$ at observation $t$, rd.X.
  • $b_{t}$, $b_{t,\,i}$: Benchmark return at observation $t$, rd.B. A vector benchmark has one value for each observation, and a matrix benchmark has one value for each observation and asset.

A gap in the benchmark gives a NaN excess return.

Algorithm

The method that Julia selects is the algorithm.

  1. x_src is nothing or DataSource. Return rd.X.
  2. x_src is ExcessSource and rd holds no returns, because its X field is nothing. Return nothing, because there is nothing to subtract from.
  3. x_src is ExcessSource and rd carries no benchmark, because its B field is nothing. Throw an ArgumentError.
  4. x_src is ExcessSource. Subtract the benchmark from the asset returns. A vector benchmark subtracts by broadcast, rd.X .- rd.B. A matrix benchmark subtracts elementwise, rd.X - rd.B. The function does not modify rd.
  5. x_src is PriorSource. Throw an ArgumentError.

Arguments

  • rd: A ReturnsResult that holds the asset returns, and optionally the benchmark returns. When it carries a benchmark, it also holds X.
  • x_src: The source of the returns that the fit reads, nothing or an AbstractReturnsSource. Only ExcessSource reads the benchmark: it subtracts B from the asset returns. An optimiser passes the source of its group, see group_source.

Validation

  • x_src is not ExcessSource, or rd carries a benchmark, or rd holds no returns. An ArgumentError is thrown otherwise.
  • x_src is not PriorSource. An ArgumentError is thrown otherwise.
  • When rd carries a benchmark, rd.X is a matrix. Otherwise no method matches, and the call raises a MethodError. A value of x_src that is not nothing and not an AbstractReturnsSource raises a MethodError.

Returns

  • X::Option{<:MatNum}: The excess returns when x_src is ExcessSource, a new matrix. Otherwise rd.X itself, which is nothing when rd holds no returns.

Examples

julia> rd = ReturnsResult(; nx = ["A", "B"], X = [0.10 0.20; 0.30 0.40], nb = ["BM"],                          B = [0.01; 0.02]);julia> benchmark_relative_returns(rd, DataSource()) === rd.X  # no change for the raw returnstruejulia> benchmark_relative_returns(rd, ExcessSource()) == rd.X .- rd.Btruejulia> rd.B  # rd keeps its benchmark2-element Vector{Float64}: 0.01 0.02

Related

source