Prices to returns

Types

PortfolioOptimisers.AbstractGapReturnAlgorithm — Type
abstract type AbstractGapReturnAlgorithm <: AbstractAlgorithm

Supertype of the policies that write a return into the cells that a price gap left non-finite.

A return is the change between two consecutive observations. So a run of k gapped prices inside the series leaves k + 1 non-finite returns, and a run at either end leaves k. Under the default rule no return records the move across the gap, and nothing on prices_to_returns selects that rule. A caller who wants the move booked gives one of these algorithms instead.

An algorithm writes only a cell that reads a gapped price, inside the asset's Listing Span, after the first observed price of its column. gap_return_writable finds that set, and apply_gap_return restores every other cell. So a return computed from two observed prices keeps its value whatever the algorithm returns. A gap changes no cell that reads none of its prices, and no return appears before an asset's first price.

Interfaces

In order to implement a new concrete type that works seamlessly with the library, subtype AbstractGapReturnAlgorithm and implement the following methods:

gap_return

  • gap_return(alg::AbstractGapReturnAlgorithm, p::AbstractVector, r::AbstractVector, ret_method::Symbol) -> Vector: One column's returns, with the writable cells resolved.

Arguments

  • alg: The concrete subtype instance.
  • p: One column's prices along the observation axis, gaps included.
  • r: The returns that the default rule computed from p, so length(p) - length(r) is 0 under padding and 1 otherwise.
  • ret_method: :simple or :log. Compute a value with gap_return_value, which holds the two branches.

Returns

  • out::Vector: The same length as r. apply_gap_return reads back only the cells that gap_return_writable admits. So a method can return every other cell unchanged, and it does not need to check the rule itself.

Related

source
PortfolioOptimisers.CatchUpGapReturn — Type
struct CatchUpGapReturn <: AbstractGapReturnAlgorithm

Books the whole move across a Held Gap on the observation that ends it, so the gap holds k non-finite returns.

A suspension of k observations leaves k + 1 non-finite returns under the default rule. This algorithm writes $p_{t+k,\,i} / p_{t-1,\,i} - 1$ on the observation where the asset trades again, and it leaves the k observations inside the gap non-finite. The compounded return across the gap then equals the price ratio, and the Held Gap is the run of unpriced observations. The algorithm writes nothing at an asset's inception, because no earlier observed price exists there.

The estimation mask reads the values and not the algorithm that wrote them. So the mask marks the cell where the asset trades again as estimable, and a moment estimator reads a (k + 1)-period return as one draw of a one-period return. Under independent increments, the scale of that draw is about $\sqrt{k + 1}$ times the scale of a one-period return.

Constructors

CatchUpGapReturn() -> CatchUpGapReturn

Examples

julia> CatchUpGapReturn()CatchUpGapReturn()

Related

source
PortfolioOptimisers.PricesToReturns — Type
struct PricesToReturns{__T_ret_method, __T_padding, __T_gap_return_alg, __T_cache} <: AbstractPreprocessingEstimator

Preprocessing estimator that converts price-level data into returns-level data.

PricesToReturns is the estimator form of prices_to_returns. It reads a PricesResult and returns a ReturnsResult. It has no fitted state, so its application to a window runs the conversion, and its fitted object is the estimator itself.

Its first three fields are the three keywords of prices_to_returns, which are the keywords that change the arithmetic of a return. A join and a collapse change the observation clock, and PriceIngestion owns them. PriceGapFill fills a gap and MissingDataFilter deletes one, and both are fitted steps. Every datum that the conversion reads is a field of the PricesResult.

The step needs no state to fix an asset universe, because the PricesResult states one. A PricesResult that price_ingestion built carries a Listing Span. This step projects the span onto the returns clock, and gives the ReturnsResult an AssetPanel with two masks. At each observation, the masks state which assets are in the universe, and which of them are estimable. The asset axis is fixed before the split, so every window of every fold carries every asset, and no window loses a column.

Warning

A PricesResult that the ingestion layer did not build states no universe, and the conversion does not infer one from the window. A span computed from the window alone reads a delisting that crosses the end of the window as an asset that was never listed. The conversion still carries the gaps of such a PricesResult. With no panel, the Coverage Universe reads finiteness alone, and the fold infers the universe that a span would state. Build the PricesResult with price_ingestion, or give a listing calendar as its span.

Algorithm

The estimator has no fitted state, so both verbs are short.

  1. fit_preprocessing returns the estimator itself. There is no state to fit.
  2. apply_preprocessing calls prices_to_returns with the whole PricesResult and the three fields as keywords, and returns the ReturnsResult.

Every row and every column of the window reaches the conversion, because the conversion cannot drop one. gap_return_alg is a field because it sets the returns that a gap leaves non-finite. That is the arithmetic of a return, and not a policy about the universe.

Fields

  • ret_method: The return formula, :simple or :log.
  • padding: When true, keep the first observation with a NaN return, so the returns keep the length of the price clock. When false, drop it.
  • gap_return_alg: The rule for the returns that a price gap leaves non-finite. nothing is the default rule, and CatchUpGapReturn books the move across the gap on the observation that ends it. See AbstractGapReturnAlgorithm.
  • cache: Optional partial-fit state. It is nothing until partial_fit! writes one. When the caller gives no data matrix, the batch verb of the estimator reads it. Each propagation channel does one thing with it. factory keeps it unchanged, because a factory call resolves the configuration, not the sample. port_opt_view copies the entries of the selected assets into a new state, so the viewed estimator holds the state of those assets alone. obs_weights_view removes it, because a state has no slice on the observation axis. Only a type with an observation-weights field has that channel. A type with no such field falls through to the identity method, so it keeps its state. A family whose state has no exact asset slice removes it on both axes and names the reason.

Constructors

PricesToReturns(;    ret_method::Symbol = :simple,    padding::Bool = false,    gap_return_alg::Option{<:AbstractGapReturnAlgorithm} = nothing,    cache::Option{<:AbstractPartialFitState} = nothing) -> PricesToReturns

Keywords correspond to the struct's fields.

Validation

  • ret_method in (:simple, :log). The constructor throws an ArgumentError otherwise.

Online form

A return reads two consecutive prices. So partial_fit_transform converts a block of prices as the whole history does, because it keeps the last price row in cache. Under a CatchUpGapReturn, it also keeps the last observed price of every series column. fit_preprocessing with no data returns the estimator with an empty cache. A caller's own Gap Return algorithm has no online form, and supports_partial_fit returns false for it.

Examples

julia> X = TimeArray(Date(2020, 1, 1):Day(1):Date(2020, 1, 3),                     [100.0 101.0; 102.0 103.0; 104.0 105.0], ["A", "B"]);julia> pr = PricesResult(; X = X);julia> rr = apply_preprocessing(PricesToReturns(), pr);julia> size(rr.X)(2, 2)julia> rr.nx2-element Vector{String}: "A" "B"

Related

source

Functions

PortfolioOptimisers.prices_to_returns — Function
prices_to_returns(
    pr::PricesResult;
    ret_method::Symbol = :simple,
    padding::Bool = false,
    gap_return_alg::Option{<:AbstractGapReturnAlgorithm} = nothing
) -> ReturnsResult
prices_to_returns(
    X::TimeSeries.TimeArray;
    kwargs...
) -> ReturnsResult

Compute returns from the price data.

A keyword belongs to this function only when it changes the arithmetic of a return, and three keywords do. Every datum that the conversion reads is a field of the PricesResult. These are the asset prices, the factors, the benchmark, the implied volatilities, the Listing Span and the AssetPanel. A keyword for one of them would state again what the PricesResult states.

The second method takes a bare price table. It runs price_ingestion with a default PriceIngestion, and converts the PricesResult that it returns. For a different join, a collapse, a declared span, or factor, benchmark and implied-volatility series, call the two steps.

An absent price is a NaN. The conversion carries it into the returns, and it does not delete the observation or the asset. PriceGapFill fills a gap and MissingDataFilter deletes one. Both are fitted steps.

Mathematical definition

The conversion computes the returns from the prices as:

\[\begin{align} x_{t,\,i} &= \begin{cases} (p_{t,\,i} - p_{t-1,\,i}) / p_{t-1,\,i} & \text{simple} \\ \ln(p_{t,\,i} / p_{t-1,\,i}) & \text{log} \end{cases}\,. \end{align}\]

Where:

  • $x_{t,\,i}$: Return of asset $i$ at observation $t$.
  • $p_{t,\,i}$: Price of asset $i$ at observation $t$ of the price clock, absent at a gap.

A price is never negative, so the conversion refuses a negative price. A zero price $p_{t,\,i} = 0$ gives $x_{t,\,i} = -1$ on the simple branch and $x_{t,\,i} = -\infty$ on the log branch. On both branches it gives $x_{t+1,\,i} = \infty$.

The conversion applies the same rule to a benchmark $B$, and carries the benchmark returns $b_{t,\,i}$ beside the asset returns. It does not subtract them. benchmark_relative_returns forms the excess return $\tilde{x}_{t,\,i} = x_{t,\,i} - b_{t,\,i}$, and only when the optimisation tracks the benchmark.

Algorithm

  1. Check with assert_distinct_series_names that the asset, factor, benchmark and Exogenous Series have distinct names. Read the asset names and the asset timestamps from pr.X. Check pr.pnl against them with check_asset_panel, and pr.span with assert_span_shape.
  2. Write the three price blocks and the Exogenous Series side by side on the clock of the PricesResult with append_price_block!, which writes every absent price as NaN with unify_gaps. The PricesResult states one clock. So the function reads a factor or benchmark series at the asset timestamps, and does not join it onto them. It refuses a series on a different clock and names the series, because a join adds or drops observations and price_ingestion owns every change of the clock. A benchmark is one shared column, or one column per asset.
  3. Check that every present price is non-negative. Then convert the prices to returns under ret_method and padding. This step applies the formula above. The simple return is $(p_{t,\,i} - p_{t-1,\,i}) / p_{t-1,\,i}$. The difference is exact when the two prices are within a factor of two of each other, so the division is then the one rounding. The log return is log1p of the simple return, which adds about one ulp to that rounding. The difference of logarithms $\ln p_{t,\,i} - \ln p_{t-1,\,i}$ cancels when the two prices are close: each logarithm carries an absolute error of about $\varepsilon \lvert \ln p \rvert$, so its error grows with the price level and not with the return. When padding is true, the step keeps the first observation with a NaN return, so the returns keep the length of the price clock.
  4. A gap carried here does not spread. The formula reads two prices, so a run of k gapped prices makes non-finite only the returns that read one of them. That is k + 1 returns for a run inside the series, and k for a run at either end, because no return reads a price before the first row or after the last. Every later return of that column reads two observed prices. A gap also stays in its own column, because the return of an asset reads no price of another asset.
  5. Resolve the returns that a gap left non-finite with apply_gap_return, under gap_return_alg. The method for nothing is the default rule. It returns the table unchanged, so the returns of step 3 stay bit for bit the same. An algorithm writes only a return that reads a gapped price, inside the Listing Span of its column, after the first observed price. So every return computed from two observed prices keeps its value. The function logs an @info when it finds no writable return.
  6. Name the four blocks. Step 1 refused a name that two tables share and the name timestamp. So the asset names nx, the factor names nf, the benchmark names nb and the Exogenous Series names ne are the column names of the four tables, and ts is the timestamp column of the converted table.
  7. Write each absent implied volatility as NaN with unify_gaps, and index pr.iv by ts. Then check the implied volatilities and pr.ivpa against the asset count. The returns clock is the price clock, less the first observation when padding is false. So the implied volatilities of a PricesResult that the ingestion layer built cover it. The conversion carries an absent implied volatility as NaN, and the estimator that reads it excludes it.
  8. View the AssetPanel on the returns clock. Find the panel rows of the returns timestamps with feature_row_indices, and view the panel with port_opt_view. Give it the asset names, so that it also cuts a square tensor Panel Field on its label axis. The conversion removes no column, so the view keeps every asset. A time-varying panel loses only the observations that the returns clock does not hold.
  9. State the universe. Cut pr.span to the asset axis with listing_span_view. Give it and the asset returns to returns_universe_masks, which projects the span onto the returns clock and intersects it with finiteness. attach_universe_masks puts the two masks on the Asset Panel and keeps its Panel Fields. When the PricesResult holds no Panel Field, it makes a panel with pf = nothing, or gives nothing when both masks are all true, because those masks state nothing that the finiteness of the returns does not state. A PricesResult with no span states no universe, so the function attaches no masks, and the panel is the one of step 8, or nothing.
  10. Build the factor and benchmark matrices from the columns of each block. A block with no column gives nothing, and a benchmark block with one column gives a vector. The asset matrix is always present, because the conversion removes no column.
  11. Build the Exogenous Series from its columns, or nothing when pr.E is nothing. It is converted with the assets, so it takes the same ret_method, the same padding and the same Gap Return rule, and a gap in it stays NaN.
  12. Return the ReturnsResult.

The conversion removes no observation and no asset. A deletion is a Universe Policy. A policy is fit on a training window and applied again by name, and a stateless conversion cannot do that. MissingDataFilter owns the deletion. Its col_thr deletes an asset and its row_thr deletes an observation.

Arguments

  • pr: The price data, as price_ingestion returns it or a caller builds it.
  • X: Asset price data (observations × assets), converted through a default PriceIngestion.
  • ret_method: The return formula, :simple or :log.
  • padding: When true, keep the first observation with a NaN return, so the returns keep the length of the price clock. When false, drop it.
  • gap_return_alg: The rule for the returns that a price gap leaves non-finite. nothing is the default rule, under which no return records the move across the gap. A run of k gapped prices inside the series then leaves k + 1 non-finite returns, and a run at either end leaves k. CatchUpGapReturn books the move on the observation where the asset trades again, so the Held Gap holds k non-finite returns. An algorithm writes only a return that reads a gapped price, inside the Listing Span of its column, after the first observed price. So a return computed from two observed prices keeps its value under every algorithm. When no return is writable, the function logs an @info.

Validation

  • ret_method is :simple or :log. The function throws an ArgumentError otherwise.
  • Every present price that reaches step 3 is non-negative, on both branches. The function throws a DomainError that names the columns otherwise. An absent price is NaN, and the check skips it. A zero price gives the returns that the mathematical definition states.
  • The asset, factor, benchmark and Exogenous Series column names are pairwise disjoint, and none of them is timestamp. The function throws a ConflictingArgumentError that names the columns otherwise.
  • If pr.F, pr.B or pr.E is not nothing, its timestamps equal the asset timestamps. The function throws a ConflictingArgumentError otherwise. The message names price_ingestion, which puts two series on one clock.
  • If pr.span is not nothing, size(pr.span) == size(values(pr.X)). The function throws a DimensionMismatch otherwise.
  • If pr.iv is not nothing, the returns timestamps are a subset of TimeSeries.timestamp(pr.iv). The function throws an ArgumentError otherwise. Then, with iv = values(unify_gaps(pr.iv)[ts]), !isempty(iv) holds, size(iv) == size(X) holds, and every present value is finite and non-negative. An absent value is NaN, and assert_nonneg_where_present skips it.
  • The function checks pr.ivpa in the same branch, so only when pr.iv is given. all(x -> x > 0, ivpa) and all(x -> isfinite(x), ivpa) hold, and a vector ivpa has length(ivpa) == size(iv, 2). The bound is strict, so the function refuses a zero adjustment.

Returns

  • rr::ReturnsResult: The asset, factor and benchmark returns, the Exogenous Series, their names, the returns timestamps, the implied volatilities and the Asset Panel.

Examples

julia> X = TimeArray(Date(2020, 1, 1):Day(1):Date(2020, 1, 3), [100 101; 102 103; 104 105],                     ["A", "B"])3×2 TimeSeries.TimeArray{Int64, 2, Dates.Date, Matrix{Int64}} 2020-01-01 to 2020-01-03┌────────────┬─────┬─────┐│            │ A   │ B   │├────────────┼─────┼─────┤│ 2020-01-01 │ 100 │ 101 ││ 2020-01-02 │ 102 │ 103 ││ 2020-01-03 │ 104 │ 105 │└────────────┴─────┴─────┘julia> prices_to_returns(X)ReturnsResult    nx ┼ Vector{String}: ["A", "B"]     X ┼ 2×2 Matrix{Float64}    nf ┼ nothing     F ┼ nothing    nb ┼ nothing     B ┼ nothing    ne ┼ nothing     E ┼ nothing    ts ┼ Vector{Dates.Date}: [Dates.Date("2020-01-02"), Dates.Date("2020-01-03")]    iv ┼ nothing  ivpa ┼ nothing   pnl ┴ nothing

Related

source
PortfolioOptimisers.gap_return — Function
gap_return(alg::CatchUpGapReturn, p::AbstractVector, r::AbstractVector, ret_method::Symbol) -> Vector

Resolve the writable cells of one column's returns.

Dispatch on alg selects the algorithm. The library ships CatchUpGapReturn alone. The family is a type and not a flag, so another rule costs one type and one method. A rule that spreads the move of a suspension across its observations is an example.

Algorithm

CatchUpGapReturn reads the observations in order, and keeps the row of the last observed price as the anchor.

  1. On a gapped price, write nothing and keep the anchor. The observation is inside the Held Gap, and its return stays non-finite.
  2. On an observed price whose predecessor is also observed, write nothing. The default rule computed that cell from two consecutive prices, and gap_return_writable does not admit it.
  3. On an observed price whose predecessor is gapped, write gap_return_value of this price against the anchor price. This observation ends the gap, and it carries the whole move across the gap.
  4. After step 2 or step 3, move the anchor to the row of this price.

The first observed price of a column has no anchor, so the algorithm writes nothing on it. That is why an inception needs no case of its own.

Arguments

  • alg: The Gap Return algorithm.
  • p: One column's prices along the observation axis, gaps included.
  • r: The returns that the default rule computed from p.
  • ret_method: :simple or :log.

Returns

  • out::Vector: The same length as r, with the writable cells resolved.

Examples

julia> PortfolioOptimisers.gap_return(CatchUpGapReturn(), [100.0, NaN, NaN, 110.0],                                      [NaN, NaN, NaN], :simple)3-element Vector{Float64}: NaN NaN   0.1

Related

source
PortfolioOptimisers.currency_excess_index — Function
currency_excess_index(fx::TimeSeries.TimeArray, cash::TimeSeries.TimeArray,
                      cash_base::TimeSeries.TimeArray) -> TimeSeries.TimeArray

Build the Currency Excess Index of each currency, a level series for the Exogenous Series of a PricesResult.

The index of a currency is the value in the base currency of a deposit in that currency, funded in the base currency. Its return is the Currency Excess Return: the exchange-rate return and the cash return of the currency, less the cash return of the base currency. prices_to_returns converts the index with the return rule of the assets, so the Currency Factors of a CrossSectionalFactorPrior read returns on the same footing as the asset returns.

Mathematical definition

\[\begin{align} I_{t,\,c} &= S_{t,\,c} \, \frac{K_{t,\,c}}{K_{t,\,b}}\,, \\ \frac{I_{t,\,c}}{I_{t - 1,\,c}} - 1 &= \frac{(1 + R^{S}_{t,\,c})(1 + r_{t,\,c})}{1 + r_{t,\,b}} - 1\,. \end{align}\]

Where:

  • $I_{t,\,c}$: Currency Excess Index of the currency $c$ at observation $t$.
  • $S_{t,\,c}$: Exchange rate of $c$, in units of the base currency per unit of $c$.
  • $K_{t,\,c}$, $K_{t,\,b}$: Cash total-return indices of $c$ and of the base currency $b$, which accumulate the cash rate of each period.
  • $R^{S}_{t,\,c}$: Return of the exchange rate over observation $t$.
  • $r_{t,\,c}$, $r_{t,\,b}$: Cash rates of $c$ and of $b$ over observation $t$.

With log returns the split of an asset return in the base currency is exact. An asset of currency $c$ with local return $R^{L}$ has the base-currency return $(1 + R^{L})(1 + R^{S}) - 1$, so

\[\begin{align} \ln(1 + R^{L}) - \ln(1 + r_{c}) + \ln\frac{I_{t,\,c}}{I_{t - 1,\,c}} &= \ln(1 + R^{B}) - \ln(1 + r_{b})\,, \end{align}\]

and the local excess return plus the log return of the index is the base-currency excess return. With simple returns the split is not exact. The base-currency excess return less the local excess return and the return of the index is $R^{L} R^{S} + (r_{b} - r_{c})(R^{S} - r_{b}) / (1 + r_{b})$. The first term is the cross term of each asset. The second term depends on the currency alone, and it is the product of the difference of the two cash rates with the exchange-rate return less the base cash rate. No series with one column per currency can hold the cross term. Under the local returns a CrossSectionalFactorPrior derives, the term stays in the local return. Under a Panel Field of local returns the caller names, it is out of the model.

The index of the base currency itself is constant, so its return is zero, and a Currency Factor of it would have no variance. Give the base currency to the base of CurrencyExposure instead: it then takes no factor and needs no index, and an asset of the base currency carries a zero exposure on every Currency Factor.

A gap of a cash index

A cash total-return index has a value at every observation from its first value to its last one, because a deposit accrues interest on every day, a holiday of its market included. A gap inside that span is a value that was not published, and the index holds its last value across it, by the walk of CarriedPrice. So $r_{t,\,c} = 0$ inside the gap, and the observation that ends the gap carries the accrual of the whole gap. The compounded cash return across the gap is the ratio of the two published values, as it is without a gap. On a clock that joins two markets, the index of every currency then has a value on a holiday of the market of one cash index, and the observation keeps the exchange-rate return of that day. A gap before the first value or after the last value stays a gap. An exchange rate has no value between two quotes, so a gap of fx stays a gap, and the index is NaN there.

Algorithm

  1. Check that the three series state one clock, that cash holds a column for every currency of fx, and that cash_base holds one column.
  2. Write every absent value as NaN with unify_gaps.
  3. Hold the last value of each cash index across each gap inside its span, see PortfolioOptimisers.carry_cash_index. The exchange rates keep their gaps.
  4. Read the cash index of each currency of fx by its name, and compute the index of each currency.

Arguments

  • fx: Exchange rates, observations × currencies, in units of the base currency per unit of each currency. Its column names name the currencies.
  • cash: Cash total-return indices, observations × currencies, one column per currency of fx, named alike. Their order is free.
  • cash_base: Cash total-return index of the base currency, observations × 1.

Validation

  • TimeSeries.timestamp(cash) == TimeSeries.timestamp(fx) and TimeSeries.timestamp(cash_base) == TimeSeries.timestamp(fx). Raises a ConflictingArgumentError.
  • cash holds a column for every column of fx. Raises an ArgumentError that names the missing ones.
  • cash_base holds exactly one column. Raises a DimensionMismatch.
  • Every value is missing, NaN or finite. Raises a DomainError, from unify_gaps.

Returns

  • I::TimeSeries.TimeArray: The Currency Excess Index, observations × currencies, named as the columns of fx.

Examples

julia> ts = Date(2020, 1, 1):Day(1):Date(2020, 1, 2);julia> fx = TimeArray(ts, [2.0 0.5; 3.0 0.75], ["EUR", "JPY"]);julia> cash = TimeArray(ts, [1.0 1.0; 1.0 1.25], ["JPY", "EUR"]);julia> base = TimeArray(ts, [1.0; 1.5;;], ["USD"]);julia> values(currency_excess_index(fx, cash, base))2×2 Matrix{Float64}: 2.0  0.5 2.5  0.5

The cash index of the base currency has no value on a holiday of its market. It holds its last value there, and the index keeps the exchange-rate move of that day:

julia> ts = Date(2020, 1, 1):Day(1):Date(2020, 1, 3);julia> fx = TimeArray(ts, [2.0; 2.5; 3.0;;], ["EUR"]);julia> cash = TimeArray(ts, [1.0; 1.0; 1.0;;], ["EUR"]);julia> base = TimeArray(ts, [1.0; NaN; 1.5;;], ["USD"]);julia> values(currency_excess_index(fx, cash, base))3×1 Matrix{Float64}: 2.0 2.5 2.0

Related

source
PortfolioOptimisers.gap_return_value — Function
gap_return_value(
    ret_method::Symbol,
    pt::Number,
    p0::Number
) -> Any

Compute one return from two prices that need not be consecutive.

This function holds the two ret_method branches for the Gap Return family. A new algorithm chooses the two prices, and this function turns them into a return. It uses the arithmetic of step 3 of prices_to_returns, which computes the simple branch as $(p_{t} - p_{0}) / p_{0}$ and the log branch as log1p of the simple return. So a value that an algorithm writes uses the same formula as the cells around it.

Arguments

  • ret_method: :simple or :log.
  • pt: The later price.
  • p0: The earlier price, the return's anchor.

Returns

  • r::Number: $\ln(p_{t} / p_{0})$ under :log, computed as log1p of the simple return, and $(p_{t} - p_{0}) / p_{0}$ otherwise.

Related

source