Base cross-sectional factor Prior: private API

The functions below are the steps of a CrossSectionalFactorPrior fit. They compute these parts of it.

  • The benchmark weights that its exposure estimators read, which the prior computes from the market capitalisation, and the check that each exposure estimator reads them from the field the prior writes.
  • The history of each factor exposure, in an order that puts each exposure after the exposures it is derived from.
  • The active mask and the estimation mask of the asset panel.
  • The regression of each observation on the factors that are not empty, and the moments of the factors, with no mean and no variance for a factor whose exposure is zero at every asset that the fit weights.
  • The factors whose mean and variance the factor prior states, and the assets whose loadings read only those factors. An asset that loads on another factor has no moment, and the fit refuses only when no asset has a moment.
  • The idiosyncratic covariance of the latest observation, and the degrees of freedom and the divisor of each idiosyncratic variance.
  • The moments of the assets, from the moments of the factors and the loadings.

The scenario weights of the factors, the return scenarios of the assets and the forecast of the expected returns are on the page of the Prior Scenarios.

Types

PortfolioOptimisers.ExposureHistory — Type
struct ExposureHistory{__T_Ms, __T_nf, __T_fam, __T_Zo}

The exposure history of a factor list, the factor axis it is written on, and the observed exposures that can share its backing.

This type is an implementation detail and is not intended for direct use. cross_sectional_exposure_history makes it. cross_sectional_exposure_series reads it.

Fields

  • Ms: Ms: The exposure history, assets × factors × observations.
  • nf: nf: Name of each factor.
  • fam: fam: Family label of each factor.
  • Zo: Zo: The observed exposures, as a view of the backing that Ms shares, or the input Zo.

Related

source
PortfolioOptimisers.HistoryBacking — Type
struct HistoryBacking{__T_Ms, __T_Zo}

The allocated exposure history of the estimated factors, and the observed exposures, which share one backing when they can.

This type is an implementation detail and is not intended for direct use. cross_sectional_history_backing makes it. cross_sectional_exposure_history reads it. In the shared case the two fields are views of one array, so they have one type, and a positional swap would be silent.

Fields

  • Ms: Ms: The history of the estimated factors, to be written, assets × factors × observations.
  • Zo: Zo: The observed exposures, or nothing.

Related

source
PortfolioOptimisers.FactorPriorMoments — Type
struct FactorPriorMoments{__T_X, __T_mu, __T_sigma, __T_w, __T_ens, __T_eoc, __T_kld, __T_ow, __T_pc}

The factor moments of a Cross-Sectional Factor Prior on the whole factor axis, with the scenario weights and the diagnostics of its factor prior.

This type is an implementation detail and is not intended for direct use. cross_sectional_factor_moments makes it. cross_sectional_assemble, cross_sectional_calibration, cross_sectional_moment_prior and cross_sectional_kept_cache read it.

Fields

  • X: X: The factor return scenarios on the whole factor axis, observations × factors.
  • mu: mu: The expected factor returns on the whole factor axis.
  • sigma: sigma: The processed factor covariance on the whole factor axis.
  • w: w: The scenario weights divided by their sum, or nothing.
  • ens: ens: The effective number of scenarios of the factor prior.
  • eoc: eoc: The effective counts of the moments of the factor prior.
  • kld: kld: The Kullback-Leibler divergence of the factor prior.
  • ow: ow: The original weights of the factor prior.
  • pc: pc: The cache of an opinion pool in the factor slot, or nothing.

Constructors

FactorPriorMoments(; X, mu, sigma, w, ens, eoc, kld, ow, pc) -> FactorPriorMoments

Keywords correspond to the struct's fields. rebuild_struct calls this form when cross_sectional_kept_cache replaces pc.

Related

source
PortfolioOptimisers.ReducedFactorAxis — Type
struct ReducedFactorAxis{__T_fcb, __T_Ms, __T_nf, __T_fam}

The Factor Family Basis of a Cross-Sectional Factor Prior, and the factor axis that it reduces.

This type is an implementation detail and is not intended for direct use. cross_sectional_family_basis makes it. The regression stage and the carry fold of a CrossSectionalFactorPrior read it.

Fields

  • fcb: fcb: The Factor Family Basis, or nothing when the prior constrains no Factor Family.
  • Ms: Ms: The exposure history on the reduced axis, assets × factors × observations.
  • nf: nf: Name of each reduced factor.
  • fam: fam: Family label of each reduced factor.

Related

source
PortfolioOptimisers.IdiosyncraticBlock — Type
struct IdiosyncraticBlock{__T_D, __T_R}

The idiosyncratic block of the asset covariance of a Cross-Sectional Factor Prior over the investable assets, and a square root of it.

This type is an implementation detail and is not intended for direct use. cross_sectional_residual_block makes it. cross_sectional_lift reads it. The two fields have one type, so a positional swap would be silent.

Fields

  • D: D: The block that the asset covariance adds, investable assets × investable assets.
  • R: R: A square root of the block, investable assets × investable assets.

Related

source
PortfolioOptimisers.LiftedMoments — Type
struct LiftedMoments{__T_mu, __T_sigma, __T_chol}

The asset moments that a Cross-Sectional Factor Prior lifts from its factor distribution, and the low-rank square root of the covariance.

This type is an implementation detail and is not intended for direct use. cross_sectional_lift makes it. cross_sectional_moment_prior reads it. sigma and chol are matrices of one type, so a positional swap would be silent.

Fields

  • mu: mu: Expected asset returns.
  • sigma: sigma: Asset covariance.
  • chol: chol: The low-rank square root of the asset covariance, (factors + investable assets) × assets.

Related

source

Functions

PortfolioOptimisers.cross_sectional_prior_pairs — Function
cross_sectional_prior_pairs(x::Nothing, sym::Sym_Str) -> nothing
cross_sectional_prior_pairs(x::Dict_VecPair, sym::Sym_Str) -> AbstractVector{<:Pair{String}}

Collect one list-valued argument of a CrossSectionalFactorPrior into a vector of Pairs.

The factors, the Neutralisation and the constrained Factor Families of the prior take the same form. Each takes Pairs in the order the caller writes them, or any AbstractDict. A dictionary states no order, so the collected order is its order of iteration. A caller who needs a stated order writes Pairs. A vector of Pairs with String keys is held as it is, not copied, as string_key_pairs states.

The Neutralisation and the constrained Factor Families are each absent or a list, so the absent case is the method on Nothing, which needs no repair and gives nothing. The factors are never absent: the keyword factors::Dict_VecPair of the prior refuses nothing before the call.

Arguments

  • x: The Pairs, the dictionary, or nothing.
  • sym: Name of the field, for the messages.

Validation

  • x is not empty. Raises an IsEmptyError.
  • No key repeats. Raises an ArgumentError.

Returns

  • pr::Option{<:AbstractVector{<:Pair{String}}}: The collected Pairs, each with a String key, or nothing.

Examples

julia> PortfolioOptimisers.cross_sectional_prior_pairs(["style" => "industry"], :neutralise)1-element Vector{Pair{String, String}}: "style" => "industry"

Related

source
PortfolioOptimisers.cross_sectional_benchmark_returns — Function
cross_sectional_benchmark_returns(
    rd::ReturnsResult,
    name::AbstractString,
    W::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}
) -> ReturnsResult

Return the returns data that the Exposure Estimators of a Cross-Sectional Factor Prior are fitted on.

An Exposure Estimator weights its cross-sectional transforms by a benchmark-weight Panel Field that it names. The prior computes those weights from the market capitalisation, and writes them onto a copy of the Asset Panel before it builds any Factor Exposure. A panel that already holds a field of that name is refused rather than overwritten, so a field the caller built is never lost and every member reads the weights of the prior.

Arguments

  • rd: The returns result to use. It must carry an Asset Panel in rd.pnl.
  • name: Name of the benchmark-weight Panel Field.
  • W: The benchmark weights, observations × assets.

Validation

Returns

  • rd::ReturnsResult: The returns data, with the benchmark weights on its Asset Panel.

Related

source
PortfolioOptimisers.assert_cross_sectional_benchmark_field — Function
assert_cross_sectional_benchmark_field(key::AbstractString,
                                       xe::AbstractExposureEstimator,
                                       bw::AbstractString) -> nothing
assert_cross_sectional_benchmark_field(key::AbstractString,
                                       xe::Union{CompositeExposure, DerivedExposure},
                                       bw::AbstractString) -> nothing

Refuse a factor whose Exposure Estimator reads its benchmark weights from a Panel Field the prior does not write.

A Cross-Sectional Factor Prior writes its benchmark weights onto the Panel Field bw, and CompositeExposure and DerivedExposure each read them from the field their own bw names. When the two names differ, the member reads another field, or fails because the panel has none, so the prior refuses the pair when it is built. A member that reads no benchmark weight passes. An ObservedExposure is checked through the member it wraps.

Arguments

  • key: The factor name, for the message.
  • xe: The Exposure Estimator of the factor.
  • bw: Name of the benchmark-weight Panel Field of the prior.

Validation

  • xe.bw == bw when xe reads benchmark weights. Raises an ArgumentError.

Returns

  • nothing.

Related

source
PortfolioOptimisers.cross_sectional_exposure_order — Function
cross_sectional_exposure_order(
    factors::AbstractVector{<:Pair}
) -> Tuple{Vector, Any}

Return the dependency layers in which a factor list is computed, and the source index of each factor.

A DerivedExposure is computed from the Factor Exposure of another factor of the same list, so the list is not always computed in the order it was written. The layers this returns put every source in a layer before the factor derived from it. No factor of a layer reads another factor of the same layer, so the members of one layer can be computed in parallel.

Algorithm

  1. Resolve the source name of every DerivedExposure to a position in the list, giving src.
  2. Pass over the list, and collect into a new layer every factor that is in no layer yet and that has no source or whose source is in an earlier layer. Repeat until the layers hold every factor.
  3. Refuse a pass that collects nothing, because the factors left over depend on each other.

Arguments

  • factors: Pairs of factor name => Exposure Estimator.

Validation

  • Every DerivedExposure names a factor of the list. Raises an ArgumentError.
  • No cycle of derived Factor Exposures. Raises an ArgumentError.

Returns

  • lay::Vector{Vector{Int}}: The positions of factors, one vector per layer, in the order the layers are computed.
  • src::Vector{Int}: The position of each factor's source, and 0 when it has none.

Related

source
PortfolioOptimisers.cross_sectional_exposure_widths — Function
cross_sectional_exposure_widths(
    factors::AbstractVector{<:Pair},
    rd::ReturnsResult
) -> Vector{Int64}

Return the number of columns each factor of a list contributes to the factor axis.

A one-hot member contributes one column per level of its categorical Panel Field, and every other member contributes one.

Arguments

  • factors: Pairs of factor name => Exposure Estimator.
  • rd: The returns result to use. It carries the Asset Panel the one-hot levels are read from.

Returns

  • wid::Vector{Int}: The column count of each factor, in the order of factors.

Related

source
PortfolioOptimisers.cross_sectional_exposure_write! — Function
cross_sectional_exposure_write!(Ms::AbstractArray{<:Number, 3}, A::MatNum,
                                rows::AbstractUnitRange, c::Integer, w::Integer,
                                nm::AbstractString) -> nothing
cross_sectional_exposure_write!(Ms::AbstractArray{<:Number, 3}, A::Arr3Num,
                                rows::AbstractUnitRange, c::Integer, w::Integer,
                                nm::AbstractString) -> nothing

Write the rows rows of one member's Factor Exposure into the exposure history, in place.

A member gives a matrix observations × assets when it contributes one factor, as the returns are, and an array assets × factors × observations when it contributes several. Each shape has its own method, and each cuts the rows on its own observation axis.

Arguments

  • Ms: The exposure history, assets × factors × observations, changed in place.
  • A: The Factor Exposure of one member.
  • rows: The observations of A that Ms holds.
  • c: First column of the member on the factor axis.
  • w: Number of columns the member contributes.
  • nm: Name of the factor, for the messages.

Validation

  • The rows of A match the observation and asset axes of Ms, and its factor axis is w. Raises a DimensionMismatch.

Returns

  • nothing. Ms carries the exposure.

Related

source
PortfolioOptimisers.cross_sectional_exposure_fill! — Function
cross_sectional_exposure_fill!(Ms::AbstractArray{<:Number, 3}, xe, rd::ReturnsResult,
                               k::Integer, c::Integer, w::Integer,
                               nm::AbstractString) -> nothing
cross_sectional_exposure_fill!(Ms::AbstractArray{<:Number, 3}, xe::OneHotExposure,
                               rd::ReturnsResult, k::Integer, c::Integer, w::Integer,
                               nm::AbstractString) -> nothing

Write the Factor Exposure of one member of a factor list into its columns of the exposure history Ms, at the last size(Ms, 3) of the k rows of rd.

A OneHotExposure expands its codes straight into the columns of the history with one_hot_fill!, so the fit makes no one-hot block of its own. Every other member gives its Factor Exposure with factor_exposure, and cross_sectional_exposure_write! copies it into the history. Both write the same values.

Algorithm

The method that Julia selects is the algorithm.

  1. A OneHotExposure: read its codes and its live cells with one_hot_inputs, and fill the columns c:(c + w - 1) of Ms from the last rows of both.
  2. Any other member: write the last rows of factor_exposure(xe, rd) with cross_sectional_exposure_write!.

Arguments

  • Ms: The exposure history, assets × factors × observations, changed in place.
  • xe: Exposure Estimator.
  • rd: The returns result to use. It carries the Asset Panel the member reads.
  • k: Number of rows of rd.
  • c: First column of the member in Ms.
  • w: Number of columns of the member.
  • nm: Name of the member, which a refusal names.

Validation

Returns

  • nothing.

Related

source
PortfolioOptimisers.cross_sectional_exposure_rows — Function
cross_sectional_exposure_rows(
    xe,
    rd::ReturnsResult,
    m::Integer
) -> Tuple{ReturnsResult, Any}

Return the rows of the returns data that one member of a factor list reads to give its Factor Exposures at the last m rows.

The exposure of a row reads the last lookback rows of the panel, so the last m rows read the last lookback(xe) + m - 1 of them. A member whose look-back is nothing reads every row.

Arguments

  • xe: Exposure Estimator.
  • rd: The returns result to use. It carries the Asset Panel the member reads.
  • m: Number of last rows whose exposures the member gives.

Returns

  • (rdi, k)::Tuple: rdi::ReturnsResult is rd when the member reads every row, or a view of its last k rows. k::Int is the number of rows of rdi.

Related

source
PortfolioOptimisers.cross_sectional_exposure_history — Function
cross_sectional_exposure_history(
    factors::AbstractVector{<:Pair},
    rd::ReturnsResult;
    ...
) -> ExposureHistory{__T_Ms, Vector{String}, Vector{String}, Nothing} where __T_Ms<:(Array{_A, 3} where _A)
cross_sectional_exposure_history(
    factors::AbstractVector{<:Pair},
    rd::ReturnsResult,
    ex::Transducers.Executor;
    n,
    Zo
) -> ExposureHistory{__T_Ms, Vector{String}, Vector{String}, Nothing} where __T_Ms<:(Array{_A, 3} where _A)

Build the exposure history of a factor list, and the factor axis it is written on.

Algorithm

  1. Read the factor axis with cross_sectional_factor_axis, and the column count of each factor with cross_sectional_exposure_widths.
  2. Take the dependency layers with cross_sectional_exposure_order.
  3. Compute the layers in order, and the members of one layer under ex through cross_sectional_foreach. Give a DerivedExposure the exposure of its source, which an earlier layer has already written. Each member writes its own columns of Ms, so every executor gives the same history.
  4. When n is an integer, give each member the rows of rd that cross_sectional_exposure_rows names, and keep the last n rows of its exposure. Those rows equal the rows of the exposure over every row of rd. A DerivedExposure has a look-back of one, so it reads the n rows of its source that Ms holds.
  5. Write each Factor Exposure into Ms with cross_sectional_exposure_fill!, and the one of a DerivedExposure with cross_sectional_exposure_write!. Ms takes the element type real(eltype(X)), which holds a fractional exposure and the NaN of an inactive cell.

Arguments

  • factors: Pairs of factor name => Exposure Estimator.
  • rd: The returns result to use. It carries the Asset Panel every member reads.
  • ex: Parallel execution strategy. It computes the members of one dependency layer.
  • n: Number of last rows of rd to compute, or nothing for every row. The carry fold of a CrossSectionalFactorPrior gives the number of rows of a step.
  • Zo: The observed exposures over the rows of Ms, which the fit joins to Ms later, or nothing. cross_sectional_history_backing gives the two one backing when it can.

Validation

Returns

  • eh::ExposureHistory: The exposure history Ms, assets × factors × observations, of the last n rows of rd or of every row, the factor names nf, the family labels fam, and the observed exposures Zo, as ExposureHistory states them. Ms and Zo are views of one backing when cross_sectional_history_backing gives one.

Related

source
PortfolioOptimisers.cross_sectional_history_backing — Function
cross_sectional_history_backing(T::Type, m::Integer, N::Integer, K::Integer,
                                Zo::Nothing) -> HistoryBacking
cross_sectional_history_backing(T::Type, m::Integer, N::Integer, K::Integer,
                                Zo::AbstractArray{<:Any, 3}) -> HistoryBacking
cross_sectional_history_backing(T::Type{Tz}, m::Integer, N::Integer, K::Integer,
                                Zo::AbstractArray{Tz, 3}) where {Tz} -> HistoryBacking

Allocate the exposure history of cross_sectional_exposure_history, and give it one backing with the observed exposures Zo when the two can share it.

The batch fit of a CrossSectionalFactorPrior joins the observed exposures to the exposure history along the factor axis, with cross_sectional_joined_exposures. When the two are views of one backing, the join is a view of it too, and the fit copies no history of the joined factors. The method copies Zo into the spare factors of the backing once. A backing has one element type, so Zo of another element type keeps its own array, and the history keeps the type of its returns.

Algorithm

The method that Julia selects is the algorithm.

  1. No Zo: allocate N × K × m of T, and return no Zo.
  2. Zo of another element type than T: allocate N × K × m of T, and return Zo unchanged.
  3. Zo of the element type T: allocate N × (K + size(Zo, 2)) × m of T, copy Zo into its last factors, and return the views of its first K factors and of its last factors. The slice of one observation of each view is one contiguous block.

Arguments

  • T::Type: Element type of the history.
  • m::Integer: Number of observations of the history.
  • N::Integer: Number of assets.
  • K::Integer: Number of estimated factors.
  • Zo: The observed exposures, assets × factors × observations, or nothing.

Validation

  • Zo has m observations and N assets. Raises a DimensionMismatch.

Returns

  • hb::HistoryBacking: The history of the estimated factors, to be written, and the observed exposures, or nothing.

Related

source
PortfolioOptimisers.assert_history_backing_axes — Function
assert_history_backing_axes(
    Zo::AbstractArray{<:Any, 3},
    m::Integer,
    N::Integer
)

Check that the observed exposures share the observation and asset axes of the exposure history that cross_sectional_history_backing allocates.

Arguments

  • Zo: The observed exposures, assets × factors × observations.
  • m::Integer: Number of observations of the history.
  • N::Integer: Number of assets.

Validation

  • size(Zo, 3) == m and size(Zo, 1) == N. Raises a DimensionMismatch.

Returns

  • nothing.

Related

source
PortfolioOptimisers.cross_sectional_eligible — Function
cross_sectional_eligible(
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    Ms::AbstractArray{<:Union{Number, AbstractJuMPScalar}, 3},
    emsk::AbstractMatrix{Bool}
) -> BitMatrix
cross_sectional_eligible(
    X::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    Ms::AbstractArray{<:Union{Number, AbstractJuMPScalar}, 3},
    emsk::AbstractMatrix{Bool},
    Zo::Union{Nothing, AbstractArray{<:Union{Number, AbstractJuMPScalar}, 3}}
) -> BitMatrix

Return the eligibility mask of a cross-sectional fit.

An asset enters the fit of an observation when it is in the estimation universe, its return is finite, and its lagged Factor Exposure is finite for every factor. The weight policy writes its weights on this mask, so an ineligible pair takes a weight of zero and does not enter the fit.

Mathematical definition

\[\begin{align} m_{ti} &= e_{ti} \, \mathbb{1}\left[x_{t,\,i} \in \mathbb{R}\right] \prod_{k=1}^{K} \mathbb{1}\left[\tilde{B}_{tik} \in \mathbb{R}\right]\,. \end{align}\]

Where:

  • $m_{ti}$: Eligibility of asset $i$ at observation $t$.
  • $\tilde{B}_{tik}$: Lagged Factor Exposure of asset $i$ to factor $k$ that the fit reads against observation $t$.
  • $e_{ti}$: Estimation flag of asset $i$ at observation $t$, $1$ when the estimation mask of the Asset Panel holds the asset and $0$ when it does not.
  • $x_{t,\,i}$: Return of asset $i$ at observation $t$.
  • $K$: Number of factors.

Arguments

  • X: Asset returns, observations × assets.
  • Ms: The lagged exposure history, assets × factors × observations, aligned with X.
  • emsk: The estimation mask, observations × assets, aligned with X.
  • Zo: The exposure history of the observed factors, aligned with X, or nothing. Its factors join the product beside those of Ms, so a caller that holds the two histories apart does not join them to test them.

Validation

  • Ms and Zo match X on the observation and asset axes, and emsk matches X. Raises a DimensionMismatch.

Returns

  • msk::BitMatrix: The eligibility mask, observations × assets.

Related

source
PortfolioOptimisers.assert_cross_sectional_coverage — Function
assert_cross_sectional_coverage(
    msk::AbstractMatrix{Bool},
    minra::Integer
)
assert_cross_sectional_coverage(
    msk::AbstractMatrix{Bool},
    minra::Integer,
    cb::Union{Nothing, ObservedBlock}
)

Refuse a fit whose observations do not carry enough eligible assets.

A cross-sectional fit of K factors needs more assets than factors, and a fit with few more assets than factors gives imprecise factor returns. The prior refuses the whole fit and does not drop the thin observations, because a dropped observation leaves a gap in the factor-return series.

Arguments

  • msk: The eligibility mask, observations × assets.
  • minra: The smallest eligible asset count an observation may carry.
  • cb: The observed factors of the fitted observations, from cross_sectional_observed_block, or nothing. An observed return that is NaN makes the net return of every asset that holds the factor NaN, so the message names the observed factors that have no return at the observation it names, with cross_sectional_observed_gap.

Validation

  • Every observation carries at least minra eligible assets. Raises an ArgumentError.

Returns

  • nothing.

Related

source
PortfolioOptimisers.cross_sectional_ready_factors — Function
cross_sectional_ready_factors(
    mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    sigma::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}
) -> BitVector

Return the factors whose moments the factor prior of a Cross-Sectional Factor Prior states.

A factor is ready when its mean and its variance are finite, which is the rule of the Investable Mask of a LowOrderPrior read over the factors. A factor prior with a warm-up can state NaN for one factor and finite moments for the others, so the prior reads the moments of the ready factors alone. The covariance of two ready factors must then be finite, or the block of the ready factors states no covariance matrix. A factor prior that is a LowOrderPrior already keeps that rule, so the check fails only on moments that a processing step or a direct call gives.

Mathematical definition

\[\begin{align} \mathcal{R} &= \left\{k : \hat{\mu}_{f,k} \in \mathbb{R},\ \hat{\Sigma}_{f,kk} \in \mathbb{R}\right\}\,. \end{align}\]

Where:

  • $\mathcal{R}$: The ready factors.
  • $\hat{\mu}_{f,k}$: Entry $k$ of the factor mean that the factor prior states.
  • $\hat{\Sigma}_{f,kl}$: Entry $(k, l)$ of the factor covariance that the factor prior states.

Arguments

  • mu: The factor means the factor prior stated.
  • sigma: The factor covariance the factor prior stated.

Validation

  • mu and sigma agree on the factor axis. Raises a DimensionMismatch.
  • Every entry of sigma between two ready factors is finite. Raises an IsNonFiniteError that names the first such pair.

Returns

  • rdy::BitVector: true at each ready factor.

Related

source
PortfolioOptimisers.cross_sectional_determined — Function
cross_sectional_determined(
    L::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    rdy::AbstractVector{Bool}
) -> BitVector

Return the assets whose factor moments a Cross-Sectional Factor Prior determines.

The mean of an asset reads the factor means only on the support of its loadings, the factors where its loading is not zero. Its covariance with another asset reads the factor covariance only on the two supports. So an asset is determined when its loadings are finite and its support holds only ready factors. A factor that is not ready changes no moment of an asset with a zero loading on it.

Mathematical definition

\[\begin{align} \mathcal{D} &= \left\{i : B_{Tik} \in \mathbb{R} \ \forall k,\ k \in \mathcal{R} \ \forall k : B_{Tik} \neq 0\right\}\,. \end{align}\]

Where:

  • $\mathcal{D}$: The determined assets.
  • $\mathcal{R}$: The ready factors, from cross_sectional_ready_factors.
  • $\mathbf{B}_{T}$: Latest Factor Exposures, $N \times K$, with entry $B_{Tik}$ for asset $i$ and factor $k$.

Arguments

  • L: The reduced loadings of the latest observation, assets × factors.
  • rdy: true at each ready factor.

Validation

  • L and rdy agree on the factor axis. Raises a DimensionMismatch.

Returns

  • det::BitVector: true at each determined asset.

Related

source
PortfolioOptimisers.assert_cross_sectional_factor_moments — Function
assert_cross_sectional_factor_moments(
    mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    sigma::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    n::Integer,
    det::AbstractVector{Bool}
)

Refuse a fit whose factor prior determines the moments of no asset.

The warm-up of the Descriptors and the warm-up of the factor prior add up. The Descriptors fix the first observation of the factor-return history, and the factor prior then warms up over that history. A window that covers the first warm-up can be too short for the second, and a factor prior then gives NaN and does not raise. An asset whose loadings read only finite factor moments keeps its moments, and every other asset is NaN and leaves through the Investable Mask. When no asset is determined, the prior states no moment at all, so the fit refuses with a message that names the cause.

Arguments

  • mu: The factor means the factor prior stated.
  • sigma: The factor covariance the factor prior stated.
  • n: The count of fitted observations the factor prior read.
  • det: true at each determined asset, from cross_sectional_determined.

Validation

  • At least one asset is determined. Raises an IsNonFiniteError that counts the non-finite factor means and covariance entries.

Returns

  • nothing.

Related

source
PortfolioOptimisers.cross_sectional_factor_moments — Function
cross_sectional_factor_moments(
    pe::Union{AbstractLowOrderPriorEstimator_A, AbstractLowOrderPriorEstimator_AF},
    f_mp::AbstractMatrixProcessingEstimator,
    f::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    lv::AbstractVector{Bool};
    strict,
    cache_reg,
    kwargs...
) -> FactorPriorMoments{_A, _B, _C, _D, _E, __T_eoc} where {_A, _B, _C, _D, _E, __T_eoc<:Tuple{Union{Nothing, Real}, Union{Nothing, Real}}}

Fit the factor prior of a Cross-Sectional Factor Prior on the factors that are not empty, and give every Empty Factor no mean and no variance.

An Empty Factor has a return of zero at every fitted observation, from cross_sectional_live_regression. A covariance estimator that turns the covariance into a correlation divides by the zero volatility of such a factor, and gives NaN. So the function fits pe and processes the covariance with f_mp on the other factors only. It then puts the answer on the whole factor axis, with a zero return in every scenario, a zero mean, and a zero row and column of the covariance at each Empty Factor. When no factor is empty, the moments are those that pe states over f.

The method over pr takes the factor prior already fitted on the factors that are not empty. The carry fold of a CrossSectionalFactorPrior reads its folded factor prior out and gives the result to it, so both routes process and expand the moments with one code.

Algorithm

  1. Take the columns of f at lv, giving fl. When every factor is in lv, fl is f.
  2. Fit pe on fl, giving pr. The method over pr starts at step 3.
  3. Process the covariance of pr in place with f_mp, over fl, with matrix_processing_block!. The denoising step reads the count of the covariance of pr from moment_observation_count, and not the number of rows of fl. A factor prior with a warm-up can state NaN for one factor and finite moments for the others, so the processing reads the block of the finite variances, and every entry outside it keeps its NaN. A covariance with no finite variance has no block, and is not processed.
  4. When every factor is in lv, return the scenarios, the mean and the covariance of pr. Otherwise, write them into the columns at lv of a zero scenario matrix X, the entries at lv of a zero mean mu, and the block at lv of a zero covariance sigma.
  5. Divide the scenario weights of pr by their sum with cross_sectional_scenario_weights.

Arguments

  • pe: The factor prior estimator.
  • pr: The factor prior that pe states over the columns of f at lv.
  • f_mp: Matrix processing estimator of the factor covariance.
  • f: The factor returns, observations × factors.
  • lv: true at each factor that is not empty.
  • strict: Whether to throw an error if sets does not contain the desired value in sets.dict[key]. It is forwarded to pe.
  • kwargs...: Additional keyword arguments passed to matrix_processing!.

Validation

Returns

  • fm::FactorPriorMoments: The factor return scenarios X, the expected factor returns mu and the processed factor covariance sigma of pr on the whole factor axis; the scenario weights w of pr divided by their sum, or nothing; the effective number of scenarios ens, the effective counts of the moments eoc, the Kullback-Leibler divergence kld and the original weights ow of pr; and the cache pc of an opinion pool. A mean that pr does not state is NaN, and so are the row and the column of a factor that the factor prior states no variance for.

Related

source
PortfolioOptimisers.cross_sectional_idiosyncratic_covariance — Function
cross_sectional_idiosyncratic_covariance(
    th::Real,
    ce::CovarianceEstimator,
    pdm::Union{Nothing, AbstractPosdefEstimator},
    S::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    ev::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    amsk::AbstractMatrix{<:Bool}
) -> Any

Return the idiosyncratic covariance of the latest observation.

The variances come from the latest idiosyncratic variances alone, and ce gives only the correlations. So the diagonal of the answer does not depend on ce, and a positive threshold changes only the entries off the diagonal.

Mathematical definition

\[\begin{align} \rho_{ij} &= \frac{C_{ij}}{\sqrt{C_{ii} \, C_{jj}}}\,, \\ \tilde{\rho}_{ij} &= \begin{cases} 1 & i = j\,, \\ \rho_{ij} \, \mathbb{1}\left[\lvert \rho_{ij} \rvert > \tau\right] & i \neq j\,, \end{cases} \\ D_{ij} &= \sqrt{v_{Ti} \, v_{Tj}} \, \tilde{\rho}_{ij}\,. \end{align}\]

Where:

  • $C_{ij}$: Entry of the covariance that ce estimates from the standardised idiosyncratic returns $z_{ti}$, with no fill. A cell with no finite value stays a gap for ce, because a value written into it is not an observation of the pair: the mean of the other assets of its observation, which the scenarios take, would correlate the asset with each of them.
  • $\rho_{ij}$, $\tilde{\rho}_{ij}$: Correlation of assets $i$ and $j$, before and after the threshold. The indicator is $0$ when $\rho_{ij}$ is not finite.
  • $\tau$: The correlation threshold. At $\tau = 0$ the answer is the vector of the latest variances $v_{Ti}$, which is the diagonal of $\mathbf{D}$.
  • $D_{ij}$: Entry of $\mathbf{D}$.
  • $\mathbf{D}$: Covariance of the idiosyncratic returns that the loadings block carries, $N \times N$. It is diagonal when the block stores a variance vector.
  • $z_{ti}$: Standardised idiosyncratic return of asset $i$ at observation $t$.
  • $v_{ti}$: Idiosyncratic variance of asset $i$ that the fit estimated from the observations up to $t$, so it is the forecast for observation $t + 1$.
  • $T$: Number of observations.

Algorithm

  1. If th is zero, return ev, the latest idiosyncratic variances. The block then carries a vector, and the asset covariance takes a diagonal.
  2. If S is nothing, the carry fold folded ce over the rows, so read C from the state of ce with the call of Statistics.cov with no data.
  3. Otherwise, take the rows and the keyword arguments of ce with cross_sectional_correlation_rows, and estimate the covariance C of the rows with ce.
  4. Threshold, rescale and repair C with cross_sectional_thresholded_covariance.

Arguments

  • th: The correlation threshold.
  • ce: Covariance estimator of the standardised idiosyncratic returns. When S is nothing, it is the estimator that the carry fold folded over them.
  • pdm: Positive definite matrix estimator, or nothing.
  • S: Standardised idiosyncratic returns with no fill, observations × assets, from cross_sectional_standardised_residuals with filled = false, or nothing when ce carries their fold.
  • ev: The latest idiosyncratic variances, one per asset.
  • amsk: The active mask, observations × assets. Only a ce with a non-finite gap_fill_value reads it.

Returns

  • esigma::VecNum: The latest idiosyncratic variances, when th is zero.
  • esigma::MatNum: The idiosyncratic covariance, when th is positive.

Related

source
PortfolioOptimisers.cross_sectional_variance_counts — Function
cross_sectional_variance_counts(cnt::Nothing, csr::CrossSectionalRegression)
    -> ResidualVarianceCounts
cross_sectional_variance_counts(cnt::VarianceCount, csr::CrossSectionalRegression)
    -> ResidualVarianceCounts

Degrees of freedom and divisor of each idiosyncratic variance of a Cross-Sectional Factor Prior.

A cross-sectional fit spends its parameters on the assets of each period, not on the observations of each asset. The variance of each asset takes the fraction of its degrees of freedom that the fits leave, and cross_sectional_residual_dof states the fraction. It reads the residual count that the fit of each period records in csr.edof, because only the fit knows the parameters that it spent: the number of columns overstates the spend of a rank-deficient period and of a penalised fit. Under the default exponentially weighted variance, the law of the variance is an approximation, so the level of a bound that reads these counts is also approximate.

Algorithm

The method that Julia selects is the algorithm.

  1. cnt is nothing: the variance estimator states no count, so neither count is known.
  2. cnt is a count: scale the effective count cnt.n of each asset with cross_sectional_residual_dof, and take the divisor cnt.m. A fit that records no count states no degrees of freedom.

Arguments

  • cnt: The count and divisor that variance_count states for the variance estimator, or nothing when it states none.
  • csr: The cross-sectional fit.

Returns

  • c::ResidualVarianceCounts: The degrees of freedom edof and the divisor ediv of each variance. Both are nothing when the variance estimator states no count, and edof alone is nothing when the fit records no count.

Related

source
PortfolioOptimisers.cross_sectional_finite_mean — Function
cross_sectional_finite_mean(
    S::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    t::Integer
) -> Any

Return the average of the finite entries of one observation of a matrix.

The one caller is cross_sectional_standardised_residuals, which writes the average into each active cell that it cannot standardise. An observation with no finite entry gives zero, which is the mean of a standardised series.

Mathematical definition

\[\begin{align} \bar{z}_{t} &= \begin{cases} \dfrac{1}{n_{t}} \sum_{i \in \mathcal{F}_{t}} z_{ti} & n_{t} > 0\,, \\ 0 & n_{t} = 0\,. \end{cases} \end{align}\]

Where:

  • $\bar{z}_{t}$: Mean standardised idiosyncratic return of observation $t$, the average of $z_{ti}$ over $\mathcal{F}_{t}$, and $0$ when $\mathcal{F}_{t}$ is empty.
  • $z_{ti}$: Standardised idiosyncratic return of asset $i$ at observation $t$.
  • $\mathcal{F}_{t}$: Finite cross-section of observation $t$, the assets whose standardised return $z_{ti}$ is finite.
  • $n_{t}$: Count of the assets in $\mathcal{F}_{t}$.

Arguments

  • S: The matrix, observations × assets.
  • t: The observation.

Returns

  • avg::Real: The average of the finite entries of observation t, or zero.

Related

source
PortfolioOptimisers.cross_sectional_standardised_residuals — Function
cross_sectional_standardised_residuals(
    eps::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    vs::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    amsk::AbstractMatrix{Bool};
    filled
) -> Matrix

Return the standardised idiosyncratic returns a scenario set is rebuilt from.

The function divides each idiosyncratic return by the idiosyncratic volatility of the same observation, so the whole history is on one scale and the latest volatilities can rescale it. An active pair whose standardised return is not finite takes the average standardised return of its own observation, so a sparse history does not shorten the scenario set. An inactive pair stays NaN.

Under filled = false the function returns $z_{ti}$ with NaN at every cell that is not finite, and writes no average. The estimate of the idiosyncratic correlation reads that form, see cross_sectional_idiosyncratic_covariance: the average of the other assets of an observation is not an observation of the pair, and it would correlate the asset with each of them.

The prior passes eps through leverage_one_residuals first, so under its default Leverage-One Mask a Leverage-One Pair is NaN in eps, and it takes $\bar{z}_{t}$ under filled = true whatever its variance.

Mathematical definition

\[\begin{align} z_{ti} &= \begin{cases} \varepsilon_{ti} / \sqrt{v_{ti}} & a_{ti} = 1\,, \\ \mathrm{NaN} & a_{ti} = 0\,, \end{cases} \\ \tilde{z}_{ti} &= \begin{cases} z_{ti} & z_{ti} \in \mathbb{R}\,, \\ \bar{z}_{t} & a_{ti} = 1,\ z_{ti} \notin \mathbb{R}\,, \\ \mathrm{NaN} & a_{ti} = 0\,. \end{cases} \end{align}\]

Where:

  • $z_{ti}$: Standardised idiosyncratic return of asset $i$ at observation $t$.
  • $\tilde{z}_{ti}$: Filled standardised idiosyncratic return of asset $i$ at observation $t$. It is $z_{ti}$ where that is finite, $\bar{z}_{t}$ where the asset is active and $z_{ti}$ is not finite, and NaN where the asset is inactive.
  • $\bar{z}_{t}$: Mean standardised idiosyncratic return of observation $t$, the average of $z_{ti}$ over $\mathcal{F}_{t}$, and $0$ when $\mathcal{F}_{t}$ is empty.
  • $\varepsilon_{ti}$: Idiosyncratic return of asset $i$ at observation $t$.
  • $v_{ti}$: Idiosyncratic variance of asset $i$ that the fit estimated from the observations up to $t$, so it is the forecast for observation $t + 1$.
  • $a_{ti}$: Panel activity of asset $i$ at observation $t$, $1$ when the active mask of the Asset Panel activates the asset and $0$ when it does not.

A zero variance gives $0 / 0$ or $\pm\infty$, and a variance still in warm-up is NaN, so an active pair takes $\bar{z}_{t}$ in each case.

Arguments

  • eps: Idiosyncratic returns, observations × assets.
  • vs: Idiosyncratic variance history, observations × assets.
  • amsk: The active mask, observations × assets.
  • filled: When true, an active cell that is not finite takes $\bar{z}_{t}$. When false, it is NaN.

Returns

  • S::Matrix{<:Real}: The standardised idiosyncratic returns, observations × assets: $\tilde{z}_{ti}$ under filled = true, and $z_{ti}$ with NaN at each cell that is not finite under filled = false.

Related

source
PortfolioOptimisers.cross_sectional_investable — Function
cross_sectional_investable(
    amsk::AbstractVector{Bool},
    ev::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    det::AbstractVector{Bool}
) -> Vector{Int64}

Return the assets a Cross-Sectional Factor Prior can state a finite moment for.

The prior fits on the coverage universe and answers on it, so an asset that it cannot state a moment for stays in the result and carries NaN. An asset is investable when the Asset Panel activates it at the latest observation, its latest idiosyncratic variance is finite, and the factor prior determines its factor moments.

Mathematical definition

\[\begin{align} \mathcal{I} &= \left\{i : a_{Ti} = 1,\ v_{Ti} \in \mathbb{R},\ i \in \mathcal{D}\right\}\,. \end{align}\]

Where:

  • $\mathcal{I}$: Investable set, the assets the prior states a finite moment for.
  • $a_{ti}$: Panel activity of asset $i$ at observation $t$, $1$ when the active mask of the Asset Panel activates the asset and $0$ when it does not.
  • $v_{ti}$: Idiosyncratic variance of asset $i$ that the fit estimated from the observations up to $t$, so it is the forecast for observation $t + 1$.
  • $\mathcal{D}$: The determined assets, from cross_sectional_determined.
  • $T$: Number of observations.

Arguments

  • amsk: The active mask of the latest observation, one entry per asset.
  • ev: The latest idiosyncratic variances, one per asset.
  • det: true at each determined asset.

Returns

  • idx::Vector{Int}: The investable assets, in ascending order.

Related

source
PortfolioOptimisers.cross_sectional_panel_masks — Function
cross_sectional_panel_masks(pnl::AssetPanel{<:Any, Nothing, Nothing}) -> Union{}
cross_sectional_panel_masks(pnl::AssetPanel) -> Tuple

Read the two universe masks a Cross-Sectional Factor Prior fits against.

A static Asset Panel carries no observation axis and no masks, so it states no point-in-time universe. The mask fields are type parameters of the Asset Panel, so each shape has its own method.

Arguments

  • pnl: The Asset Panel.

Validation

  • The Asset Panel is time-varying. Raises an ArgumentError.

Returns

  • amsk::AbstractMatrix{Bool}: The active mask, observations × assets.
  • emsk::AbstractMatrix{Bool}: The estimation mask, observations × assets.

Related

source
PortfolioOptimisers.cross_sectional_cap_finite! — Function
cross_sectional_cap_finite!(msk::BitMatrix, mcap::Nothing) -> nothing
cross_sectional_cap_finite!(msk::BitMatrix, mcap::MatNum) -> nothing

Drop from a mask every pair whose market capitalisation is not finite, in place.

A weight raised from a market capitalisation needs that capitalisation. The prior makes the same cut on its two weight masks, the benchmark mask and the regression mask. A power of zero reads no capitalisation, so the nothing method leaves the mask as it is.

Arguments

  • msk: The mask, observations × assets, changed in place.
  • mcap: The market capitalisation, observations × assets, or nothing.

Validation

  • mcap matches msk. Raises a DimensionMismatch.

Returns

  • nothing. msk carries the cut.

Related

source
PortfolioOptimisers.cross_sectional_needs_market_cap — Function
cross_sectional_needs_market_cap(
    bp::Real,
    alg::AbstractCrossSectionalWeightsAlgorithm
) -> Bool

Return whether a Cross-Sectional Factor Prior reads the market capitalisation.

The benchmark power, which the prior carries, and the regression power, which the weight policy carries, each raise a weight from the market capitalisation. When both are zero, every eligible asset takes the same weight, and the prior reads no Panel Field for the capitalisation. Every member of the weight family carries p, which cs_weights_initial also reads.

Arguments

  • bp: The benchmark market-capitalisation power.
  • alg: The weight policy of the cross-sectional fit.

Returns

  • ans::Bool: Whether the prior reads the market capitalisation.

Related

source
PortfolioOptimisers.cross_sectional_rows — Function
cross_sectional_rows(A::Nothing, r) -> nothing
cross_sectional_rows(A::MatNum, r) -> MatNum

Take a set of observations out of an optional matrix.

The prior trims every parallel array to the same observations. The market capitalisation is the one array that can be absent, and the nothing method keeps it absent.

Arguments

  • A: The matrix, observations × assets, or nothing.
  • r: The observations to keep.

Returns

  • A::Option{<:MatNum}: The rows, or nothing.

Related

source
PortfolioOptimisers.cross_sectional_reduced_loadings — Function
cross_sectional_reduced_loadings(fcb::Nothing, L::MatNum) -> nothing
cross_sectional_reduced_loadings(fcb::FactorFamilyBasis, L::MatNum) -> MatNum

Return the reduced loadings a CrossSectionalFactorModel stores, or nothing.

The block holds L exactly when it holds a Factor Family Basis. One dispatch on the basis decides both, so the two cannot disagree.

Arguments

  • fcb: The Factor Family Basis, or nothing.
  • L: The loadings on the reduced axis.

Returns

  • L::Option{<:MatNum}: The reduced loadings, or nothing.

Related

source
PortfolioOptimisers.cross_sectional_family_basis — Function
cross_sectional_family_basis(families::Nothing, Ms, bw, nf, fam) -> ReducedFactorAxis
cross_sectional_family_basis(families::AbstractVector{<:Pair}, Ms, bw, nf, fam)
    -> ReducedFactorAxis

Build the Factor Family Basis of a Cross-Sectional Factor Prior, and reduce the factor axis through it.

A prior that constrains no Factor Family fits on the raw axis. The nothing method takes that case. It returns no basis, and returns the exposure history and the two label vectors unchanged.

Arguments

  • families: Pairs of family label => dropped member, or nothing.
  • Ms: The exposure history on the raw axis, assets × factors × observations.
  • bw: The benchmark weights, observations × assets.
  • nf: Name of each raw factor.
  • fam: Family label of each raw factor.

Validation

Returns

  • fb::ReducedFactorAxis: The basis fcb, or nothing, the exposure history Ms on the reduced axis, and the name nf and the family label fam of each reduced factor.

Related

source
PortfolioOptimisers.cross_sectional_expand — Function
cross_sectional_expand(fcb::Nothing, r, lag::Integer, f) -> Nothing
cross_sectional_expand(fcb::FactorFamilyBasis, r, lag::Integer, f) -> MatNum
cross_sectional_expand(fcb::Nothing, r, mu, sigma) -> RawAxisMoments
cross_sectional_expand(fcb::FactorFamilyBasis, r, mu, sigma) -> RawAxisMoments

Expand the realised factor returns, or the factor moments, from the reduced axis onto the raw one.

The nested factor prior of a LowOrderPrior is on the raw axis, so a constraint written in the name of a dropped factor still resolves. A prior that constrains no Factor Family already fits on the raw axis, and the nothing methods take that case: the moments pass through, and the returns give nothing, because a block with no re-basis carries no second copy of its factor returns.

The fit of observation t regresses the returns of t on the exposures of t - lag, so its coefficients are coordinates in the basis of t - lag. So the realised factor returns expand with the lagged ratios, and the prior stores the answer on the block as fr before the moments exist. The block's own fcb covers its rows alone, so it cannot expand the first lag rows, and this is the one place where their basis still exists. The moments describe the next observation, so they expand with the ratios of the last fitted observation. Each method takes the whole basis and slices it. Two slice arguments could hold one basis and one nothing, and no method takes that pair.

Arguments

  • fcb: The Factor Family Basis over the post-warm-up history, or nothing.
  • r: The fitted observations, as an index into the post-warm-up axis.
  • lag: The exposure lag.
  • f: Realised factor returns on the reduced axis, observations × factors.
  • mu: Expected factor returns on the reduced axis.
  • sigma: Factor covariance on the reduced axis.

Validation

Returns

  • fr::Option{<:MatNum}: The realised factor returns on the raw axis, or nothing with no re-basis.
  • ex::RawAxisMoments: The expected factor returns mu and the factor covariance sigma on the raw axis.

Related

source
PortfolioOptimisers.cross_sectional_residual_block — Function
cross_sectional_residual_block(esigma::VecNum, idx, mtx_sqrt = EigenFallbackSquareRoot()) -> IdiosyncraticBlock
cross_sectional_residual_block(esigma::MatNum, idx, mtx_sqrt = EigenFallbackSquareRoot()) -> IdiosyncraticBlock

Return the idiosyncratic block of the asset covariance and a square root of it.

The idiosyncratic covariance is a vector of variances or a full matrix, and each shape has its own method. Both methods return the block over the investable assets alone, because a non-investable asset can carry a non-finite variance, which has no square root.

Mathematical definition

\[\begin{align} \mathbf{R} &= \begin{cases} \operatorname{diag}\left(\sqrt{v_{Ti}}\right)_{i \in \mathcal{I}} & \mathbf{D} \text{ diagonal}\,, \\ \operatorname{sqrt}\left(\mathbf{D}_{\mathcal{I}\mathcal{I}}\right) & \text{otherwise}\,. \end{cases} \end{align}\]

Where:

  • $\operatorname{sqrt}$: The square root that matrix_square_root takes under mtx_sqrt. By default it is the lower Cholesky factor of a positive definite matrix, and the eigen square root of a singular one.
  • $\mathbf{R}$: Idiosyncratic square root, lower triangular, with $\mathbf{R} \mathbf{R}^{\intercal} = \mathbf{D}_{\mathcal{I}\mathcal{I}}$.
  • $\mathbf{D}$: Covariance of the idiosyncratic returns that the loadings block carries, $N \times N$. It is diagonal when the block stores a variance vector.
  • $\mathcal{I}$: Investable set, the assets the prior states a finite moment for.
  • $v_{ti}$: Idiosyncratic variance of asset $i$ that the fit estimated from the observations up to $t$, so it is the forecast for observation $t + 1$.
  • $T$: Number of observations.

Arguments

  • esigma: The idiosyncratic variances, or the idiosyncratic covariance.
  • idx: The investable assets.
  • mtx_sqrt: Square-root algorithm of the factor covariance, and of the idiosyncratic covariance when that block is a full matrix, whose square roots make up chol. The default EigenFallbackSquareRoot takes the Cholesky factor where it exists, and the eigen square root of a singular positive semidefinite matrix otherwise. RidgeCholeskySquareRoot takes the Cholesky factor of the matrix with a small ridge added, and nothing takes the plain Cholesky factor, which raises a LinearAlgebra.PosDefException on a matrix that is not positive definite. matrix_square_root states each algorithm. The vector method reads no square-root algorithm.

Validation

  • A full block restricted to idx has a square root under mtx_sqrt. Raises a LinearAlgebra.PosDefException.

Returns

  • rb::IdiosyncraticBlock: The block D that the asset covariance adds, and a square root R of it, investable assets × investable assets.

Related

source
PortfolioOptimisers.cross_sectional_lift — Function
cross_sectional_lift(
    srep::Union{Nothing, AbstractSystematicRepair},
    mp::AbstractMatrixProcessingEstimator,
    L::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    f_mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    f_sigma::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    esigma::Union{AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}, AbstractVector{<:Union{Number, AbstractJuMPScalar}}},
    idx::AbstractVector{<:Integer},
    Xs::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}};
    ...
) -> LiftedMoments
cross_sectional_lift(
    srep::Union{Nothing, AbstractSystematicRepair},
    mp::AbstractMatrixProcessingEstimator,
    L::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    f_mu::AbstractVector{<:Union{Number, AbstractJuMPScalar}},
    f_sigma::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    esigma::Union{AbstractMatrix{<:Union{Number, AbstractJuMPScalar}}, AbstractVector{<:Union{Number, AbstractJuMPScalar}}},
    idx::AbstractVector{<:Integer},
    Xs::AbstractMatrix{<:Union{Number, AbstractJuMPScalar}},
    T::Real;
    mtx_sqrt,
    kwargs...
) -> LiftedMoments

Lift a factor distribution onto the assets of a Cross-Sectional Factor Prior.

The square root chol factorises the factor model before mp processes it, as in factor_lift. matrix_square_root takes the square root of each block under mtx_sqrt. So chol' * chol equals sigma only when the processing leaves the matrix unchanged, which the default mp does to a positive definite matrix. A detoning mp moves sigma and leaves chol where it was.

Mathematical definition

\[\begin{align} \mathcal{S} &= \left\{i : B_{Tik} \in \mathbb{R} \ \forall k \in \{1, \ldots, K\}\right\}\,, \\ \boldsymbol{\mu}_{\mathcal{S}} &= \mathbf{B}_{T,\,\mathcal{S}} \odot \boldsymbol{\mu}_{f}\,, \\ \mathbf{\Sigma}_{\mathcal{S}\mathcal{S}} &= \mathbf{B}_{T,\,\mathcal{S}} \odot \mathbf{F} \odot \mathbf{B}_{T,\,\mathcal{S}}^{\intercal} + \mathbf{D}_{\mathcal{S}\mathcal{S}}\,, \\ \mathbf{C}_{\cdot\mathcal{I}} &= \begin{bmatrix} \mathbf{B}_{T,\,\mathcal{I}} \, \operatorname{chol}(\mathbf{F}_{\mathcal{R}\mathcal{R}}) & \mathbf{R} \end{bmatrix}^{\intercal}\,. \end{align}\]

Where:

  • $\mathcal{S}$: Assets with finite latest loadings, the assets whose systematic part the model states. It holds $\mathcal{I}$. The contract of factor_exposure writes NaN at every inactive cell, so an asset of $\mathcal{S}$ is active at the latest observation.
  • $\boldsymbol{\mu}$: Expected asset returns.
  • $\mathbf{\Sigma}$: Asset covariance.
  • $\mathbf{C}$: Low-rank square root of the asset covariance, $(K + \lvert \mathcal{I} \rvert) \times N$ over the full asset universe.
  • $\mathbf{B}_{T,\,\mathcal{S}}$, $\mathbf{B}_{T,\,\mathcal{I}}$: The rows of $\mathbf{B}_{T}$ at the assets of $\mathcal{S}$ and of $\mathcal{I}$.
  • $\odot$: The product over the support of each row of loadings, from support_product. It reads a factor moment only where the loading is not zero, so a factor moment that is NaN reaches only the entries of the assets that load on it.
  • $\operatorname{chol}$: The square root that matrix_square_root takes under mtx_sqrt. By default it is the lower Cholesky factor of a positive definite matrix, and the eigen square root of a singular one. Of a factor covariance with a zero row and column, which an Empty Factor carries, it is the square root of the block of the other factors, with a zero row and column at the Empty Factor. It is written on the factor axis, with a zero row and column at each factor outside $\mathcal{R}$.
  • $\mathcal{R}$: The ready factors, from cross_sectional_ready_factors.
  • $\mathbf{B}_{T}$: Latest Factor Exposures, $N \times K$, with entry $B_{Tik}$ for asset $i$ and factor $k$.
  • $\boldsymbol{\mu}_{f}$: Expected factor returns, $K \times 1$, the mean of the factor distribution that the prior result carries.
  • $\mathbf{F}$: Factor covariance, $K \times K$, the covariance of the factor distribution that the prior result carries.
  • $\mathbf{D}$: Covariance of the idiosyncratic returns that the loadings block carries, $N \times N$. It is diagonal when the block stores a variance vector.
  • $\mathbf{R}$: Idiosyncratic square root, lower triangular, with $\mathbf{R} \mathbf{R}^{\intercal} = \mathbf{D}_{\mathcal{I}\mathcal{I}}$.
  • $\mathcal{I}$: Investable set, the assets the prior states a finite moment for.

A consequence of the definition: $\mathbf{C}_{\cdot\mathcal{I}}^{\intercal} \mathbf{C}_{\cdot\mathcal{I}} = \mathbf{\Sigma}_{\mathcal{I}\mathcal{I}}$.

The answer states an entry exactly when the model determines it. An asset of $\mathcal{S}$ outside $\mathcal{I}$ has no idiosyncratic variance, as in the warm-up of its variance. Its entry of $\boldsymbol{\mu}$ is finite, and so is its covariance with each other asset of $\mathcal{S}$ when $\mathbf{D}$ is diagonal, because the model sets the idiosyncratic covariance of two assets to zero. Its variance is NaN, and so is each covariance that reads its variance through an idiosyncratic correlation. The first $K$ entries of its column of $\mathbf{C}$, its systematic root, are finite, and the rest are NaN. Every entry outside $\mathcal{S}$ is NaN. An asset of $\mathcal{S}$ whose loadings read a factor that the factor prior states no moment for is outside $\mathcal{I}$ too. Its entries are NaN exactly where the product reads a NaN factor moment, and its systematic root is NaN, because a root of its factor covariance does not exist. The Investable Mask needs a finite mean and a finite variance, so it still leaves out every asset outside $\mathcal{I}$. mp processes the block over $\mathcal{I}$ alone, so an entry outside that block is the one the model states before any processing.

Algorithm

  1. Take Li, the rows of L at the investable assets. Get D and R from cross_sectional_residual_block.
  2. Project the factor mean through Li, giving mui, and the factor covariance, giving si, each with support_product.
  3. Process si under srep with systematic_processing!, at the count T of the factor covariance. SystematicRepair runs every step of mp, as factor_lift does, and nothing runs every step except the :pdm step.
  4. Add D to si, and make the sum positive definite under the estimator that matrix_processing_posdef gives for mp.
  5. Take the ready factors of f_mu and f_sigma with cross_sectional_ready_factors, and lf, the ready factors whose column of f_sigma over the ready factors is not zero. Take the square root Lf of the block of f_sigma at lf under mtx_sqrt, and write it into the block at lf of a zero matrix Cf.
  6. Build ci, the low-rank square root [Li * Cf R].
  7. Take sdx, the assets whose row of L is finite, and Ls, their rows. Project the factor covariance through Ls with support_product, and add the block of esigma at sdx, giving ss. A NaN variance in esigma makes NaN the entries that read it. Take cdx, the assets of sdx whose loadings are zero at every factor that is not ready.
  8. Write the product of Ls and f_mu with support_product and ss at sdx, and the systematic root L * Cf at cdx, into the full asset universe, over NaN. Then write mui, si and ci at idx over them, giving mu, sigma and chol. Write NaN on the diagonal of sigma at every asset of sdx outside idx, so the Investable Mask of the answer is never wider than idx.

Arguments

  • srep: Systematic Repair rule of the lift, a member of AbstractSystematicRepair, or nothing. nothing runs every step of mp on the systematic block except the :pdm step, and SystematicRepair runs every step. Under each rule the lift then adds the idiosyncratic block and repairs the sum under the estimator that matrix_processing_posdef gives for mp.
  • mp: Matrix processing estimator.
  • L: The reduced loadings of the latest observation, assets × factors.
  • f_mu: Expected factor returns on the reduced axis.
  • f_sigma: Factor covariance on the reduced axis.
  • esigma: The idiosyncratic variances, or the idiosyncratic covariance.
  • idx: The investable assets.
  • Xs: The asset return scenarios, scenarios × assets, which the processing reads.
  • T: The effective number of observations of f_sigma, which the denoising step reads. It is size(Xs, 1) by default.
  • mtx_sqrt: Square-root algorithm of the factor covariance, and of the idiosyncratic covariance when that block is a full matrix, whose square roots make up chol. The default EigenFallbackSquareRoot takes the Cholesky factor where it exists, and the eigen square root of a singular positive semidefinite matrix otherwise. RidgeCholeskySquareRoot takes the Cholesky factor of the matrix with a small ridge added, and nothing takes the plain Cholesky factor, which raises a LinearAlgebra.PosDefException on a matrix that is not positive definite. matrix_square_root states each algorithm.

Validation

  • L, f_mu and f_sigma agree on the factor axis. Raises a DimensionMismatch.
  • The rules of cross_sectional_ready_factors on f_mu and f_sigma.
  • The block of f_sigma at lf has a square root under mtx_sqrt, as matrix_square_root states. Raises a LinearAlgebra.PosDefException.
  • Each asset of idx reads finite factor moments alone, as cross_sectional_determined states. The function does not check it.

Returns

  • lift::LiftedMoments: The record of three fields.

    • mu::Vector{<:Real}: Expected asset returns, NaN at an asset outside $\mathcal{S}$ and at an asset whose loadings read a NaN factor mean.
    • sigma::Matrix{<:Real}: Asset covariance, NaN at every entry the model does not determine.
    • chol::Matrix{<:Real}: The low-rank square root, NaN in the column of an asset outside $\mathcal{S}$ or of an asset whose loadings read a factor that is not ready, and below the first $K$ rows of an asset outside $\mathcal{I}$.

Related

source